Congratulations on making it to the end of this short course. In the short course, you tried your hands at different variants of linear quantization methods and implemented them from scratch using PyTorch. You also built a quantizer to quantize any model in eight-bit precision. Finally, you learned about some important challenges when it comes to quantization, such as weights packing. And you implemented the packing and unpacking algorithm by hand. We encourage you to explore other quantization methods available on the Hugging Face transformers. And we hope this course will give you all the tools you need to get started with quantizing any model. And if you find this course helpful, maybe you can even share it with your friends.
Quantization in Depth
- Introduction
- Overview
- Quantize and De-quantize a Tensor
- Get the Scale and Zero Point
- Symmetric vs Asymmetric Mode
- Finer Granularity for more Precision
- Per Channel Quantization
- Per Group Quantization
- Quantizing Weights & Activations for Inference
- Custom Build an 8-Bit Quantizer
- Replace PyTorch layers with Quantized Layers
- Quantize any Open Source PyTorch Model
- Load your Quantized Weights from HuggingFace Hub
- Weights Packing
- Packing 2-bit Weights
- Unpacking 2-Bit Weights
- Beyond Linear Quantization
- Conclusion