Model Quantization (GGUF, AWQ, GPTQ)

Run 70B models on a laptop or 7B models on a phone via 4-bit and 2-bit quantization. Master GGUF, AWQ, GPTQ, and bitsandbytes quantization formats.

Start Skill → View All Lessons

6

Lessons

💻

Code Examples

✅

Production-Ready

100%

Free

Lessons in This Skill

Work through these 6 lessons in order, or jump to whichever topic you need most.

GGUF, AWQ, GPTQ Compared

Intermediate

Post-Training Quantization

Intermediate

Quantization-Aware Training

Advanced

Running Quantized Models with llama.cpp

Intermediate

Measuring Quality Degradation

Intermediate

Mixed Precision Inference

Advanced