Quantization compared: file size and VRAM for ten popular models
One page per model, built from the GGUF file sizes in the catalog. The Q4_K_M to Q8_0 gap runs from about 3.5 GiB on 7B–8B models to 14 GiB on 32B models; the table shows each.
| Model | Q4_K_M | Q8_0 | BF16 | Page |
|---|---|---|---|---|
| Qwen3-8B | 4.7 GiB | 8.1 GiB | 15.3 GiB | Qwen3-8B quantization |
| Qwen3-14B | 8.4 GiB | 14.6 GiB | 27.5 GiB | Qwen3-14B quantization |
| Qwen3-32B | 18.4 GiB | 32.4 GiB | 61.0 GiB | Qwen3-32B quantization |
| Qwen3-30B-A3B | 17.3 GiB | 30.3 GiB | 56.9 GiB | Qwen3-30B-A3B quantization |
| Qwen2.5-7B-Instruct | 4.4 GiB | 7.5 GiB | 14.2 GiB | Qwen2.5-7B-Instruct quantization |
| Qwen2.5-14B-Instruct | 8.4 GiB | 14.6 GiB | 27.5 GiB | Qwen2.5-14B-Instruct quantization |
| Qwen2.5-32B-Instruct | 18.5 GiB | 32.4 GiB | 61.0 GiB | Qwen2.5-32B-Instruct quantization |
| Mistral-7B-Instruct-v0.3 | 4.1 GiB | 7.2 GiB | 13.5 GiB | Mistral-7B-Instruct-v0.3 quantization |
| DeepSeek-R1-Distill-Llama-8B | 4.6 GiB | 8.0 GiB | 15.0 GiB | DeepSeek-R1-Distill-Llama-8B quantization |
| phi-4 | 8.3 GiB | 14.5 GiB | 27.3 GiB | phi-4 quantization |
Sizes are published file bytes in GiB, not peak GPU memory. Each model page lists the Hugging Face repo and revision the bytes came from.