Qwen3-8B official GGUF Q4_K_M is 5.93 to 6.48 GiB at 8,192 tokens. Qwen3-32B is 20.53 to 21.41 GiB. Qwen3-0.6B is 1.37 to 1.81 GiB, and its KV (0.88 GiB) is larger than its weights (0.37 GiB). Formula estimates, calculator 0.3.0, dataset 2026-10-02.1, llama.cpp, 1 sequence, FP16 KV, typical extras. No GPU is attached to this table, so the status is demand-only. Software support is unverified. Tokens per second are Unknown. MoE Qwen3 models are a different memory story and are not in this table.
Q4_K_M at two contexts
| Model | Repo | File bytes | Weights GiB | Total at 8,192 | KV at 8,192 | Total at 32,768 | KV at 32,768 |
|---|---|---|---|---|---|---|---|
| Qwen3-0.6B | unsloth/Qwen3-0.6B-GGUF | 396,705,472 | 0.37 | 1.37 to 1.81 | 0.88 | 3.99 to 4.62 | 3.50 |
| Qwen3-1.7B | unsloth/Qwen3-1.7B-GGUF | 1,107,409,472 | 1.03 | 2.03 to 2.49 | 0.88 | 4.66 to 5.30 | 3.50 |
| Qwen3-4B | Qwen/Qwen3-4B-GGUF | 2,497,280,256 | 2.33 | 3.58 to 4.08 | 1.13 | 6.95 to 7.69 | 4.50 |
| Qwen3-8B | Qwen/Qwen3-8B-GGUF | 5,027,783,488 | 4.68 | 5.93 to 6.48 | 1.13 | 9.31 to 10.09 | 4.50 |
| Qwen3-14B | Qwen/Qwen3-14B-GGUF | 9,001,752,960 | 8.38 | 9.76 to 10.39 | 1.25 | 13.51 to 14.40 | 5.00 |
| Qwen3-32B | Qwen/Qwen3-32B-GGUF | 19,762,149,024 | 18.40 | 20.53 to 21.41 | 2.00 | 26.53 to 27.83 | 8.00 |
From 8,192 to 32,768 tokens every KV column quadruples. These six checkpoints use standard GQA, so the all-layer formula applies. Hybrid Qwen3.8 does not: see hybrid attention KV.
Qwen3-8B and Qwen3-4B share a 1.13 GiB KV at 8,192 tokens (36 layers, 8 KV heads, head dim 128) even though the 8B file is about twice the 4B file. Qwen3-0.6B and Qwen3-1.7B share 0.88 GiB of KV. Memory planning that only sorts by parameter count misses that.
On an RTX 4090 the 32B file at 8,192 tokens is tight (20.53 to 21.41 GiB against 22.00 GiB net) and at 32,768 tokens it is insufficient. That pair is on context length VRAM tradeoff. Q5_K_M and Q6_K for the 8B are on Q4 vs Q5 vs Q6 vs Q8.
Model pages: Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen3-32B.
FAQ
How much VRAM does Qwen3-8B Q4_K_M need?
5.93 to 6.48 GiB at 8,192 tokens and 9.31 to 10.09 GiB at 32,768 tokens, llama.cpp, FP16 KV, 1 sequence. The official file is Qwen3-8B-Q4_K_M.gguf, 5,027,783,488 bytes.
Why is Qwen3-0.6B mostly cache at 8K?
The Q4_K_M file is 396,705,472 bytes (0.37 GiB). FP16 KV at 8,192 tokens is 0.88 GiB, so the total is 1.37 to 1.81 GiB. At 32,768 tokens KV is 3.50 GiB and the total is 3.99 to 4.62 GiB.
Are the 0.6B and 1.7B files from Qwen?
In this catalog they are Unsloth GGUF files: unsloth/Qwen3-0.6B-GGUF and unsloth/Qwen3-1.7B-GGUF. The 4B, 8B, 14B and 32B rows are Qwen's own GGUF repos. Publishers are not swapped.