DeepSeek-R1-Distill-Qwen-32B at Q4_K_M needs 20.6 to 21.5 GiB at 8K context, which is tight on a 24 GB card like the RTX 4090. The 7B and 8B distills need about 5.4 and 6.2 GiB at 8K, so they fit on 8 GB cards. At the full 128K context, the 7B still fits a 24 GB card comfortably, the 8B is tight, and the 32B needs more than twice what a 4090 has.
Context matters more than usual here. Reasoning models write a long chain of thought before the answer, and every one of those tokens sits in the KV cache.
The three distills at 8K, 32K and 128K context
| Model (Q4_K_M) | File size (bytes) | KV at 8K (GiB) | Total at 8K (GiB) | Total at 32K (GiB) | KV at 128K (GiB) | Total at 128K (GiB) | 128K on RTX 4090 |
|---|---|---|---|---|---|---|---|
| R1-Distill-Qwen-7B | 4,683,073,248 | 0.44 | 4.92 to 5.42 | 6.24 to 6.82 | 7.00 | 11.49 to 12.44 | Runs comfortably |
| R1-Distill-Llama-8B | 4,920,737,216 | 1.00 | 5.71 to 6.24 | 8.71 to 9.45 | 16.00 | 20.71 to 22.29 | Tight |
| R1-Distill-Qwen-32B | 19,851,335,584 | 2.00 | 20.61 to 21.50 | 26.61 to 27.92 | 32.00 | 50.61 to 53.00 | Does not fit |
Unsloth GGUF files, one sequence, FP16 KV cache, llama.cpp. 128K (131,072 tokens) is the native context for all three. A 24 GB card has 22 GiB usable after the desktop and driver reserve.
Why the 7B and 8B differ so much at long context: the 7B shares Qwen2.5-7B’s layout (28 layers, 4 KV heads) and costs 56 KiB of KV per token. The 8B shares Llama 3.1 8B’s 128 KiB per token. At 128K that’s 7 GiB versus 16 GiB of cache for models of nearly the same file size.
Which distill for your card
- 8 GB (RTX 4060, 3060 8GB): 7B or 8B at Q4_K_M with a modest context.
- 24 GB (RTX 4090, 3090): the 32B fits only around 8K context, and even then it’s tight. If you want long reasoning traces, the 7B at 128K uses about half the card.
- 32K context on the 32B: already over 24 GB (26.61 GiB at the low end). You need a bigger card, a quantized KV cache, or a shorter context.
The full DeepSeek-R1 and DeepSeek-V3 are separate mixture-of-experts checkpoints with far larger files; nothing in this table applies to them. For dense Qwen3 sizes see Qwen3 VRAM by size, and for how context length moves the total see context length VRAM tradeoff.
Model pages: DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Qwen-32B.
Numbers are estimates from model configs and published file sizes, not measurements; reasoning quality and speed aren’t covered. How we calculate.
FAQ
How much VRAM does DeepSeek-R1-Distill-Qwen-32B need at Q4?
About 20.6 to 21.5 GiB at 8K context with the Unsloth Q4_K_M file. That is tight on a 24 GB card. At 32K it needs 26.61 GiB or more and no longer fits.
Does the 7B distill handle 128K context on a 24 GB card?
Yes. Q4_K_M at 131,072 tokens comes to 11.49 to 12.44 GiB, well inside the 22 GiB a 24 GB card has available. The 8B distill at 128K is tight at 20.71 to 22.29 GiB.
Can I run the 7B or 8B distill on an 8 GB GPU?
At 8K context, yes: the 7B needs 4.92 to 5.42 GiB and the 8B 5.71 to 6.24 GiB at Q4_K_M. Long reasoning traces need more KV cache, so keep the context modest on 8 GB.
Is this the full DeepSeek-R1?
No. These are the dense distilled checkpoints built on Qwen and Llama. The full DeepSeek-R1 is a much larger mixture-of-experts model with its own files and is not covered here.
Numbers from calculator 0.3.0, model data 2026-10-02.1. Sources last checked 2026-10-29.