Which local LLMs fit on an RTX 4070 (12 GB GDDR6X)?
The RTX 4070 has 12 GB GDDR6X of discrete VRAM. After a 1 GiB reserve for the OS and display, about 11 GiB is usable for a model. At an 8K context the largest model in our list that fits is Mistral-Nemo-Instruct-2407 at Q4_K_M (about 8.9 GiB). Qwen3.8-27B at UD-IQ3_XXS is tight (about 11 GiB–13.3 GiB).
These are estimates from the model's config and file sizes, not measurements. How we calculate
Which models fit at 8K
Each model uses the highest-precision file that fits, one chat at a time.
Fits (36)
| Model | Quant | Total |
|---|---|---|
| DeepSeek-R1-Distill-Llama-8B | GGUF Q8_0 (Unsloth) | 9.1 GiB–9.7 GiB |
| Gemma 3 12B IT | GGUF Q4_0 (Google QAT) | 8.5 GiB–9.0 GiB |
| Mistral-Nemo-Instruct-2407 | GGUF Q4_K_M (Unsloth) | 8.3 GiB–8.9 GiB |
| Mistral-7B-Instruct-v0.3 | GGUF Q8_0 (bartowski) | 8.3 GiB–8.9 GiB |
| GLM-4-9B-0414 | GGUF Q6_K (Unsloth) | 8.1 GiB–8.7 GiB |
| Qwen2.5-Coder-7B | GGUF Q8_0 (mradermacher) | 8.1 GiB–8.7 GiB |
| Qwen2.5-Coder-7B-Instruct | GGUF Q8_0 (Qwen) | 8.1 GiB–8.7 GiB |
| Qwen2.5-7B-Instruct | GGUF Q8_0 | 8.1 GiB–8.7 GiB |
| DeepSeek-R1-Distill-Qwen-7B | GGUF Q8_0 (Unsloth) | 8.1 GiB–8.7 GiB |
| OpenMath-Nemotron-7B | GGUF Q8_0 (mradermacher) | 8.1 GiB–8.7 GiB |
| Qwen2.5-VL-7B-Instruct | GGUF Q8_0 (Unsloth) | 8.1 GiB–8.7 GiB |
| Qwen2-7B-Instruct | GGUF Q8_0 (Qwen) | 8.1 GiB–8.7 GiB |
| granite-3.3-8b-instruct | GGUF Q6_K (IBM) | 7.6 GiB–8.2 GiB |
| Qwen3-VL-8B-Instruct | GGUF Q6_K (Unsloth) | 7.5 GiB–8.1 GiB |
| Qwen3-8B-Base | GGUF Q6_K (mradermacher) | 7.5 GiB–8.1 GiB |
| Qwen3-8B | GGUF Q6_K (Qwen) | 7.5 GiB–8.1 GiB |
| Qwen2.5-Math-7B-Instruct | GGUF Q6_K (bartowski) | 6.4 GiB–7.9 GiB |
| Phi-3.5-mini-instruct | GGUF Q8_0 (bartowski) | 6.9 GiB–7.6 GiB |
| Llama 3.2 3B Instruct | Official BF16 safetensors | 7.0 GiB–7.5 GiB |
| Qwen3-VL-4B-Instruct | GGUF Q8_0 (Qwen) | 5.2 GiB–5.8 GiB |
| Qwen3-4B-Instruct-2507 | GGUF Q8_0 (Unsloth) | 5.2 GiB–5.8 GiB |
| Qwen3-4B | GGUF Q8_0 | 5.2 GiB–5.8 GiB |
| Phi-4-mini-instruct | GGUF Q8_0 (Unsloth) | 4.9 GiB–5.5 GiB |
| Qwen2.5-Coder-3B-Instruct | GGUF Q8_0 (Qwen) | 3.8 GiB–4.2 GiB |
| Qwen2.5-3B-Instruct | GGUF Q8_0 | 3.8 GiB–4.2 GiB |
| Qwen2.5-VL-3B-Instruct | GGUF Q8_0 (Unsloth) | 3.5 GiB–3.9 GiB |
| SmolLM2-1.7B-Instruct | GGUF Q8_0 (Unsloth) | 3.3 GiB–3.8 GiB |
| Gemma 3 4B IT | GGUF Q4_0 (Google QAT) | 3.3 GiB–3.8 GiB |
| Qwen3-1.7B-Base | GGUF Q8_0 (mradermacher) | 2.7 GiB–3.2 GiB |
| Qwen3-1.7B | GGUF Q8_0 | 2.7 GiB–3.2 GiB |
| Llama 3.2 1B Instruct | Official BF16 safetensors | 2.7 GiB–3.1 GiB |
| Qwen2.5-Coder-1.5B-Instruct | GGUF Q8_0 (Qwen) | 2.1 GiB–2.5 GiB |
| Qwen2.5-1.5B-Instruct | GGUF Q8_0 (Qwen) | 2.1 GiB–2.5 GiB |
| Qwen3-0.6B | GGUF Q8_0 (Qwen) | 1.6 GiB–2.0 GiB |
| Gemma 3 1B IT | GGUF Q4_0 (Google QAT) | 1.1 GiB–1.5 GiB |
| Qwen2.5-0.5B-Instruct | GGUF Q8_0 (Qwen) | 0.8 GiB–1.2 GiB |
Tight fit (8)
| Model | Quant | Total |
|---|---|---|
| Qwen3.8-27B | GGUF UD-IQ3_XXS (Unsloth) | 11.0 GiB–13.3 GiB |
| DeepSeek-V2-Lite-Chat | GGUF Q4_K_M (mradermacher) | 10.0 GiB–12.2 GiB |
| Phi-4-reasoning | GGUF Q4_K_M (Unsloth) | 10.1 GiB–10.8 GiB |
| Qwen2.5-14B-Instruct | GGUF Q4_K_M | 10.0 GiB–10.6 GiB |
| Qwen2.5-Coder-14B-Instruct | GGUF Q4_K_M (Qwen) | 10.0 GiB–10.6 GiB |
| OpenMath-Nemotron-14B | GGUF Q4_K_M (mradermacher) | 10.0 GiB–10.6 GiB |
| phi-4 | GGUF Q4_K_M (Unsloth) | 10.0 GiB–10.6 GiB |
| Qwen3-14B | GGUF Q4_K_M | 9.8 GiB–10.4 GiB |
Won't fit (42)
- gpt-oss-20b · MXFP4 · 13.8 GiB
- Mistral-Small-24B-Instruct-2501 · Q4_K_M · 15.5 GiB
- Llama 3.1 8B Instruct · BF16 · 16.8 GiB
- Gemma 3 27B IT · Q4_0 · 18 GiB
- Qwen3-30B-A3B · Q4_K_M · 18.9 GiB
- Qwen3-30B-A3B-Instruct-2507 · Q4_K_M · 18.9 GiB
- Qwen3-VL-30B-A3B-Instruct · Q4_K_M · 18.9 GiB
- Qwen3-32B · Q4_K_M · 21.4 GiB
- OpenMath-Nemotron-32B · Q4_K_M · 21.5 GiB
- Qwen2.5-VL-32B-Instruct · Q4_K_M · 21.5 GiB
- DeepSeek-R1-Distill-Qwen-32B · Q4_K_M · 21.5 GiB
- QwQ-32B · Q4_K_M · 21.5 GiB
- Qwen2.5-Coder-32B-Instruct · Q4_K_M · 21.5 GiB
- Qwen2.5-32B-Instruct · Q4_K_M · 21.5 GiB
- Drama 3 · Unofficial int4 W4A16 · 27 GiB
- Mixtral-8x7B-Instruct-v0.1 · Q4_K_M · 28.6 GiB
- Llama-3.1-Nemotron-70B-Instruct-HF · Q4_K_M · 43.6 GiB
- Qwen2.5-72B-Instruct · Q4_K_M · 45 GiB
- Qwen3-Next-80B-A3B-Instruct · Q4_K_M · 47.3 GiB
- gpt-oss-120b · MXFP4 · 62.8 GiB
- GLM-4.5-Air · Q4_K_M · 71.4 GiB
- Qwen3.8-Flash-Next · Q4_K_M · 131.5 GiB
- MiniMax-M2 · Q4_K_M · 134 GiB
- Qwen3-235B-A22B · Q4_K_M · 137.1 GiB
- Qwen3-235B-A22B-Instruct-2507 · Q4_K_M · 137.1 GiB
- Llama 3.3 70B Instruct · BF16 · 137.2 GiB
- DeepSeek-V4-Flash-0731 · FP8 · 183.7 GiB
- GLM-4.6 · Q4_K_M · 208.4 GiB
- GLM-4.5 · Q4_K_M · 209.2 GiB
- MiniMax-M3 · Q4_K_M · 249.7 GiB
- GLM-5.3-Flash · Q8_0 · 373.6 GiB
- DeepSeek-V3 · Q4_K_M · 443.1 GiB
- DeepSeek-R1 · Q4_K_M · 443.1 GiB
- DeepSeek-V3-0324 · Q4_K_M · 443.1 GiB
- DeepSeek-R1-0528 · Q4_K_M · 443.7 GiB
- DeepSeek-V3.1 · Q4_K_M · 444.2 GiB
- Muse Spark Open · NVFP4 · 508.9 GiB
- DeepSeek-V4.1-Flash · FP8 · 558.8 GiB
- Ling-3.1-flash · FP8 · 628.3 GiB
- Kimi-K2-Instruct · Q4_K_M · 679.5 GiB
- Kimi-K2-Thinking · Q4_K_M · 680 GiB
- GLM-5.3 · Q8_0 · 877 GiB
Detailed checks
Memory
- NVIDIAGeForce RTX 4070 12GB desktop
- Memory kindDiscrete VRAM
- Marketed12 GB GDDR6X
- Gross12.00 GiB (12,884,901,888 bytes)
- Default reserve1.00 GiB (1,073,741,824 bytes)
- Net budget11.00 GiB (11,811,160,064 bytes)
The reserve is an adjustable assumption. On a headless machine you can lower it in the calculator.
Two of these cards do not add up to one bigger card. Each copy of a model has to fit on one card unless the engine splits it with tensor parallelism.
Related pages
Sources
- NVIDIA GeForce RTX 40 Series specs — Original 12 GB GDDR6X SKU, not the later GDDR6 variant
Calculator
Loading catalog snapshot…
Pick a GPU or Mac to see which models fit, or pick a model to see how much memory it needs.
Pick a GPU to see which models fit.