At 8K context, a 12 GB card handles models up to about 12B at 4-bit (Gemma 3 12B, or Qwen3-8B at Q6_K). A 16 GB card handles 14B at 4 to 5 bits (Qwen3-14B Q5_K_M). A 24 GB card handles the 24B to 30B class at 4 to 5 bits (Gemma 3 27B, Mistral Small 24B, Qwen3-30B-A3B), with Qwen3-32B at Q4_K_M as a tight fit. Long context (32K) knocks most of these down a step.
The tables below are per tier. Each row is one specific file, so you can see how much room each quant leaves.
How to read the tables
- Usable memory: we subtract what the desktop and driver need. 12 GB cards leave 11 GiB, 16 GB cards 14 GiB (15 GiB on the RTX 4060 Ti and RTX 4070 Ti SUPER), 24 GB cards 22 GiB. Card labels in GB are treated as GiB, since memory chips come in binary sizes. System RAM is never added.
- Verdicts compare the high end of the estimate with usable memory. Runs comfortably is under 70%, Fits is under 90%, Tight is above 90% or crossing the line, Does not fit means even the low end is over.
- Settings: one sequence, FP16 KV cache, llama.cpp. gpt-oss rows use an unspecified engine, because the official MXFP4 checkpoint targets vLLM-class runtimes rather than llama.cpp.
- n/a means the context is longer than the model supports.
12 GB cards: 11 GiB usable
RTX 3060 12GB, RTX 3080 12GB, RTX 4070, RTX 5070. More on the tier: /vram/12gb/.
| Model and file | Weights (GiB) | KV at 8K (GiB) | Total at 8K (GiB) | At 8K | Total at 32K (GiB) | At 32K |
|---|---|---|---|---|---|---|
| Gemma 3 4B IT GGUF Q4_0 | 2.94 | 0.27 | 3.33–3.79 | Runs comfortably | 3.80–4.29 | Runs comfortably |
| Llama 3.2 3B Instruct BF16 | 5.98 | 0.88 | 6.98–7.54 | Runs comfortably | 9.61–10.35 | Tight |
| Qwen3-8B GGUF Q6_K | 6.26 | 1.13 | 7.51–8.09 | Fits | 10.89–11.70 | Tight |
| Qwen3-8B GGUF Q8_0 | 8.11 | 1.13 | 9.36–9.98 | Tight | 12.74–13.59 | Does not fit |
| Gemma 3 12B IT GGUF Q4_0 | 7.52 | 0.81 | 8.46–9.04 | Fits | 9.96–10.64 | Tight |
| Qwen3-14B GGUF Q4_K_M | 8.38 | 1.25 | 9.76–10.39 | Tight | 13.51–14.40 | Does not fit |
| phi-4 GGUF Q4_K_M | 8.28 | 1.56 | 9.97–10.62 | Tight | n/a | n/a |
| Llama 3.1 8B Instruct BF16 | 14.96 | 1.00 | 16.08–16.83 | Does not fit | 19.08–20.04 | Does not fit |
Pick: Gemma 3 12B Q4_0 or Qwen3-8B Q6_K for everyday use at 8K. Qwen3-14B and phi-4 at Q4_K_M fit on paper, but leave well under 1 GiB spare. For 32K context, stay with Gemma 3 4B or accept a tight fit. Meta only publishes Llama 3.1 8B in BF16 (16,060,556,376 bytes), which is too big here; community GGUF conversions exist, but we don’t list them.
10 GB and 8 GB cards. The RTX 3080 10GB has 9 GiB usable: Qwen3-8B Q6_K (up to 8.09 GiB) fits, Q8_0 (from 9.36 GiB) doesn’t. For 8 GB cards such as the RTX 5060 Ti 8GB, see /vram/8gb/.
16 GB cards: 14 GiB usable
RTX 5060 Ti 16GB, RTX 4060 Ti 16GB, RTX 4070 Ti SUPER, RTX 4080, RTX 4080 SUPER, RTX 5080, RX 7800 XT, RX 9070. More on the tier: /vram/16gb/. The table uses 14 GiB; the two cards with 15 GiB usable have one more GiB of room.
| Model and file | Weights (GiB) | KV at 8K (GiB) | Total at 8K (GiB) | At 8K | Total at 32K (GiB) | At 32K |
|---|---|---|---|---|---|---|
| Gemma 3 12B IT GGUF Q4_0 | 7.52 | 0.81 | 8.46–9.04 | Runs comfortably | 9.96–10.64 | Fits |
| Qwen3-8B GGUF Q8_0 | 8.11 | 1.13 | 9.36–9.98 | Fits | 12.74–13.59 | Tight |
| Qwen3-14B GGUF Q4_K_M | 8.38 | 1.25 | 9.76–10.39 | Fits | 13.51–14.40 | Tight |
| Qwen3-14B GGUF Q5_K_M | 9.79 | 1.25 | 11.17–11.83 | Fits | 14.92–15.84 | Does not fit |
| Qwen3-14B GGUF Q6_K | 11.29 | 1.25 | 12.66–13.35 | Tight | 16.41–17.37 | Does not fit |
| phi-4 GGUF Q4_K_M | 8.28 | 1.56 | 9.97–10.62 | Fits | n/a | n/a |
| gpt-oss-20b MXFP4 | 12.82 | 0.19 | 13.07–13.63 | Tight | 13.63–14.19 | Tight |
| Llama 3.1 8B Instruct BF16 | 14.96 | 1.00 | 16.08–16.83 | Does not fit | 19.08–20.04 | Does not fit |
| Gemma 3 27B IT GGUF Q4_0 | 16.05 | 1.03 | 17.20–17.97 | Does not fit | 19.08–19.98 | Does not fit |
Pick: Qwen3-14B at Q4_K_M or Q5_K_M for 8K context; at 32K even Q4_K_M is tight. For long context on 16 GB, Gemma 3 12B holds up best: it only grows from 8.46 to 9.96 GiB between 8K and 32K, while Qwen3-14B grows from 9.76 to 13.51 GiB because every layer keeps the full context. gpt-oss-20b fits, but its 12.82 GiB of weights leave little room; OpenAI’s model card says it runs within 16 GB of memory, which matches. Gemma 3 27B needs 16.05 GiB for weights alone, so it’s out.
24 GB cards: 22 GiB usable
RTX 4090, RTX 3090, RX 7900 XTX. More on the tier: /vram/24gb/.
| Model and file | Weights (GiB) | KV at 8K (GiB) | Total at 8K (GiB) | At 8K | Total at 32K (GiB) | At 32K |
|---|---|---|---|---|---|---|
| Qwen3-14B GGUF Q8_0 | 14.62 | 1.25 | 16.00–16.75 | Fits | 19.75–20.76 | Tight |
| Mistral-Small-24B-Instruct-2501 GGUF Q5_K_M | 15.61 | 1.25 | 16.99–17.76 | Fits | 20.74–21.77 | Tight |
| Mistral-Small-24B-Instruct-2501 GGUF Q6_K | 18.02 | 1.25 | 19.39–20.22 | Tight | 23.14–24.23 | Does not fit |
| gpt-oss-20b MXFP4 | 12.82 | 0.19 | 13.07–13.63 | Runs comfortably | 13.63–14.19 | Runs comfortably |
| Llama 3.1 8B Instruct BF16 | 14.96 | 1.00 | 16.08–16.83 | Fits | 19.08–20.04 | Tight |
| Gemma 3 27B IT GGUF Q4_0 | 16.05 | 1.03 | 17.20–17.97 | Fits | 19.08–19.98 | Tight |
| Qwen3-30B-A3B GGUF Q4_K_M | 17.28 | 0.75 | 18.16–18.93 | Fits | 20.41–21.34 | Tight |
| Qwen3-32B GGUF Q4_K_M | 18.40 | 2.00 | 20.53–21.41 | Tight | 26.53–27.83 | Does not fit |
| Qwen3-32B GGUF Q5_K_M | 21.62 | 2.00 | 23.75–24.69 | Does not fit | 29.75–31.11 | Does not fit |
Pick: Gemma 3 27B Q4_0, Mistral Small 24B Q5_K_M or Qwen3-30B-A3B Q4_K_M at 8K; all three turn tight at 32K. Qwen3-32B Q4_K_M is already tight at 8K because its KV cache is twice Gemma 27B’s (2.00 vs 1.03 GiB). gpt-oss-20b runs comfortably even at 32K. Qwen3-30B-A3B is a mixture-of-experts model: only a few experts work per token, but all of them have to be in memory, which is why its file is 17.28 GiB. See MoE models load every expert and the gpt-oss memory guide.
What these tables don’t cover
- Speed. No tokens-per-second figures; none of these rows was run on hardware.
- Software support. A fit means the bytes fit. Whether your CUDA, ROCm or Metal build loads that format is a separate question.
- Cards between tiers. The RX 7900 XT 20GB and RTX 3080 10GB sit between the tables; subtract their reserve and read the nearest one.
- Context beyond 32K. Use KV cache size by model to extend any row.
If a “Tight” setup still crashes, see VRAM OOM troubleshooting. For one worked 24 GB example, see Qwen2.5-7B on an RTX 4090. The method behind every total is in how much VRAM a local LLM needs.
Other tiers: 8 GB, 32 GB, 48 GB.
Numbers are estimates from model configs and published file sizes, not measurements. How we calculate.
FAQ
What's the biggest model I can run on a 12 GB GPU?
Gemma 3 12B at Q4_0 (8.46 to 9.04 GiB at 8K context) fits with room to spare. Qwen3-14B and phi-4 at Q4_K_M squeeze in at about 10.4 to 10.6 GiB, which is tight against the 11 GiB a 12 GB card has free.
What should I run on a 16 GB GPU?
14B-class models at 4 to 5 bits. Qwen3-14B at Q4_K_M or Q5_K_M fits at 8K context. gpt-oss-20b also fits, but tightly, because its weights alone are 12.82 GiB.
What does 24 GB get me over 16 GB?
The 24B to 32B class. Gemma 3 27B Q4_0, Mistral Small 24B Q5_K_M and Qwen3-30B-A3B Q4_K_M fit at 8K. Qwen3-32B Q4_K_M fits, tightly, and Llama 3.1 8B runs unquantized.
Why do you subtract memory from the card's size?
The desktop and driver use some VRAM before any model loads. We hold back 1 GiB on 12 GB cards and 2 GiB on most 16 and 24 GB cards, so 12, 16 and 24 GB cards leave 11, 14 and 22 GiB for the model.
Numbers from calculator 0.3.0, model data 2026-10-02.1. Sources last checked 2026-10-18.