What LLMs Fit in 24 GB of VRAM? (Oct 2026)

Radeon RX 7900 XTX 24GB (desktop) is the catalog SKU in this 24 GB tier with the smallest net budget: 22.00 GiB after the sourced reserve. 58 catalog models fit at a file-backed artifact. The month follows dataset 2026-10-02.1 (Oct 2026), not the build clock.

Catalog dataset 2026-10-02.1 (Oct 2026).

Catalog SKUs in this tier

SKUMarketed capacityNet budget
Radeon RX 7900 XTX 24GB (desktop)
Smallest net budget
24 GB GDDR622.00 GiB
GeForce RTX 3090 24GB (desktop)24 GB GDDR6X22.00 GiB
GeForce RTX 4090 24GB (desktop)24 GB GDDR6X22.00 GiB

Catalog models that fit

Each row is one file-backed artifact on the representative SKU. A missing quantization is omitted, not replaced. Context search stops at the sourced cap. With no sourced cap, the only verdict shown is at 8192 tokens.

When catalog bandwidth is missing, the decode ceiling says not available. That is not a memory verdict.

ModelArtifactMax context that fitsMax context including tightVerdict at 8192 tokensDecode ceilingLinks
Qwen2.5-0.5B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
0.68 GiB–1.07 GiB
Not available
Qwen3-0.6BGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
40,96040,960
8192
Runs comfortably
1.37 GiB–1.81 GiB
Not available
Gemma 3 1B ITGGUF Q4_0 (Google QAT)
gguf-q4_0 · llama.cpp
32,76832,768
8192
Runs comfortably
1.10 GiB–1.50 GiB
Not available
Llama 3.2 1B InstructOfficial BF16 safetensors
bf16 · vllm
131,072131,072
8192
Runs comfortably
3.18 GiB–5.06 GiB
Not available
Qwen2.5-1.5B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
1.38 GiB–1.80 GiB
Not available
Qwen2.5-Coder-1.5B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
1.38 GiB–1.80 GiB
Not available
SmolLM2-1.7B-InstructGGUF Q4_K_M (Hugging Face)
gguf-q4_k_m · llama.cpp
8,1928,192
8192
Runs comfortably
2.61 GiB–3.11 GiB
Not available
Qwen3-1.7B-BaseGGUF Q4_K_M (mradermacher)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
2.03 GiB–2.49 GiB
Not available
Qwen3-1.7BGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
40,96040,960
8192
Runs comfortably
2.03 GiB–2.49 GiB
Not available
Qwen2.5-3B-InstructGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
2.37 GiB–2.80 GiB
Not available
Qwen2.5-Coder-3B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
2.37 GiB–2.80 GiB
Not available
Llama 3.2 3B InstructOfficial BF16 safetensors
bf16 · vllm
102,855131,072
8192
Runs comfortably
7.48 GiB–9.39 GiB
Not available
Qwen2.5-VL-3B-InstructGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
128,000128,000
8192
Runs comfortably
2.20 GiB–2.63 GiB
Not available
Phi-3.5-mini-instructGGUF Q4_K_M (bartowski)
gguf-q4_k_m · llama.cpp
43,45253,647
8192
Runs comfortably
5.35 GiB–5.98 GiB
Not available
Phi-4-mini-instructGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
129,639131,072
8192
Runs comfortably
3.45 GiB–3.94 GiB
Not available
Qwen3-4BGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
40,96040,960
8192
Runs comfortably
3.58 GiB–4.08 GiB
Not available
Qwen3-4B-Instruct-2507GGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
115,199142,353
8192
Runs comfortably
3.58 GiB–4.08 GiB
Not available
Gemma 3 4B ITGGUF Q4_0 (Google QAT)
gguf-q4_0 · llama.cpp
131,072131,072
8192
Runs comfortably
3.33 GiB–3.79 GiB
Not available
Qwen3-VL-4B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
115,199142,353
8192
Runs comfortably
3.58 GiB–4.08 GiB
Not available
Mistral-7B-Instruct-v0.3GGUF Q4_K_M (bartowski)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
5.20 GiB–5.72 GiB
Not available
DeepSeek-R1-Distill-Qwen-7BGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
131,072131,072
8192
Runs comfortably
4.92 GiB–5.42 GiB
Not available
OpenMath-Nemotron-7BGGUF Q4_K_M (mradermacher)
gguf-q4_k_m · llama.cpp
131,072131,072
8192
Runs comfortably
4.92 GiB–5.42 GiB
Not available
Qwen2-7B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
4.92 GiB–5.42 GiB
Not available
Qwen2.5-7B-InstructGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
4.92 GiB–5.42 GiB
Not available
Qwen2.5-Coder-7BGGUF Q4_K_M (mradermacher)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
4.92 GiB–5.42 GiB
Not available
Qwen2.5-Coder-7B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
4.92 GiB–5.42 GiB
Not available
Qwen2.5-Math-7B-InstructGGUF Q4_K_M (bartowski)
gguf-q4_k_m · llama.cpp
4,0964,096
8192
Runs comfortably · Rough estimate
4.92 GiB–6.23 GiB
Not available
DeepSeek-R1-Distill-Llama-8BGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
111,974131,072
8192
Runs comfortably
5.71 GiB–6.24 GiB
Not available
Llama 3.1 8B InstructOfficial BF16 safetensors
bf16 · vllm
18,63052,571
8192
Fits
16.58 GiB–18.49 GiB
Not available
granite-3.3-8b-instructGGUF Q4_K_M (IBM)
gguf-q4_k_m · llama.cpp
89,450113,191
8192
Runs comfortably
5.98 GiB–6.53 GiB
Not available
Qwen3-8BGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
40,96040,960
8192
Runs comfortably
5.93 GiB–6.48 GiB
Not available
Qwen3-8B-BaseGGUF Q4_K_M (mradermacher)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
5.93 GiB–6.48 GiB
Not available
Qwen2.5-VL-7B-InstructGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
128,000128,000
8192
Runs comfortably
4.92 GiB–5.42 GiB
Not available
Qwen3-VL-8B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
98,840125,192
8192
Runs comfortably
5.93 GiB–6.48 GiB
Not available
GLM-4-9B-0414GGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
6.18 GiB–6.69 GiB
Not available
Gemma 3 12B ITGGUF Q4_0 (Google QAT)
gguf-q4_0 · llama.cpp
131,072131,072
8192
Runs comfortably
8.46 GiB–9.04 GiB
Not available
Mistral-Nemo-Instruct-2407GGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
74,70597,722
8192
Runs comfortably
8.34 GiB–8.94 GiB
Not available
phi-4GGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
16,38416,384
8192
Runs comfortably
9.97 GiB–10.62 GiB
Not available
Phi-4-reasoningGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
10.12 GiB–10.77 GiB
Not available
Qwen3-14BGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
40,96040,960
8192
Runs comfortably
9.76 GiB–10.39 GiB
Not available
OpenMath-Nemotron-14BGGUF Q4_K_M (mradermacher)
gguf-q4_k_m · llama.cpp
54,92873,750
8192
Runs comfortably
10.00 GiB–10.64 GiB
Not available
Qwen2.5-14B-InstructGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
10.00 GiB–10.64 GiB
Not available
Qwen2.5-Coder-14B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Runs comfortably
10.00 GiB–10.64 GiB
Not available
DeepSeek-V2-Lite-ChatGGUF Q4_K_M (mradermacher)
gguf-q4_k_m · llama.cpp
163,840163,840
8192
Runs comfortably · Rough estimate
10.01 GiB–12.23 GiB
Not available
gpt-oss-20bMXFP4 safetensors
mxfp4 · vllm
131,072131,072
8192
Fits
13.63 GiB–15.51 GiB
Not available
Mistral-Small-24B-Instruct-2501GGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
32,76832,768
8192
Fits
14.72 GiB–15.45 GiB
Not available
Gemma 3 27B ITGGUF Q4_0 (Google QAT)
gguf-q4_0 · llama.cpp
30,60071,072
8192
Fits
17.20 GiB–17.97 GiB
Not available
Qwen3.8-27BGGUF UD-IQ1_S (Unsloth)
gguf-ud-iq1_s · llama.cpp
163,484261,490
8192
Runs comfortably · Rough estimate
6.54 GiB–8.14 GiB
Not available
Qwen3-30B-A3BGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
17,06840,960
8192
Fits
18.16 GiB–18.93 GiB
Not available
Qwen3-30B-A3B-Instruct-2507GGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
17,06850,164
8192
Fits
18.16 GiB–18.93 GiB
Not available
Qwen3-VL-30B-A3B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
17,06850,164
8192
Fits
18.16 GiB–18.93 GiB
Not available
Qwen3-32BGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
2,11614,213
8192
Tight fit
20.53 GiB–21.41 GiB
Not available
DeepSeek-R1-Distill-Qwen-32BGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
1,77513,873
8192
Tight fit
20.61 GiB–21.50 GiB
Not available
OpenMath-Nemotron-32BGGUF Q4_K_M (mradermacher)
gguf-q4_k_m · llama.cpp
1,77513,873
8192
Tight fit
20.61 GiB–21.50 GiB
Not available
Qwen2.5-32B-InstructGGUF Q4_K_M
gguf-q4_k_m · llama.cpp
1,77513,873
8192
Tight fit
20.61 GiB–21.50 GiB
Not available
Qwen2.5-Coder-32B-InstructGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
1,77513,873
8192
Tight fit
20.61 GiB–21.50 GiB
Not available
QwQ-32BGGUF Q4_K_M (Qwen)
gguf-q4_k_m · llama.cpp
1,77513,873
8192
Tight fit
20.61 GiB–21.50 GiB
Not available
Qwen2.5-VL-32B-InstructGGUF Q4_K_M (Unsloth)
gguf-q4_k_m · llama.cpp
1,77513,873
8192
Tight fit
20.61 GiB–21.50 GiB
Not available

Allowlisted combos