Qwen2.5-32B-Instruct on RX 7900 XTX: Tight fit (Sep 2026)
Conditional memory judgement for one configuration. Change context, quant, or engine and the conclusion can change. Configuration used on this page: GGUF Q4_K_M, 8,192 cached tokens, 1 sequence, engine llama.cpp. Status: Tight. Software: Not verified..
Verdict on Radeon RX 7900 XTX 24GB (desktop)
Only just: Q4_K_M is the one file that gets in. It needs 20.6 GiB–21.5 GiB at 8,192 tokens against 22.0 GiB of usable VRAM (after the driver reserve). Close other GPU apps first, and expect to cut context if the upper estimate is hit.
With Q4_K_M, the longest context that still fits at all (tight, no headroom) is about 13,873 tokens of a 32,768-token native window, one sequence, fp16 KV cache.
| GGUF quant | File weights | KV cache | Estimated total | Verdict |
|---|---|---|---|---|
| Q8_0 | 32.4 GiB | 2.0 GiB | 34.6 GiB–35.7 GiB | Does not fit |
| Q6_K | 25.0 GiB | 2.0 GiB | 27.2 GiB–28.2 GiB | Does not fit |
| Q5_K_M | 21.7 GiB | 2.0 GiB | 23.8 GiB–24.7 GiB | Does not fit |
| Q4_K_M (recommended) | 18.5 GiB | 2.0 GiB | 20.6 GiB–21.5 GiB | Tight |
Engine llama.cpp, one sequence, fp16 KV. File weights are the published GGUF sizes in the catalog. Totals come from the same calculator that powers the tool below; they are estimates, not a measurement of this setup.
Fixed page configuration
GGUF Q4_K_M needs 20.6 GiB-21.5 GiB at 8,192 tokens against 22.00 GiB net on Radeon RX 7900 XTX 24GB (desktop), a tight fit. Q8_0, Q6_K, Q5_K_M are tight or over budget.
- ModelQwen2.5-32B-Instruct
- HardwareRadeon RX 7900 XTX 24GB (desktop)
- QuantizationGGUF Q4_K_M
- Context length (cached tokens)8,192
- Engine
llama.cpp - Concurrent sequences1
- KV cache typefp16
Technical identifiers and share data
- Model
qwen2.5-32b-instruct - Hardware
amd-radeon-rx-7900-xtx-24gb-desktop - Quantization
gguf-q4_k_m - Share fragment for this identity
v2.eyJzY2hlbWFWZXJzaW9uIjoyLCJjYWxjdWxhdG9yVmVyc2lvbiI6IjAuMy4wIiwiZGF0YXNldFZlcnNpb24iOiIyMDI2LTEwLTAyLjEiLCJjb25maWciOnsibW9kZWxJZCI6InF3ZW4yLjUtMzJiLWluc3RydWN0IiwiaGFyZHdhcmVJZCI6ImFtZC1yYWRlb24tcngtNzkwMC14dHgtMjRnYi1kZXNrdG9wIiwiZGV2aWNlQ291bnQiOjEsInF1YW50aXphdGlvbklkIjoiZ2d1Zi1xNF9rX20iLCJjb250ZXh0VG9rZW5zIjo4MTkyLCJjb25jdXJyZW50U2VxdWVuY2VzIjoxLCJlbmdpbmUiOiJsbGFtYS5jcHAiLCJrdlR5cGUiOiJmcDE2In19
Memory breakdown
- Weight artifact
- 18.49 GiB (19,851,336,384 bytes)
- Raw KV cache
- 2.00 GiB (2,147,483,648 bytes)
- Engine extras / workspace
- 0.13 GiB–1.01 GiB
- Total estimated demand
- 20.61 GiB–21.50 GiB
- Net calculator budget
- 22.00 GiB
- OS / display reserve
- 2.00 GiB
- Remaining vs net budget
- 0.50 GiB–1.39 GiB
21.50 GiB high estimate vs 22.00 GiB net
Enough memory does not mean this engine, quant, and OS will run. Support does not mean it will be fast.
Other published files on this device
Same model, device and context (8,192 tokens, 1 sequence, FP16 KV). Each row is one published weight-file set with its usual engine, sorted by memory.
| Quantization | Engine | Estimated memory | Fit |
|---|---|---|---|
| BF16 safetensors | vllm | 63.65 GiB–65.59 GiB | Does not fit |
| GGUF Q8_0 (Qwen) | llama.cpp | 34.55 GiB–35.72 GiB | Does not fit |
| GGUF Q6_K (Qwen) | llama.cpp | 27.16 GiB–28.18 GiB | Does not fit |
| GGUF Q5_K_M (Qwen) | llama.cpp | 23.79 GiB–24.74 GiB | Does not fit |
| GGUF Q4_K_M This page | llama.cpp | 20.61 GiB–21.50 GiB | Tight fit |
Assumptions that affect this result
- Requested context is within the recorded native window of 32768 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Counted files: qwen2.5-32b-instruct-q4_k_m-00001-of-00005.gguf, qwen2.5-32b-instruct-q4_k_m-00002-of-00005.gguf, qwen2.5-32b-instruct-q4_k_m-00003-of-00005.gguf, qwen2.5-32b-instruct-q4_k_m-00004-of-00005.gguf, qwen2.5-32b-instruct-q4_k_m-00005-of-00005.gguf.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
- KV dtype is independent of weight quantization unless you change it.
Related pages
Sources
- Qwen2.5-32B-Instruct model card
- config.json — revision 5ede1c97bbab6ce5cda5812749b4c0bdf79b18dd
- safetensors index — metadata.total_size=65527752704
- Artifact repo Qwen/Qwen2.5-32B-Instruct-GGUF
- AMD Radeon RX 7900 XTX — Max Memory Size 24 GB GDDR6
Calculator
This combo page keeps the route model and device. Customize to change them on the main calculator; the share then keeps that identity.
Loading catalog snapshot…
Search or pick a catalog checkpoint to see memory demand.