Qwen2.5-7B-Instruct Q4_K_M on a GeForce RTX 4060 8GB is 4.60 to 5.06 GiB at 2,048 tokens, 4.92 to 5.42 GiB at 8,192 tokens, and 6.24 to 6.82 GiB at 32,768 tokens. Weights stay 4.36 GiB (4,683,073,632 bytes). The catalog card is 8 GiB with a 1 GiB reserve, so the net budget is 7.00 GiB. The 2,048 and 8,192 rows are likely-fits. The 32,768 row is tight because 6.82 GiB is above 90% of 7.00 GiB. Formula estimates, calculator 0.3.0, llama.cpp, 1 sequence, FP16 KV. Software support is unverified. Measured process memory for this file on a 4060 is Unknown.
The formula page is context length and KV cache. This page is the planning matrix: one published file, three lengths, then the cards where the length changes the verdict.
RTX 4060 8GB, Qwen2.5-7B Q4_K_M
| Tokens | KV GiB | Total GiB | Verdict |
|---|---|---|---|
| 2,048 | 0.11 | 4.60 to 5.06 | likely-fits |
| 8,192 | 0.44 | 4.92 to 5.42 | likely-fits |
| 32,768 | 1.75 | 6.24 to 6.82 | tight |
32,768 is the native max_position_embeddings for this checkpoint. The KV ratio 1.75 / 0.44 is 4, which is 32,768 / 8,192.
Qwen3-32B Q4_K_M on 24 GB and 32 GB
Official Qwen/Qwen3-32B-GGUF, file Qwen3-32B-Q4_K_M.gguf, 19,762,149,024 bytes, weights 18.40 GiB. RTX 4090 net budget 22.00 GiB. RTX 5090 net budget 30.00 GiB (32 GiB minus a 2 GiB reserve).
| Tokens | KV GiB | Total GiB | RTX 4090 | RTX 5090 |
|---|---|---|---|---|
| 8,192 | 2.00 | 20.53 to 21.41 | tight | likely-fits |
| 32,768 | 8.00 | 26.53 to 27.83 | insufficient | tight |
| 40,960 | 10.00 | 28.53 to 29.97 | insufficient | tight |
40,960 is this model’s native context. At 32,768 tokens the 4090 row is insufficient because the low end, 26.53 GiB, is over 22.00 GiB. The 5090 row is tight because 27.83 GiB is under 30.00 GiB and above 90% of it. At 40,960 tokens the 5090 high end, 29.97 GiB, is still under 30.00 GiB and still tight.
Llama 3.1 8B BF16 on an RTX 4090
Official BF16 safetensors, 16,060,556,376 bytes. This catalog has no Llama GGUF, so a Q4 file size is Unknown. KV is 128 KiB per token.
| Tokens | KV GiB | Total GiB | Verdict |
|---|---|---|---|
| 8,192 | 1.00 | 16.08 to 16.83 | likely-fits |
| 32,768 | 4.00 | 19.08 to 20.04 | tight |
| 131,072 | 16.00 | 31.08 to 32.88 | insufficient |
The exception
Gemma 3 27B IT Q4_0 goes from 1.03 GiB of KV at 8,192 tokens to 2.91 GiB at 32,768 tokens. That is the sliding-window adapter, written up on Gemma 3 sliding-window VRAM. Per-model curves are on KV cache size by model.
More than one sequence multiplies this KV term. See concurrent sequences. Quantized KV dtypes are a separate switch: FP16 vs Q8_0 vs Q4_0 KV.
FAQ
Does a longer context reload the weights?
No. Qwen2.5-7B-Instruct Q4_K_M stays 4.36 GiB of weights at 2,048, 8,192 and 32,768 tokens. The KV term moves from 0.11 GiB to 0.44 GiB to 1.75 GiB.
Does Qwen3-32B Q4_K_M fit 32,768 tokens on an RTX 4090?
The calculator says insufficient. The total is 26.53 to 27.83 GiB against a 22.00 GiB net budget. The same file at 8,192 tokens is 20.53 to 21.41 GiB, verdict tight. On an RTX 5090 the 32,768-token row is 26.53 to 27.83 GiB against a 30.00 GiB net budget, verdict tight.
Do all models quadruple KV from 8K to 32K?
Standard GQA does. Gemma 3 27B Q4_0 moves from 1.03 GiB to 2.91 GiB of KV over that step because most layers keep a 1,024-token window. See the Gemma 3 sliding-window page.