The same Qwen2.5-7B-Instruct Q4_K_M file (4,683,073,632 bytes) at 8,192 tokens on a GeForce RTX 4060 8GB is 4.92 to 5.42 GiB for llama.cpp and for Ollama, and 5.42 to 7.31 GiB for the vLLM scenario. The 4060 net budget is 7.00 GiB. llama.cpp and Ollama are likely-fits. vLLM is tight. An unspecified engine is 4.86 to 5.42 GiB, likely-fits. Formula estimates, calculator 0.3.0, dataset 2026-10-02.1, 1 sequence, FP16 KV, typical extras. Software support is unverified. Tokens per second are Unknown.
Weights and raw KV do not change with the engine. The gap is the activation workspace and the engine reserve in src/calc/memory/overhead.ts.
RTX 4060 8GB, Qwen2.5-7B Q4_K_M, 8,192 tokens
KV is 0.44 GiB on every row. Weights are 4.36 GiB.
| Engine | Activation GiB | Engine reserve GiB | Total GiB | Verdict |
|---|---|---|---|---|
| llama.cpp | 0.06 to 0.37 | 0.06 to 0.25 | 4.92 to 5.42 | likely-fits |
| Ollama | 0.06 to 0.37 | 0.06 to 0.25 | 4.92 to 5.42 | likely-fits |
| vLLM | 0.13 to 0.51 | 0.50 to 2.00 | 5.42 to 7.31 | tight |
| unspecified | 0.06 to 0.38 | 0.00 to 0.25 | 4.86 to 5.42 | likely-fits |
Ollama matches llama.cpp because the calculator treats Ollama as a wrapper around that scenario. It does not add a second copy of the weights. The vLLM reserve is a scenario range, not a captured nvidia-smi reading. PagedAttention block padding is not given a separate measured byte count here. The note in the calculator says vLLM often preallocates KV from leftover GPU memory, and this estimate does not report that leftover grab.
RTX 4090, Qwen3-32B Q4_K_M, 8,192 tokens
File Qwen3-32B-Q4_K_M.gguf, 19,762,149,024 bytes. Net budget 22.00 GiB. KV is 2.00 GiB.
| Engine | Total GiB | Verdict |
|---|---|---|
| llama.cpp | 20.53 to 21.41 | tight |
| Ollama | 20.53 to 21.41 | tight |
| vLLM | 21.03 to 22.96 | tight |
The llama.cpp high end stays under 22.00 GiB and above 90% of the budget, so the verdict is tight. The vLLM high end is over 22.00 GiB while the low end is still inside, so that row is also tight. Neither row is a promise that the server starts.
The only runtime-resident measurements in this repository are three llama.cpp runs of Qwen2.5-3B-Instruct Q4_K_M on an RTX 4090. They are on the accuracy page. They are not a stand-in for the 7B or 32B rows above.
Quant choice is independent of this engine gap: Q4 through Q8. A longer context moves KV for every engine: context length VRAM tradeoff.
FAQ
Does Ollama load the weights twice?
Not in this calculator. Ollama uses the llama.cpp overhead scenario. On Qwen2.5-7B Q4_K_M at 8,192 tokens both engines are 4.92 to 5.42 GiB.
Is the vLLM number a measurement?
No. The vLLM high bound is the documented engine-reserve scenario, up to 2.00 GiB, plus a larger activation scenario. It is not a trace of vLLM grabbing leftover GPU memory. Process bytes for these files are Unknown.
Why does the same 32B file go tight on vLLM?
Qwen3-32B Q4_K_M at 8,192 tokens is 20.53 to 21.41 GiB for llama.cpp and 21.03 to 22.96 GiB for vLLM, both on an RTX 4090 with a 22.00 GiB net budget. Both verdicts are tight. The vLLM high end crosses 22.00 GiB.