How VRAM Calculator estimates memory
GPU memory for a local LLM is the weight file, plus the KV cache for your context, plus the runtime's own buffers: a few hundred MB on llama.cpp, a few GiB on vLLM. The calculator adds those up and compares the total with what your card can use after the OS and display take their share. Every number is an estimate; the accuracy page shows how it compares with real measurements.
1. Weights: the file you download
We use the byte size of the file you would actually run: a GGUF Q4_K_M file, or the official BF16 safetensors shards. Only the file you pick is counted, not every quant in the repository. When no published file exists, the fallback is the textbook formula, and the result is marked rough:
weights = total parameters × bits per weight ÷ 8Mixture-of-experts models (Qwen3-30B-A3B, gpt-oss, Mixtral) load every expert, so we count total parameters, not the active ones. A vision encoder, a draft model for speculative decoding, or similar extras are added only when you switch them on.
2. KV cache: grows with context length
The KV cache stores attention keys and values for every token in the context. For standard attention (including GQA, which most current models use):
KV bytes = 2 × layers × KV heads × head dim × tokens × sequences × bytes per elementWe use the KV heads from config.json, not the query heads, and an FP16 cache is 2 bytes per element. “Tokens” is the prompt plus the room you reserve for the reply. Several parallel sequences multiply the KV cache but share one copy of the weights.
Some architectures need their own rule. Gemma 3 and gpt-oss keep only a sliding window on part of their layers. Qwen3.8 hybrid models keep a KV cache only on their attention layers and a fixed-size recurrent state on the rest. DeepSeek-style MLA stores a compressed latent. When a field we need is missing from the config, the estimate falls back to the standard formula and is marked rough. It is never set to zero.
3. Runtime overhead
Each engine adds compute buffers and its own reserve. These are the ranges the calculator uses at its “typical” setting:
| Engine | Compute buffers | Engine reserve |
|---|---|---|
| llama.cpp, Ollama | 64 MiB to 256 MiB + 2% of weights and KV; at 8K+ context, another 5% of KV (at most 1 GiB) | 64–256 MiB |
| vLLM | 128 MiB to 512 MiB + 3% of KV | 512 MiB–2 GiB |
| Not specified | 64–384 MiB | 0–256 MiB |
vLLM usually grabs most of the free GPU memory up front for its cache. The estimate reports what the model needs, not that grab, so nvidia-smi will show more under vLLM. The low and high ends of these ranges are scenario bounds, not a statistical confidence interval.
4. Usable memory: capacity minus a reserve
usable = capacity − reserveA card sold as “24 GB” is treated as 24 GiB. The reserve covers the desktop, display and driver: 1 GiB on most 8–12 GB cards and 2 GiB on most 16 GB and larger cards, so a 24 GB RTX 4090 has 22 GiB to work with. Unified-memory machines reserve more because the OS runs from the same pool: 10 GiB on a 64 GB Mac, 12 GiB on a 128 GB Mac, 16 GiB on a DGX Spark. Each GPU page lists its reserve, and you can change it in the calculator.
5. The fit verdict
| Verdict | Rule (high end of the estimate vs usable memory) |
|---|---|
| Comfortable | uses 70% or less |
| Fits | uses 90% or less |
| Tight | over 90%, but the low end still fits |
| Won't fit | even the low end is over |
When part of the estimate rests on a fallback (a missing config field, a context longer than the model's native window, no published file), the high end is raised by 15% once and the result is labelled rough. A memory fit does not prove your runtime supports the format; that is shown separately.
6. Two GPUs, Macs and DGX Spark
Two 24 GB cards are not one 48 GB card. With a tensor-parallel split, each card holds its share of the weights and KV cache, and each card is checked against its own usable memory. If the runtime has no documented way to split the model, the estimate is marked incomplete. The same rule applies to several DGX Spark nodes: their memory is never added into one pool.
Unified memory on a Mac, DGX Spark or Ryzen AI Max is one shared pool for the CPU and GPU. We count it once. It is not VRAM plus system RAM.
Worked example: Qwen2.5-7B-Instruct on a GeForce RTX 4060 8GB
GGUF Q4_K_M, 8,192 tokens of context, one sequence, FP16 KV cache, llama.cpp. The model has 28 layers, 4 KV heads and a head dimension of 128.
KV = 2 × 28 × 4 × 128 × 8,192 × 1 × 2 bytes = 469,762,048 bytes (0.44 GiB)| Part | Memory |
|---|---|
| Weights (file size) | 4.36 GiB |
| KV cache | 0.44 GiB |
| Compute buffers | 0.06 GiB–0.37 GiB |
| Engine reserve | 0.06 GiB–0.25 GiB |
| Total | 4.92 GiB–5.42 GiB |
| Usable on this card | 7.00 GiB |
Verdict: fits. See this pair on its own pageor change any input in the calculator.
What we leave out
- Training, LoRA, image and video generation.
- Tokens per second and price comparisons.
- Offloading part of the model to system RAM. That is a separate calculation we have not shipped yet.
Units: 1 GB = 1,000,000,000 bytes, 1 GiB = 1,073,741,824 bytes. Calculator 0.3.0, data as of 2026-10-02.1.