Free · runs in your browser · no sign-up
LLM VRAM calculator for local models
This LLM VRAM calculator estimates 4.92 GiB–5.42 GiB for Qwen2.5-7B-Instruct Q4_K_M at 8,192 tokens. A VRAM calculator for LLM local inference; demand only until you pick a GPU.
- 86 models
- 32 devices
- 5 deployment recipes
- 6 languages
This LLM VRAM calculator, including a GGUF checkpoint, adds weight bytes from the selected checkpoint, KV cache from that model's attention layout, and runtime overhead. The result is an estimate. See the methodology, the KV cache calculator, the model list, and the GPU list.
GPU and model pages
- RTX 5090 VRAM: 32 GB GDDR7
- RTX 4070 VRAM: 12 GB GDDR6X
- Qwen3-8B
- Qwen2.5-7B-Instruct
- Ling-3.1-flash
- Muse Spark Open
- Can Qwen3-8B run on RTX 4070?
- Can DeepSeek-V4.1-Flash run on DGX Spark (GB10)?
- Can Qwen3-VL-8B-Instruct run on RTX 5090?
- Can Qwen3.8-27B run on RTX 5090?
- Can Qwen3.8-Flash-Next run on DGX Spark (GB10)?
- Can DeepSeek-V4.1-Flash run on RTX 5090?
Device and workload
Pick a GPU or unified-memory machine. Candidate models appear immediately.
Popular hardware
Full-resident weight estimates use total parameters, including inactive MoE experts.
How the estimate works
Read the full methodologyMemory for local inference is the sum of the parts below, compared with the memory your device can actually use.
Weights
Bytes of the selected weight files (GGUF or safetensors), not a parameter-count guess. MoE models count every expert.
KV cache
Grows with context length and concurrent sequences, using each architecture's own attention layout.
Runtime overhead
Engine workspace and buffers, shown as a range for llama.cpp, vLLM, SGLang and other engines.
Usable memory
Device capacity minus an OS and display reserve. Two GPUs are never simply added together.
Proven deployments
View all: Deployment RecipesMaintainer-measured setups. These are exact hardware and runtime recipes, not generic memory estimates.
- ✓ generation
- ✓ 262,144 context
- ✓ tool calling
- ✓ vision
- ✓ speculative decoding
2-node DGX Spark setup serving DeepSeek-V4.1-Flash with vLLM tensor-parallel size 2 over ConnectX-7 RoCE, EXL3 2.9 bpw mul1 weights, and native NVMe-resident Engram tables.
- ✓ generation
- ✓ ~208K Tested (256K cfg)
- ✓ tool calling
- ✓ vision
- ✓ speculative decoding
3-node DGX Spark cluster running native MXFP4 experts and FP8 dense weights with SGLang TP=3 over ConnectX-7 RoCE triangle, with NVMe-resident Engram tables.
- ✓ generation
- ✓ 1,048,576 context
- ✓ tool calling
- ✓ vision
- ✓ speculative decoding
4-node DGX Spark cluster running native MXFP4 MoE and FP8 dense weights with SGLang TP=4, verified 1M context needle test, and 8M pinned KV token pool.
- ✓ generation
- ✓ 262,144 context
- ✓ tool calling
- ✓ speculative decoding
Single NVIDIA RTX 5090 (32 GB, sm_120) deployment serving RadixArk/Qwen3.8-27B-NVFP4 with vLLM 0.27.1, native 256K context, TurboQuant 4-bit KV cache, and MTP-3 speculative decoding.