LLM VRAM calculator for local models

Pick your GPU and a model to see if it fits, with weights, KV cache and runtime overhead included.

Your GPU or Mac

Model

Context length

Longer context needs more memory for the KV cache, the model's working memory for the conversation.

More options

Quantity

QuantizationGGUF Q8_0Recommended8.10 GiB – 8.66 GiBRuns comfortablyBF16 safetensors14.75 GiB – 15.44 GiBFitsGGUF Q6_K (Qwen)6.39 GiB – 6.91 GiBRuns comfortablyGGUF Q5_K_M5.63 GiB – 6.14 GiBRuns comfortably

GGUF Q8_0 · Auto · gguf-q8_0

Checkpoint: Qwen/Qwen2.5-7B-Instruct-GGUF

Show historical models

Runtime

Engine and KV cache type for this estimate.

EngineKV cache typeRecurrent / conv state dtype Include theoretical estimates (expert) Allow a substitute if the filter has no match

Workload

Overhead scenario and the prefill workspace.

Overhead scenario Include heavier prefill workspace

Topology

How the model is split across devices. Memory is not added together.

DeploymentIntra-replica splitDevice count

Replica count is an advanced deployment option. Quantity is nodes or GPUs, not replicas.

Replica groupsDevices per replica

Budget

System reserve and safety margin. An empty number keeps the previous value.

System / display reserve

GiB (2^30 bytes)

Budget already excludes system reserveExtra safety margin

Custom bits per weightDisplay unit

Custom model

Import a config or type the architecture. Nothing is scanned until you import.

Import local config.json

Parsed in the browser. The file is not uploaded.

Public Hugging Face repoRevision

Only huggingface.co, config.json, no token. Private repos are rejected.

Total parametersActive parametersLayersQuery headsKV headsHead dimension

Fits

Qwen2.5-7B-Instruct · on RTX 4090 24GB · 8K context

Needed

8.66 GiB

Usable on RTX 4090 24GB

22.00 GiB

Estimated range: 8.10 GiB to 8.66 GiB.

Weights 7.54 GiB
Context (KV cache) 0.44 GiB
Overhead 0.13 GiB
Free 13.34 GiB

File: GGUF Q8_0 · picked automatically · Qwen/Qwen2.5-7B-Instruct-GGUF

Try this

There is room for 32K context: about 10.07 GiB needed.

Runtime: unverified

Details

File bytes

Checkpoint: Qwen/Qwen2.5-7B-Instruct

Software compatibility

Runtime	Status	Notes
llama.cpp	Not verified	Enough memory does not mean this engine, quant, and OS will run. Support does not mean it will be fast.
Ollama (llama.cpp-based)	Not verified	Enough memory does not mean this engine, quant, and OS will run. Support does not mean it will be fast.
vLLM	Not verified	Not verified
SGLang	Not verified	Not verified
MLX	Not applicable on this hardware	Not verified
ExLlamaV3	Not verified	Not verified

Evidence and limits

Memory headroom does not prove this engine, quantization, and OS combination is supported.
The artifact format lists this engine as a possible reader. That is not a verified run of this checkpoint, engine version, hardware, and OS.

Assumptions that affect this result

Requested context is within the recorded native window of 32768 tokens.
Full-resident weight estimates use total parameters, including inactive MoE experts.
Weight bytes come from the selected weight files only, not from every file in the repository.
Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
Counted files: qwen2.5-7b-instruct-q8_0-00001-of-00003.gguf, qwen2.5-7b-instruct-q8_0-00002-of-00003.gguf, qwen2.5-7b-instruct-q8_0-00003-of-00003.gguf.
Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
Concurrent sequences share one model replica; weights are not multiplied by sequence count.
KV dtype is independent of weight quantization unless you change it.
Using 4 KV heads, not 28 query heads.
Prefill peak and steady decode occupancy are not the same. The high scenario is closer to prefill/workspace pressure.
Engine reserve is not the same as bytes the model actually uses for weights and KV.
llama.cpp compute buffers grow with context and batch. Values here are user-adjustable scenarios, not measured traces — unless a matching calibration is shown.
Host RAM is not added to discrete GPU VRAM.
Automatic quantization selected gguf-q8_0. That choice is shown in the result.

Dataset 2026-10-02.1 · calculator 0.3.0

Numbers are estimates from the model config and file sizes, not measurements. Calculation method

How the estimate works

Each result adds up the parts below and compares the total with the memory your GPU or Mac can actually use. The numbers are estimates, not measurements: the methodology shows the math, and the KV cache calculator works out context memory on its own.

Weights
The weight bytes from the selected checkpoint (GGUF or safetensors files), not a guess from the parameter count. MoE models count every expert.
KV cache
Grows with context length and with how many requests run at once. Each model's attention layout sets the rate.
Runtime overhead
Buffers and workspace the inference engine needs. Shown as a range, since llama.cpp, vLLM and others differ.
Usable memory
What your GPU or Mac can really use after the OS and display take their share. Memory on two GPUs is not simply added together.