Phi-4-mini-instruct VRAM & Memory Requirements
Specific checkpoints with sourced architecture and weight files. GGUF Q4_K_M (Unsloth): 3.45 GiB–3.94 GiB at 8,192 tokens, 1 sequence, FP16 KV, llama.cpp.
This result is an estimate, not a measured trace.Not verifiedFile-backed weights
Publisher and provenance
- PublisherMicrosoft
- Hugging Face
microsoft/Phi-4-mini-instruct - Revision
cfbefacb99257ffa30c83adab238a50856ac3083 - Total parameters3,836,021,760
- Layers32
- Query heads24
- KV heads8 (gqa)
- Head dimension128
- Recorded context capability131,072
- Completenesscomplete · full
Memory layers in this estimate
Artifact bytes, raw KV, engine extras, and total demand are separate. This is not a runtime trace.
- Weight artifact
- 2.32 GiB (2,491,874,272 bytes)
- Raw KV cache
- 1.00 GiB (1,073,741,824 bytes)
- Engine extras / workspace
- 0.13 GiB–0.62 GiB
- Total estimated demand
- 3.45 GiB–3.94 GiB
Published artifacts
| Quantization | Selected files | Bytes | Engine | Evidence |
|---|---|---|---|---|
| BF16 safetensors bf16 | model-00001-of-00002.safetensors, model-00002-of-00002.safetensors | 7,672,043,520 (7.15 GiB / 7.67 GB) | vllm, sglang, unspecified | file-metadata |
| GGUF Q4_K_M (Unsloth) gguf-q4_k_m | Phi-4-mini-instruct-Q4_K_M.gguf | 2,491,874,272 (2.32 GiB / 2.49 GB) | llama.cpp, ollama, unspecified | file-metadata |
| GGUF Q5_K_M (Unsloth) gguf-q5_k_m | Phi-4-mini-instruct-Q5_K_M.gguf | 2,848,127,968 (2.65 GiB / 2.85 GB) | llama.cpp, ollama, unspecified | file-metadata |
| GGUF Q6_K (Unsloth) gguf-q6_k | Phi-4-mini-instruct-Q6_K.gguf | 3,155,622,880 (2.94 GiB / 3.16 GB) | llama.cpp, ollama, unspecified | file-metadata |
| GGUF Q8_0 (Unsloth) gguf-q8_0 | Phi-4-mini-instruct.Q8_0.gguf | 4,084,611,040 (3.80 GiB / 4.08 GB) | llama.cpp, ollama, unspecified | file-metadata |
| INT4 theoretical | No published file | 1,918,010,880 (theoretical) | — | theoretical |
Memory by quantization and context
Concurrent sequences = 1. KV defaults to FP16. The range includes documented runtime extras for the listed engine. These figures use the same core as the interactive calculator.
| Quantization | 2,048 tokens | 8,192 tokens | 32,768 tokens |
|---|---|---|---|
| BF16 safetensors | 8.02 GiB–9.90 GiB | 8.77 GiB–10.68 GiB | 11.77 GiB–13.77 GiB |
| GGUF Q4_K_M (Unsloth) | 2.70 GiB–3.12 GiB | 3.45 GiB–3.94 GiB | 6.45 GiB–7.15 GiB |
| GGUF Q5_K_M (Unsloth) | 3.03 GiB–3.46 GiB | 3.78 GiB–4.28 GiB | 6.78 GiB–7.49 GiB |
| GGUF Q6_K (Unsloth) | 3.31 GiB–3.75 GiB | 4.06 GiB–4.57 GiB | 7.06 GiB–7.78 GiB |
| GGUF Q8_0 (Unsloth) | 4.18 GiB–4.64 GiB | 4.93 GiB–5.45 GiB | 7.93 GiB–8.66 GiB |
| INT4 theoretical Theoretical rows are not downloadable artifacts. | 2.16 GiB–3.17 GiB | 2.91 GiB–4.11 GiB | 5.91 GiB–7.80 GiB |
Raw KV contribution
Standard GQA/MHA raw KV is 2 × layers × kvHeads × headDim × tokens × sequences × bytes per element. This page uses KV heads (8), not query heads (24). Using query heads would overstate this cache by 3×.
- 8,192 tokens, 1 sequence, FP16: 1,073,741,824 bytes (1.00 GiB)
- 32,768 tokens: 4,294,967,296 bytes (4.00 GiB)
Related pages
Sources
- Phi-4-mini-instruct model card
- config.json — revision cfbefacb99257ffa30c83adab238a50856ac3083
- safetensors index — metadata.total_size=7672043520
- Artifact repo unsloth/Phi-4-mini-instruct-GGUF
Assumptions that affect this result
- Requested context is within the recorded native window of 131072 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Counted files: Phi-4-mini-instruct-Q4_K_M.gguf.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
- KV dtype is independent of weight quantization unless you change it.
Calculator
Loading catalog snapshot…
Search or pick a catalog checkpoint to see memory demand.
Search or pick a catalog checkpoint to see memory demand.