Qwen3-VL-8B BF16 needs 19.99 GiB at 8,192 tokens
Qwen3-VL-8B-Instruct BF16 safetensors (Qwen/Qwen3-VL-8B-Instruct) is 17,534,247,392 bytes (16.33 GiB, 17.53 GB). At 8,192 tokens, 1 sequence, FP16 KV, vllm, the calculator estimates 18.08 GiB–19.99 GiB (19,413,295,584–21,462,800,291 bytes). Quantized catalog files: GGUF Q4_K_M (Qwen) is 5,027,784,800 bytes, GGUF Q8_0 (Qwen) is 8,709,519,456 bytes, GGUF Q5_K_M (Unsloth) is 5,851,114,336 bytes, and GGUF Q6_K (Unsloth) is 6,725,901,152 bytes.
- VRAM by quant
- Quantized files on this page
- BF16 on RTX 5090 (32 GB GDDR7)
- Q4_K_M on RTX 4070 (12 GB GDDR6X)
Specific checkpoints with sourced architecture and weight files. GGUF Q4_K_M (Qwen): 5.93 GiB–6.48 GiB at 8,192 tokens, 1 sequence, FP16 KV, llama.cpp.
VRAM by quant at 8,192 tokens
1 sequence, FP16 KV, typical overhead. Each total is the calculator for that catalog file. Optional mmproj bytes stay out of the weight file and out of the total. Links go only to allowlisted can-i-run pages.
| Quantization | Selected files | Bytes | Engine | 8,192 tokens | Optional mmproj | Can I run |
|---|---|---|---|---|---|---|
| GGUF Q4_K_M (Qwen) | Qwen3VL-8B-Instruct-Q4_K_M.gguf | 5,027,784,800 (4.68 GiB) | llama.cpp | 5.93 GiB–6.48 GiB | 1,911,319,552 (1.78 GiB) | |
| GGUF Q5_K_M (Unsloth) | Qwen3-VL-8B-Instruct-Q5_K_M.gguf | 5,851,114,336 (5.45 GiB) | llama.cpp | 6.70 GiB–7.26 GiB | 4,627,174,080 (4.31 GiB) | — |
| GGUF Q6_K (Unsloth) | Qwen3-VL-8B-Instruct-Q6_K.gguf | 6,725,901,152 (6.26 GiB) | llama.cpp | 7.51 GiB–8.09 GiB | 4,627,174,080 (4.31 GiB) | — |
| GGUF Q8_0 (Qwen) | Qwen3VL-8B-Instruct-Q8_0.gguf | 8,709,519,456 (8.11 GiB) | llama.cpp | 9.36 GiB–9.98 GiB | 1,911,319,552 (1.78 GiB) | — |
| BF16 safetensors | 4 files | 17,534,247,392 (16.33 GiB) | vllm | 18.08 GiB–19.99 GiB | — |
Runs on
Publisher and provenance
- PublisherQwen
- Hugging Face
Qwen/Qwen3-VL-8B-Instruct - Revision
0c351dd01ed87e9c1b53cbc748cba10e6187ff3b - Total parameters8,767,123,696
- Layers36
- Query heads32
- KV heads8 (gqa)
- Head dimension128
- Recorded context capability262,144
- Completenesscomplete · full
Memory layers in this estimate
Artifact bytes, raw KV, engine extras, and total demand are separate. This is not a runtime trace.
- Weight artifact
- 4.68 GiB (5,027,784,800 bytes)
- Raw KV cache
- 1.13 GiB (1,207,959,552 bytes)
- Engine extras / workspace
- 0.13 GiB–0.67 GiB
- Total estimated demand
- 5.93 GiB–6.48 GiB
Published artifacts
| Quantization | Selected files | Bytes | Engine | Evidence |
|---|---|---|---|---|
| BF16 safetensors bf16 | model-00001-of-00004.safetensors, model-00002-of-00004.safetensors, model-00003-of-00004.safetensors, model-00004-of-00004.safetensors | 17,534,247,392 (16.33 GiB / 17.53 GB) | vllm, sglang, unspecified | file-metadata |
| GGUF Q4_K_M (Qwen) gguf-q4_k_m | Qwen3VL-8B-Instruct-Q4_K_M.gguf | 5,027,784,800 (4.68 GiB / 5.03 GB) | llama.cpp, ollama, unspecified | file-metadata |
| GGUF Q8_0 (Qwen) gguf-q8_0 | Qwen3VL-8B-Instruct-Q8_0.gguf | 8,709,519,456 (8.11 GiB / 8.71 GB) | llama.cpp, ollama, unspecified | file-metadata |
| GGUF Q5_K_M (Unsloth) gguf-q5_k_m | Qwen3-VL-8B-Instruct-Q5_K_M.gguf | 5,851,114,336 (5.45 GiB / 5.85 GB) | llama.cpp, ollama, unspecified | file-metadata |
| GGUF Q6_K (Unsloth) gguf-q6_k | Qwen3-VL-8B-Instruct-Q6_K.gguf | 6,725,901,152 (6.26 GiB / 6.73 GB) | llama.cpp, ollama, unspecified | file-metadata |
| INT4 theoretical | No published file | 4,383,561,848 (theoretical) | — | theoretical |
Memory by quantization and context
Concurrent sequences = 1. KV defaults to FP16. The range includes documented runtime extras for the listed engine. These figures use the same core as the interactive calculator.
| Quantization | 2,048 tokens | 8,192 tokens | 32,768 tokens |
|---|---|---|---|
| BF16 safetensors | 17.24 GiB–19.12 GiB | 18.08 GiB–19.99 GiB | 21.46 GiB–23.47 GiB |
| GGUF Q4_K_M (Qwen) | 5.09 GiB–5.56 GiB | 5.93 GiB–6.48 GiB | 9.31 GiB–10.09 GiB |
| GGUF Q8_0 (Qwen) | 8.52 GiB–9.06 GiB | 9.36 GiB–9.98 GiB | 12.74 GiB–13.59 GiB |
| GGUF Q5_K_M (Unsloth) | 5.86 GiB–6.35 GiB | 6.70 GiB–7.26 GiB | 10.07 GiB–10.87 GiB |
| GGUF Q6_K (Unsloth) | 6.67 GiB–7.18 GiB | 7.51 GiB–8.09 GiB | 10.89 GiB–11.70 GiB |
| INT4 theoretical Theoretical rows are not downloadable artifacts. | 4.49 GiB–6.16 GiB | 5.33 GiB–7.22 GiB | 8.71 GiB–11.37 GiB |
Raw KV contribution
Standard GQA/MHA raw KV is 2 × layers × kvHeads × headDim × tokens × sequences × bytes per element. This page uses KV heads (8), not query heads (32). Using query heads would overstate this cache by 4×.
- 8,192 tokens, 1 sequence, FP16: 1,207,959,552 bytes (1.1250 GiB)
- 32,768 tokens: 4,831,838,208 bytes (4.5000 GiB)
Related pages
Sources
- Qwen3-VL-8B-Instruct model card
- config.json — revision 0c351dd01ed87e9c1b53cbc748cba10e6187ff3b
- safetensors index — metadata.total_size=17534247392
- Artifact repo Qwen/Qwen3-VL-8B-Instruct-GGUF
- Artifact repo unsloth/Qwen3-VL-8B-Instruct-GGUF
Assumptions that affect this result
- Requested context is within the recorded native window of 262144 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Counted files: Qwen3VL-8B-Instruct-Q4_K_M.gguf.
- This checkpoint has a vision component. Text-only inference does not include vision encoder or projector bytes.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
Calculator
Loading catalog snapshot…
Search or pick a catalog checkpoint to see memory demand.