Can a DGX Spark run Qwen2.5-32B-Instruct?
Yes, with room to spare Qwen2.5-32B-Instruct at BF16 needs 63.65 GiB–65.59 GiB at 8,192 tokens of context. The DGX Spark (GB10) has about 112 GiB usable.
17 files, 65.5 GB in total.
These are estimates from the model's config and file sizes, not measurements. How we calculate
How much room is left
- It still fits at the model's full 32,768-token window.
Other files on this GPU
| Quant | File size | Engine | Total | Fit |
|---|---|---|---|---|
| BF16 safetensors This page | 65.5 GB | vllm | 63.7 GiB–65.6 GiB | Runs comfortably |
| GGUF Q8_0 (Qwen) | 34.8 GB | llama.cpp | 34.6 GiB–35.7 GiB | Runs comfortably |
| GGUF Q6_K (Qwen) | 26.9 GB | llama.cpp | 27.2 GiB–28.2 GiB | Runs comfortably |
| GGUF Q5_K_M (Qwen) | 23.3 GB | llama.cpp | 23.8 GiB–24.7 GiB | Runs comfortably |
| GGUF Q4_K_M | 19.9 GB | llama.cpp | 20.6 GiB–21.5 GiB | Runs comfortably |
Same context (8,192 tokens), one chat at a time, each file with its usual engine.
Settings used
- ModelQwen2.5-32B-Instruct
- HardwareNVIDIA DGX Spark (GB10, 128 GB unified)
- QuantizationBF16 safetensors
- Context length (cached tokens)8,192
- Engine
vllm - Concurrent sequences1
- KV cache typefp16
Technical identifiers and share data
- Model
qwen2.5-32b-instruct - Hardware
nvidia-dgx-spark-128gb-unified - Quantization
bf16 - Share fragment for this identity
v2.eyJzY2hlbWFWZXJzaW9uIjoyLCJjYWxjdWxhdG9yVmVyc2lvbiI6IjAuMy4wIiwiZGF0YXNldFZlcnNpb24iOiIyMDI2LTEwLTAyLjEiLCJjb25maWciOnsibW9kZWxJZCI6InF3ZW4yLjUtMzJiLWluc3RydWN0IiwiaGFyZHdhcmVJZCI6Im52aWRpYS1kZ3gtc3BhcmstMTI4Z2ItdW5pZmllZCIsImRldmljZUNvdW50IjoxLCJxdWFudGl6YXRpb25JZCI6ImJmMTYiLCJjb250ZXh0VG9rZW5zIjo4MTkyLCJjb25jdXJyZW50U2VxdWVuY2VzIjoxLCJlbmdpbmUiOiJ2bGxtIiwia3ZUeXBlIjoiZnAxNiJ9fQ
How this number is calculated
Total = weight file + KV cache + runtime buffers. The range spans low to high runtime overhead for the engine shown.
- Weights (file)
- 61.03 GiB (65,527,752,704 bytes)
- KV cache
- 2.00 GiB (2,147,483,648 bytes)
- Runtime buffers
- 0.63 GiB–2.56 GiB
- Total
- 63.65 GiB–65.59 GiB
- Usable memory
- 112.00 GiB
- OS / display reserve
- 16.00 GiB
- Left over
- 46.41 GiB–48.35 GiB
65.59 GiB high estimate vs 112.00 GiB net
Unified memory is one shared pool. System RAM is not added on top, and several machines are not one pool.
- Requested context is within the recorded native window of 32768 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Weight bytes come from 17 published safetensors shards totaling 65527752704 bytes.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
- KV dtype is independent of weight quantization unless you change it.
Related pages
Sources
- Qwen2.5-32B-Instruct model card
- config.json — revision 5ede1c97bbab6ce5cda5812749b4c0bdf79b18dd
- safetensors index — metadata.total_size=65527752704
- Artifact repo Qwen/Qwen2.5-32B-Instruct-GGUF
- NVIDIA DGX Spark product page — 128 GB LPDDR5x coherent unified system memory. This is not dedicated VRAM. Default reserve deducts OS and other processes. Multiple Sparks are not one pool.
- NVIDIA DGX Spark hardware docs — 128 GB LPDDR5x unified memory on the GB10 Superchip.
Calculator
This combo page keeps the route model and device. Customize to change them on the main calculator; the share then keeps that identity.
Loading catalog snapshot…
Pick a model to see how much memory it needs.