Can a DGX Spark run Qwen2.5-72B-Instruct?
Yes, with room to spare Qwen2.5-72B-Instruct at Q4_K_M needs 43.61 GiB–44.98 GiB at 8,192 tokens of context. The DGX Spark (GB10) has about 112 GiB usable.
12 files, 44.0 GB in total.
These are estimates from the model's config and file sizes, not measurements. How we calculate
How much room is left
- It still fits at the model's full 32,768-token window.
- A larger file also fits: GGUF Q8_0 (Qwen) needs about 76.8 GiB (Runs comfortably).
Other files on this GPU
| Quant | File size | Engine | Total | Fit |
|---|---|---|---|---|
| BF16 safetensors | 145.4 GB | vllm | 138.6 GiB–140.5 GiB | Does not fit |
| GGUF Q8_0 (Qwen) | 77.5 GB | llama.cpp | 74.8 GiB–76.8 GiB | Runs comfortably |
| GGUF Q6_K (Qwen) | 59.9 GB | llama.cpp | 58.4 GiB–60.0 GiB | Runs comfortably |
| GGUF Q5_K_M (Qwen) | 51.7 GB | llama.cpp | 50.8 GiB–52.3 GiB | Runs comfortably |
| GGUF Q4_K_M This page | 44.0 GB | llama.cpp | 43.6 GiB–45.0 GiB | Runs comfortably |
Same context (8,192 tokens), one chat at a time, each file with its usual engine.
Settings used
- ModelQwen2.5-72B-Instruct
- HardwareNVIDIA DGX Spark (GB10, 128 GB unified)
- QuantizationGGUF Q4_K_M
- Context length (cached tokens)8,192
- Engine
llama.cpp - Concurrent sequences1
- KV cache typefp16
Technical identifiers and share data
- Model
qwen2.5-72b-instruct - Hardware
nvidia-dgx-spark-128gb-unified - Quantization
gguf-q4_k_m - Share fragment for this identity
v2.eyJzY2hlbWFWZXJzaW9uIjoyLCJjYWxjdWxhdG9yVmVyc2lvbiI6IjAuMy4wIiwiZGF0YXNldFZlcnNpb24iOiIyMDI2LTEwLTAyLjEiLCJjb25maWciOnsibW9kZWxJZCI6InF3ZW4yLjUtNzJiLWluc3RydWN0IiwiaGFyZHdhcmVJZCI6Im52aWRpYS1kZ3gtc3BhcmstMTI4Z2ItdW5pZmllZCIsImRldmljZUNvdW50IjoxLCJxdWFudGl6YXRpb25JZCI6ImdndWYtcTRfa19tIiwiY29udGV4dFRva2VucyI6ODE5MiwiY29uY3VycmVudFNlcXVlbmNlcyI6MSwiZW5naW5lIjoibGxhbWEuY3BwIiwia3ZUeXBlIjoiZnAxNiJ9fQ
How this number is calculated
Total = weight file + KV cache + runtime buffers. The range spans low to high runtime overhead for the engine shown.
- Weights (file)
- 40.99 GiB (44,010,465,472 bytes)
- KV cache
- 2.50 GiB (2,684,354,560 bytes)
- Runtime buffers
- 0.13 GiB–1.49 GiB
- Total
- 43.61 GiB–44.98 GiB
- Usable memory
- 112.00 GiB
- OS / display reserve
- 16.00 GiB
- Left over
- 67.02 GiB–68.39 GiB
44.98 GiB high estimate vs 112.00 GiB net
Unified memory is one shared pool. System RAM is not added on top, and several machines are not one pool.
- Requested context is within the recorded native window of 32768 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Weight bytes come from 12 published safetensors shards totaling 44010465472 bytes.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
- KV dtype is independent of weight quantization unless you change it.
Related pages
Sources
- Qwen2.5-72B-Instruct model card
- config.json — revision 495f39366efef23836d0cfae4fbe635880d2be31
- safetensors index — metadata.total_size=145412407296
- Artifact repo Qwen/Qwen2.5-72B-Instruct-GGUF
- NVIDIA DGX Spark product page — 128 GB LPDDR5x coherent unified system memory. This is not dedicated VRAM. Default reserve deducts OS and other processes. Multiple Sparks are not one pool.
- NVIDIA DGX Spark hardware docs — 128 GB LPDDR5x unified memory on the GB10 Superchip.
Calculator
This combo page keeps the route model and device. Customize to change them on the main calculator; the share then keeps that identity.
Loading catalog snapshot…
Pick a model to see how much memory it needs.