Can Qwen2.5-72B-Instruct run on NVIDIA DGX Spark (GB10, 128 GB unified)?
Conditional memory judgement for one configuration. Change context, quant, or engine and the conclusion can change.
72B Q4 on unified memory after a realistic system reserve.
Configuration used on this page: gguf-q4_k_m, 8,192 cached tokens, 1 sequence, engine llama.cpp.
Status: Likely fits. Software: Documented support.
Weights 40.99 GiB · raw KV 2.50 GiB · total 43.61 GiB–44.98 GiB.
Available budget 112.00 GiB after the default reserve.
This page is an estimate unless a matching runtime-resident measurement is attached.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Counted files: qwen2.5-72b-instruct-q4_k_m-00001-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00002-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00003-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00004-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00005-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00006-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00007-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00008-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00009-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00010-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00011-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00012-of-00012.gguf.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
- KV dtype is independent of weight quantization unless you change it.
- Using 8 KV heads, not 64 query heads.
Related: Qwen2.5-72B-Instruct· NVIDIA DGX Spark (GB10, 128 GB unified)
Estimate
Likely fits
Under the current assumptions, the high estimate stays below 90% of the available budget.
Estimated memory range: 43.61 GiB – 44.98 GiB
Weights use selected file metadata. Runtime extras are still estimated.
Available budget: 112.00 GiB · Remaining after estimate: 67.02 GiB – 68.39 GiB
Software compatibility
Documented support. Enough memory does not mean this engine, quant, and OS will run. Support does not mean it will be fast.
Assumptions that affect this result
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Counted files: qwen2.5-72b-instruct-q4_k_m-00001-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00002-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00003-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00004-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00005-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00006-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00007-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00008-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00009-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00010-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00011-of-00012.gguf, qwen2.5-72b-instruct-q4_k_m-00012-of-00012.gguf.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
- KV dtype is independent of weight quantization unless you change it.
- Using 8 KV heads, not 64 query heads.
- Prefill peak and steady decode occupancy are not the same. The high scenario is closer to prefill/workspace pressure.
- Engine reserve is not the same as bytes the model actually uses for weights and KV.
- llama.cpp compute buffers grow with context and batch. Values here are user-adjustable scenarios, not measured traces — unless a matching calibration is shown.
- Unified memory is a shared CPU/GPU pool. Host RAM is not added on top of this capacity.