DeepSeek-V4.1-Flash on 4× NVIDIA DGX Spark (GB10, 128 GB unified) — SGLang TP4 Deployment

4-node DGX Spark cluster running native MXFP4 MoE and FP8 dense weights with SGLang TP=4, verified 1M context needle test, and 8M pinned KV token pool.

  • Cuantización Native MXFP4 MoE + FP8 Dense (TP=4)
  • Repositorio deepseek-ai/DeepSeek-V4.1-Flash
  • Contexto máximo verificado1.048.576
  • Cantidad de equipos4 × NVIDIA DGX Spark (GB10, 128 GB unified)

Dato informado https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks

  • 45.4 tok/s (decode-tok-s)
  • 72.9 tok/s (decode-tok-s)
  • 103.1 tok/s (decode-tok-s)
  • 114.1 tok/s (decode-tok-s)
  • 134.2 tok/s (decode-tok-s)
  • 3350 tok/s (prefill-tok-s)
  • 3782 tok/s (prefill-tok-s)
  • 3768 tok/s (prefill-tok-s)
  • 3531 tok/s (prefill-tok-s)
  • 3251 tok/s (prefill-tok-s)

This deployment recipe is currently maintained in English because commands and technical instructions are source-aligned.

DeepSeek-V4.1-Flash on 4× DGX Spark (1M Long-Context Setup) is a working community deployment for DeepSeek-V4.1-Flash running on 4 × NVIDIA DGX Spark (GB10, 128 GB unified) using SGLang and Native MXFP4 MoE + FP8 Dense (TP=4). Verified up to 1,048,576 tokens context.

Measured by upstream maintainerworking

Tested Setup & Specifications

  • Model CheckpointDeepSeek-V4.1-Flash
  • Hardware SKUNVIDIA DGX Spark (GB10, 128 GB unified)
  • Device Count4 devices
  • InterconnectConnectX-7 RoCE Switch or Switchless Ring
  • Runtime EngineSGLang
  • Patches RequiredCustom kernel patch
  • Artifact FormatSAFETENSORS
  • Active Weights305.00 GiB
  • Tensor ParallelismTP = 4
  • Placement4 nodes via RoCE switch or switchless ring
  • Verified Context1,048,576 tokens
  • Last Verified2026-09-23

Memory & Storage Strategy

This setup relies on tiered storage and offloading strategies to fit larger weights and caches into physical memory:

  • Engram n-gram tables (189 GiB) offloaded to NVME — Packed to local NVMe (~48 GiB/node at TP4) via ./start-tp4.sh pack

Tested & Reported Capabilities

  • ✓Text generation verified
  • ✓Clean model loading & initialisation
  • ✓Context scaling verified up to 1,048,576 tokens
  • ✓Function & Tool calling (Verified)
  • ✓Multimodal vision input (Verified)
  • ✓Multi-Token Prediction / Speculative draft (Verified)

Source-Backed Benchmarks

Measured performance on actual physical machines recorded by the upstream maintainer or community report. No linear extrapolations or simulated numbers.

MetricMeasured ValueContextConcurrencyDetails & Source
Decode Throughput45.4 tok/s—1 streamprose decode (1 stream, 256 tokens)Single stream decode, TTFT 212 ms [source ↗]
Decode Throughput72.9 tok/s—2 streamsprose decode (aggregate)2 streams aggregate (37.9 tok/s per stream), TTFT 232 ms [source ↗]
Decode Throughput103.1 tok/s—4 streamsprose decode (aggregate)4 streams aggregate (26.7 tok/s per stream), TTFT 270 ms [source ↗]
Decode Throughput114.1 tok/s—8 streamsprose decode (aggregate)8 streams aggregate (23.2 tok/s per stream), TTFT 2.70 s [source ↗]
Decode Throughput134.2 tok/s—16 streamsprose decode (aggregate)16 streams aggregate (22.0 tok/s per stream), TTFT 12.45 s [source ↗]
Prefill Throughput3,350 tok/s4,096 tokens1 streamprompt: 4,118 tok4K prompt prefill, TTFT 1.23 s [source ↗]
Prefill Throughput3,782 tok/s16,384 tokens1 streamprompt: 16,400 tok16K prompt prefill, TTFT 4.34 s [source ↗]
Prefill Throughput3,768 tok/s32,768 tokens1 streamprompt: 32,788 tok32K prompt prefill, TTFT 8.70 s [source ↗]
Prefill Throughput3,531 tok/s65,536 tokens1 streamprompt: 65,557 tok64K prompt prefill, TTFT 18.57 s [source ↗]
Prefill Throughput3,251 tok/s131,072 tokens1 streamprompt: 131,093 tok128K prompt prefill, TTFT 40.32 s [source ↗]

Quick Start & Deployment Commands

Condensed deployment procedure extracted directly from the upstream repository.

Prerequisites

  • 4× NVIDIA DGX Spark with RoCE switch or switchless ring interconnect
  • Docker with NVIDIA Container Toolkit
  • ~520 GB NVMe storage on head node

Review these commands, then run them yourself. This page does not run them. This is not the deployment brief.

1Configure TP4 environment
cp -n .env.tp4.example .env.tp4
2Run doctor, build, and share on 4 nodes
./start-tp4.sh doctor && ./start-tp4.sh build && ./start-tp4.sh share
3Pack Engram shards across 4 nodes

Writes engram-l*-r<rank>of4.bin shards (~48 GiB/node) to local NVMe

./start-tp4.sh pack
4Start TP4 service
./start-tp4.sh serve

Caveats & Operational Constraints

Open Calculator With This Exact Setup

Source & Provenance