3-node DGX Spark cluster running native MXFP4 experts and FP8 dense weights with SGLang TP=3 over ConnectX-7 RoCE triangle, with NVMe-resident Engram tables.
- 量子化 Native MXFP4 MoE + FP8 Dense
- リポジトリ deepseek-ai/DeepSeek-V4.1-Flash
- 検証済み文脈208,000
- 台数3 × NVIDIA DGX Spark (GB10, 128 GB unified)
報告値 https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks
- 37.9 tok/s (decode-tok-s)
- 58.9 tok/s (decode-tok-s)
- 71.2 tok/s (decode-tok-s)
- 78.6 tok/s (decode-tok-s)
This deployment recipe is currently maintained in English because commands and technical instructions are source-aligned.
DeepSeek-V4.1-Flash on 3× DGX Spark (SGLang MXFP4 / FP8) is a working community deployment for DeepSeek-V4.1-Flash running on 3 × NVIDIA DGX Spark (GB10, 128 GB unified) using SGLang and Native MXFP4 MoE + FP8 Dense. Configured context ceiling: 262,144 tokens (maintainer-verified prompt envelope: ≈208,000 tokens).
Tested Setup & Specifications
- Model CheckpointDeepSeek-V4.1-Flash
- Hardware SKUNVIDIA DGX Spark (GB10, 128 GB unified)
- Device Count3 devices
- InterconnectConnectX-7 RoCE Triangle
- Runtime EngineSGLang
- Patches RequiredCustom kernel patch
- Artifact FormatSAFETENSORS
- Active Weights305.00 GiB
- Tensor ParallelismTP = 3
- Placement3 nodes in ConnectX-7 triangle topology
- Configured context ceiling262,144 tokens
- Maintainer-verified prompt envelope≈208,000 tokens
- Last Verified2026-09-23
Memory & Storage Strategy
This setup relies on tiered storage and offloading strategies to fit larger weights and caches into physical memory:
- Engram n-gram tables (189 GiB) offloaded to
NVME— Packed to local NVMe (~63 GiB/node) via ./start.sh pack
Tested & Reported Capabilities
- ✓Text generation verified
- ✓Clean model loading & initialisation
- ~Configured context ceiling: 262,144 tokens
- ✓Maintainer-verified prompt envelope: ≈208,000 tokens
- ✓Function & Tool calling (Verified)
- ✓Multimodal vision input (Verified)
- ✓Multi-Token Prediction / Speculative draft (Verified)
Context & Verification Notes:
- 262,144 tokens is the configured per-request context ceiling, not an end-to-end verified prompt length.
- The pinned upstream source explicitly reports single prompts verified to approximately 208K tokens.
- The upstream maintainer documents head-node memory pressure at long context and recommends a substantially smaller practical limit for unattended serving.
Source-Backed Benchmarks
Measured performance on actual physical machines recorded by the upstream maintainer or community report. No linear extrapolations or simulated numbers.
| Metric | Measured Value | Context | Concurrency | Details & Source |
|---|---|---|---|---|
| Decode Throughput | 37.9 tok/s | — | 1 stream | prose decode (1 stream)TTFT 248 ms (~82 ms per speculative step) [source ↗] |
| Decode Throughput | 58.9 tok/s | — | 2 streams | prose decode (aggregate)30.5 tok/s per stream, TTFT 424 ms [source ↗] |
| Decode Throughput | 71.2 tok/s | — | 3 streams | prose decode (aggregate)24.5 tok/s per stream, TTFT 311 ms [source ↗] |
| Decode Throughput | 78.6 tok/s | — | 4 streams | prose decode (aggregate)20.9 tok/s per stream, TTFT 383 ms [source ↗] |
Quick Start & Deployment Commands
Condensed deployment procedure extracted directly from the upstream repository.
Prerequisites
- 3× NVIDIA DGX Spark in ConnectX-7 point-to-point RoCE triangle topology
- Docker with NVIDIA Container Toolkit
- ~520 GB NVMe storage on head node
Review these commands, then run them yourself. This page does not run them. This is not the deployment brief.
cp -n .env.example .env./start.sh doctor./start.sh build./start.sh share./start.sh pack./start.sh serveCaveats & Operational Constraints
Source & Provenance
- Mia's AI Lab: MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks — Commit: 25379c22e8ff177c7587b21b09eac33a92efeb54
- Artifact: deepseek-ai/DeepSeek-V4.1-Flash — Native MXFP4 MoE + FP8 Dense