Deployment Recipes

Assigned whole experts on each Spark, not a copy of every expert on every rank. Engram rows stay on that node's NVMe and the row cache is off. Native MXFP4 experts and FP8 dense weights.

NVIDIA DGX Spark (GB10, 128 GB unified). Native MXFP4 experts and FP8 dense checkpoint weights, BF16 activations. Snapshot config names quant_method fp8 and expert_dtype fp4.

Four-rank RoCEnante. The pinned README does not qualify a switch versus a ring for this run.

• serving-repo: GNU Affero General Public License v3.0. Commercial use unknown. The rhys LICENSE file is the AGPL-3.0 license text. Its how-to-apply appendix is not a project header electing later versions. This module does not give legal clearance.
• checkpoint: MIT on the Hugging Face card snapshot only. Commercial use unknown. Card license MIT was read on snapshot sha dba1be0a, which is not serving revision df42c109. Confirm the pinned revision's own license file.
• The configured 1M context limit and 4M logical KV pool were not stress-tested at full occupancy.
• Published tok/s figures are the maintainer's own repeat suite. They are not a forecast.
• The lock says its base-image pin is not proof of the image used for Mia's reported results.

Source reported; not reproduced by this site.

Open the planning page

A quarter of every expert on each Spark. Engram rows on local NVMe with a 4 GiB host row cache. Native MXFP4 experts and FP8 dense weights.

NVIDIA DGX Spark (GB10, 128 GB unified). Native MXFP4 experts and FP8 dense weights on the production TP4 line. The 3-Spark profile in the same README is a different setup and is not this pin.

Published TP4 numbers are on a switched RoCE fabric. A switchless ring is a different path and is not this measured line.

• serving-repo: GNU Affero General Public License v3.0. Commercial use unknown. The Mia repository LICENSE file is the AGPL-3.0 license text. Its how-to-apply appendix is not a project header electing later versions. Weights are not in the repository. This module does not give legal clearance.
• checkpoint: MIT on the Hugging Face card snapshot only. Commercial use unknown. Card license MIT was read on snapshot sha dba1be0a, which is not serving revision fb2764a5. Confirm that revision's license file.
• The reported ~77 GiB resident weights per rank are maintainer prose. 476 GiB minus 189 GiB is 287 GiB, not the README's ~305 GiB, and neither quotient is an exact byte ledger.
• A 1,011,084-token needle passed. Sixteen request slots and an 8M-token pin are not a measured 16 by 1,048,576 occupancy.
• The cited validation boot reported a KV pool of 6.0-6.6M tokens, which is not the same integer as the 8,000,000 pin.
• b12x routed MoE is described as numerically close to FlashInfer, not bit-identical.
• The fast loader leaves a smaller and more variable KV pool than the stock loader.
• Boot memory moves with page cache. A KV pin the budget cannot cover can fail the boot or clamp. That warning is not a fit calculation.

Source reported; not reproduced by this site.

Open the planning page

Two Sparks, TensorFold, EXL3 experts in the unified pool. The default drafter is non-commercial DFlash2.

NVIDIA DGX Spark (GB10, 128 GB unified). EXL3 routed experts at 4 bits a weight, BF16 elsewhere in the checkpoint file. Serving default DENSE=q4 re-quantizes those BF16 dense weights. KV default fp8.

Direct ConnectX-7 link between the two Sparks. One or two rails. Not a four-node switch.

• serving-repo: Apache License 2.0. Commercial use allowed. Apache-2.0 covers this recipe repository. It does not relicense the weights or the drafter.
• tensorfold: Apache-2.0 from TensorFold v0.6.0, with an earlier MIT notice for older TensorFold code. Commercial use allowed. The changelog says TensorFold is Apache-2.0 from v0.6.0. That engine license is not the EXL3 weight license.
• exl3-weights: ShapleyMcg License 1.0. Commercial use unknown. The EXL3 checkpoint is ShapleyMcg License 1.0, an attribution license. It is not the MIT license of the TensorFold engine or of the original base-model card.
• dflash2: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0. Commercial use restricted. Default drafter. Non-commercial terms. DRAFTER=mtp avoids this download and serves one request at a time. No commercial recommendation is made for the default stack.
• base-model-card: MIT on the Hugging Face card snapshot of zai-org/GLM-5.3-Flash. Commercial use unknown. Snapshot sha eb9eb208 card says MIT for the base checkpoint. That card is not the EXL3 file and not the DFlash2 file.
• DENSE=q4 lost the end of turn on 12 of 48 short French coding prompts. BF16 or FP8 dense lost it on 1 of 48.
• FP8 KV is lossy. The shared pool measured 2,922,496 tokens at one start. That pool is not 1,048,576 tokens per stream.
• KDA chunking is documented as close to the serial kernel and not bit-identical.
• Two nodes are the validated shape. Four nodes were not validated. Two independent replicas are not twice the throughput.
• The default stack includes the non-commercial DFlash2 drafter.
• Each Spark needs about 110 GiB free when the server starts. The recipe warns below that. Meeting the warning is not a memory fit.

Source reported; not reproduced by this site.

Open the planning page

A separate two-Spark TensorFold kit with C8 admission. Not the Mia image, and not a four-node result.

NVIDIA DGX Spark (GB10, 128 GB unified). EXL3 4-bit TR3 plus the DFlash2 drafter. Compared arms include a different Mia EXL3 kit and a vLLM NVFP4 path. Those arms are not this profile's weights.

CX7 cabled to CX7. Not a four-node fabric.

• kit: Apache-2.0 for jayleaton/glm53-tensorfold-spark. Commercial use allowed. The kit license is not the weight license.
• tensorfold: MIT for ashhart/TensorFold, as credited by LimeChain. Commercial use allowed. Engine credit only. It does not license the EXL3 checkpoint.
• exl3-weights: Checkpoint license must be read on the downloaded repo. Commercial use unknown. LimeChain says to check every weight's license. This cache does not include the checkpoint LICENSE text. Do not assume MIT.
• dflash2: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0. Commercial use restricted. Pinned drafter revision 7d74cdd881ed7e32c31175984a67823127b66cfe. Non-commercial. An empty DRAFTER uses MTP only and is slower. No commercial recommendation.
• Each bake-off arm ran once. Differences under about 5% are called noise by the source.
• C1 short was 99 tok/s on this TensorFold profile, 72 on the vLLM Mia EXL3 kit, and 37 on vLLM NVFP4. The quantizations differ. The 72 is not the Mia TensorFold profile's 60.4 tok/s prose cell.
• C8 short aggregate was 368 versus 251 versus 185 across those same three arms. That is not a same-precision general ranking and not the Mia TensorFold concurrency-4 cell.
• Eight cold ~23K prompts: this profile's slowest first token was 183 s, against 127 s on the vLLM NVFP4 arm. Cold prefill is not a win for this profile in that cell.
• C1 prose and code cells were measured on an earlier kit build 80dba15, not on 5624dfc.
• Not validated at four nodes. Not a double-replica throughput claim.
• The README requires two cabled Sparks and refuses READY until 8 slots load after a page-cache eviction. That gate is not a memory fit.

Source reported; not reproduced by this site.

Open the planning page

One RTX 4090 report: llama.cpp and an Unsloth Q4 main file. This site did not run it.

GeForce RTX 4090 24GB (desktop). Unsloth UD-Q4_K_XL main GGUF, F16 projector, q4_0 KV in the reported llama.cpp setup. Not an official BF16 or FP8 file and not an NVFP4 file.

One local GPU. No multi-node fabric.

• manifest-gguf: Apache-2.0 on the current Unsloth GGUF manifest card. Commercial use unknown. The manifest card says apache-2.0. The measured file's revision was not named, so this is not proof about the bytes that were timed.
• official-checkpoint: Apache-2.0 on the current Qwen/Qwen3.8-27B manifest card. Commercial use unknown. The official card is not the GGUF file used in the report.
• recipe-repo: Unknown. Commercial use unknown. The cached tree at e99d7d05e3e555d556bd8386017a10e27c3f79d8 has the README and setup scripts and no license file. The recipe license stays unknown. It is not inferred from llama.cpp or from the model card.
• One headless Linux report. It is not a Windows result and not a 32 GB Blackwell recipe.
• The report's speed figures are not a forecast and were not reproduced here.
• Old and new quant file sizes differ. Until the measured GGUF revision is known, the manifest bytes can be the wrong generation.
• Built-in MTP still occupies space inside the main file. Speculation off does not delete it.
• The report is one headless Linux process. A display-attached or different CUDA build is outside the pin.

Source reported; not reproduced by this site.

Open the planning page

One DGX Spark, SGLang, NVFP4 weights and a separate drafter. Not a four-node pin.

NVIDIA DGX Spark (GB10, 128 GB unified). RadixArk NVFP4 weights plus an external DSpark drafter. Not the Unsloth GGUF and not llama.cpp.

One Spark. The report's 100Gbps ConnectX-7 note belongs to the multi-node arms, which are not this pin.

• nvfp4-manifest: Apache-2.0 on the current RadixArk NVFP4 manifest card. Commercial use unknown. Current card only. The timed weight revision is unknown.
• dspark: other. Commercial use unknown. The DSpark manifest license field is other. It does not inherit Apache-2.0 from the NVFP4 card or the base model.
• The same report lists 51.8 tok/s at two nodes and 77.3 tok/s at four nodes. Those are different topologies. Four nodes are not four times one node.
• TP2 times data-parallel 2 at C32 was 16.8 tok/s per user in that report, against 11.0 for TP4. Two pairs are not this one-node pin and are not a TP4 group.
• Weight and drafter revisions for the timed run were not printed.

Source reported; not reproduced by this site.

Open the planning page

One Spark and an MLX 4-bit file. The n-gram table can stay on NVMe. Not a two-Spark pin.

NVIDIA DGX Spark (GB10, 128 GB unified). MLX 4-bit group 32 with the built-in MTP, int8 KV, as configured for TensorFold v0.6.1.

One Spark. No TP2 partner.

• mlx-weights: Qwen Community License 1.0 on the current MLX manifest card. Commercial use unknown. The manifest card says qwen-community-1.0. The timed file revision is unknown. This is not a commercial recommendation.
• tensorfold-engine: MIT License. Commercial use allowed. The cached LICENSE at repo 5840c513 is MIT for this recipe and states that TensorFold itself is MIT. This is the code license. It is not the Qwen Community License on the weights and it is not a commercial recommendation.
• Chinese and Japanese prompts need the matching draft vocabulary variant. The current README names that limit.
• Current startup prose is an estimate: about 103 GiB free, a 2 GiB reserve floor, 85.39 GiB for weights plus fixed buffers, and 22.4 GiB for five full windows. None of those are exact byte fields.
• Five streams at 262144 tokens are the configured default. The current table has no five-stream speed cell.
• The measured binary hash is unknown. Manifest file bytes are not that binary and not resident memory.
• Speed figures belong to this patched v0.6.1 recipe only. They are not SGLang speeds and not the GLM two-Spark recipe.
• The current README wants about 103 GiB MemAvailable before start and sets a 2 GiB reserve floor. That warning is not a fit.

Source reported; not reproduced by this site.

Open the planning page

One Spark, SGLang, NVFP4 weights with file-backed PLE. Not the MLX file.

NVIDIA DGX Spark (GB10, 128 GB unified). NVIDIA mixed NVFP4 artifact: NVFP4 experts, FP8 PLE, block-FP8 MTP, and retained high-precision pieces. Not the MLX 4-bit file.

One Spark. Dual-Spark TP2 is outside this pin. That dual arm's borrowed RadixArk BF16 MTP is not this MTP.

• nvfp4-weights: NVIDIA Open Model License, and the card also references Qwen Community License 1.0. Commercial use unknown. Both terms are named on the current manifest card. The timed file revision is unknown. No commercial recommendation is made.
• runtime: Unknown. Commercial use unknown. The SGLang runtime license for commit 4ccff141db was not in the cache. It is not the weight license.
• NVIDIA MTP block 128 does not match TP2's 640/2 = 320 in that report. The dual-Spark recipe borrows a RadixArk BF16 MTP. Do not copy that MTP onto this one-Spark pin.
• 47.7 GiB for the NVMe table is prose. CPU offload of the same table does not free unified memory.
• C8 aggregate 71.7 tok/s is one reported cell, not a per-stream promise and not a dual-Spark number.
• The 6B active label is not the NVFP4 file size.

Source reported; not reproduced by this site.

Open the planning page