Copies text for another assistant or a person to review. Nothing is deployed.
This is not a fit and not a finished deployment. Source reported; not reproduced by this site.
Hardware matches this pin. 2 nodes. TP 2. EP unknown.
Weight revision is unresolved.
As-pinned recipe.
serving-repo: Apache License 2.0. Commercial use allowed.tensorfold: Apache-2.0 from TensorFold v0.6.0, with an earlier MIT notice for older TensorFold code. Commercial use allowed.exl3-weights: ShapleyMcg License 1.0. Commercial use unknown. The EXL3 checkpoint is ShapleyMcg License 1.0, an attribution license. It is not the MIT license of the TensorFold engine or of the original base-model card.dflash2: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0. Commercial use restricted. Default drafter. Non-commercial terms. DRAFTER=mtp avoids this download and serves one request at a time. No commercial recommendation is made for the default stack.base-model-card: MIT on the Hugging Face card snapshot of zai-org/GLM-5.3-Flash. Commercial use unknown. Snapshot sha eb9eb208 card says MIT for the base checkpoint. That card is not the EXL3 file and not the DFlash2 file.DENSE=q4 lost the end of turn on 12 of 48 short French coding prompts. BF16 or FP8 dense lost it on 1 of 48.FP8 KV is lossy. The shared pool measured 2,922,496 tokens at one start. That pool is not 1,048,576 tokens per stream.KDA chunking is documented as close to the serial kernel and not bit-identical.Two nodes are the validated shape. Four nodes were not validated. Two independent replicas are not twice the throughput.The default stack includes the non-commercial DFlash2 drafter.Each Spark needs about 110 GiB free when the server starts. The recipe warns below that. Meeting the warning is not a memory fit.
Do not add these rows together. Main weights overlap subcomponents. CPU and a unified pool are one residence, not extra capacity.
Placements are not applied to the calculator memory verdict.
Logical checkpoint
File ledger. Not a per-rank residence.
File ledger.
File size unknown.
UMA
GPU and CPU share this pool.
Shared UMA pool. Sparse routed matmul.
File size unknown.
Shared UMA pool. Resident matmul.
File size unknown.
Shared UMA pool. Attention cache.
File size unknown.
Shared UMA pool. Resident matmul.
File size unknown.
Shared UMA pool. Runtime scratch.
File size unknown.
Shared UMA pool. Resident matmul.
File size unknown.
Shared UMA pool. Recurrent state.
File size unknown.
Absent: Engram, PLE. Absence is not zero bytes.
Resident bytes: unknown.
Placement
Unified pool
Local disk path
Not a drag-to-fit control.
Other rejected paths
Other residences
Full config and source evidence
- EXL3 checkpoint
The EXL3 file is about 176 GB, which the README equates to 164 GiB. Later starts load about 80 GiB on each Spark. Those are not two different artifacts.Unified pool after startup. About half the file on each Spark. Exact bytes unknown.File size unknown.Per-rank storage: unknown. File ledger.Subcomponents can overlap this main-weight row. Do not add them into the checkpoint.Access: Resident matmul.The README does not print an exact byte size.88.09 GiB is a startup estimate, not a file size.Do not mix 176 GB and 164 GiB as two weights. - Routed experts
EXL3 stores routed experts at 4 bits a weight. They load into the unified pool. The per-rank expert split was not stated.Unified pool. Not an NVMe expert tier.File size unknown.Per-rank storage: unknown. Overlaps the file ledger.Subcomponents can overlap this main-weight row. Do not add them into the checkpoint.Access: Sparse routed matmul.Per-rank expert count was not stated.An 88B EXL3 parameter count was not in the source. - Dense weights
The checkpoint stores dense weights in BF16. The serving default quantizes them again to q4 groups of 64. That q4 path has a documented end-of-turn miss on short French coding prompts.Unified pool. Default DENSE=q4. FP8 or BF16 are different settings with a shorter or equal quality note, not extra memory tiers.File size unknown.Per-rank storage: unknown. Overlaps the file ledger.Subcomponents can overlap this main-weight row. Do not add them into the checkpoint.Access: Resident matmul.Exact dense bytes after q4 were not published. - Engram
This model profile has no Engram table. Dragging an Engram row onto an SSD does not describe a real component.Not present.File size unknown.Per-rank storage: unknown. Not in this recipe.This row can overlap main weights. Do not add it again.Access: Not in this recipe.No Engram bytes exist to report. - KV cache
Four requests share one lossy FP8 pool of 2,922,496 tokens at the measured start. The per-request window is 1,048,576. The pool is not four separate 1M reservations.One shared pool in the unified memory of the serving budget. FP8. Not per stream.File size unknown.Per-rank storage: unknown. Runtime, not weights.This row can overlap main weights. Do not add it again.Access: Attention cache.The 12.5 GiB cap was not converted into a claimed resident allocation.Lowest free memory under a 1M prompt was 5.6 GiB on rank 0 and 9.5 GiB on rank 1, which is not the pool size. - Drafter
The default drafter is IncoAI DFlash2 under CC-BY-NC-ND-4.0. It cannot serve as its own model. This pin uses it only for the four-request mode. MTP-only is the checkpoint head, one request at a time, and it avoids that non-commercial download.DFlash2 is a separate download, about 2.3 GB of prose, loaded with the ranks. MTP, when loaded, is described as 1.77 GiB a Spark. Neither byte figure is exact.File size unknown.Per-rank storage: unknown. Separate from the file ledger.This row can overlap main weights. Do not add it again.Access: Resident matmul.Exact DFlash2 bytes and the revision hash were not in the README.DFlash2 is not licensed like the EXL3 file or the Apache recipe. - Workspace
Vision on rank 0 adds about 1.05 GiB of BF16 tower weights and 0.75 GiB of workspace. The server also wants about 110 GiB free at start. None of these prose figures is a fit.The same unified pool. Vision workspace is on rank 0 only.File size unknown.Per-rank storage: unknown. Runtime, not weights.This row can overlap main weights. Do not add it again.Access: Runtime scratch.110 GiB free is a warning threshold, not measured free memory on your machine.The recipe says the server uses about 10 GiB beyond its own estimate at peak. - PLE
This checkpoint has no Qwen-style n-gram PLE table. Engram, when this model has it, is a different component.Not present.File size unknown.Per-rank storage: unknown. Not in this recipe.This row can overlap main weights. Do not add it again.Access: Not in this recipe.No PLE bytes exist on this profile. - Vision tower
The vision tower is a separate row from the language weights and from the drafter. The README's GiB labels stay prose.Rank 0, inside the same unified pool. Not a second pool and not an SSD tier.File size unknown.Per-rank storage: unknown. Overlaps the file ledger.Subcomponents can overlap this main-weight row. Do not add them into the checkpoint.Access: Resident matmul.The GiB labels were not published as integer byte counts. - Recurrent state
KDA linear-attention state stays in the unified pool. It is not the KV pool and it is not a per-token cache that this module sizes.Unified pool. Fixed recurrent state. Byte size unpublished.File size unknown.Per-rank storage: unknown. Runtime, not weights.This row can overlap main weights. Do not add it again.Access: Recurrent state.No integer byte size was published for the KDA state.
- Pinned source
TP2 settings, disk units, KV pool, licenses 92bf731c3aac61927726ef0c422b21b42111f2c4 - Pinned source
v1.3.2, 53 patches, French 12/48 note, image pin 92bf731c3aac61927726ef0c422b21b42111f2c4 - Pinned source
Apache-2.0 for the recipe repository 92bf731c3aac61927726ef0c422b21b42111f2c4 - Pinned source
Official parameter total and file bytes. Not the EXL3 file. eb9eb208eb0d988989d07a6a12d0fdeb5f52574a
Speed figures in the copied brief are maintainer observations, not a measurement by this site.
Hashes, preflight, and reported observations
Hardware nvidia-dgx-spark-128gb-unified. Pool Each DGX Spark has one unified memory pool. Host RAM is that pool. Two or four nodes are separate pools, not one contiguous capacity. The recipe says 128 GB unified on each Spark. That label is one pool per machine.. Weight pin unset.
Checkpoint Mia-AiLab/GLM-5.3-Flash-EXL3-TR3-4bpw revision unknown. Manifest unknown. Runtime TensorFold v0.6.0. Recipe v1.3.2. Image tag v0.6.0-ae8d1c789b47. v1.3 changelog digest sha256:22789f0cb3dc308f0b2ce52a33961b88bd624af1725e91e8aba0a74a671bb969, and v1.3.2 says that image was unchanged.. Commit v0.6.0.
Concurrency 4 requires DFlash2. MTP is a single request. The window is 1,048,576 tokens per request. The measured shared pool is 2,922,496 tokens, not 1,048,576 per stream. KV_POOL_GIB 12.5 is a named cap, not a measured resident byte count.
Commercial recommendation: false. Memory fit claimed: false. Deployment done: false.
serving-repo: Apache License 2.0. SPDX Apache-2.0. Commercial use allowed. Apache-2.0 covers this recipe repository. It does not relicense the weights or the drafter.tensorfold: Apache-2.0 from TensorFold v0.6.0, with an earlier MIT notice for older TensorFold code. SPDX Apache-2.0. Commercial use allowed. The changelog says TensorFold is Apache-2.0 from v0.6.0. That engine license is not the EXL3 weight license.exl3-weights: ShapleyMcg License 1.0. SPDX LicenseRef-ShapleyMcg-1.0. Commercial use unknown. The EXL3 checkpoint is ShapleyMcg License 1.0, an attribution license. It is not the MIT license of the TensorFold engine or of the original base-model card.dflash2: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0. SPDX CC-BY-NC-ND-4.0. Commercial use restricted. Default drafter. Non-commercial terms. DRAFTER=mtp avoids this download and serves one request at a time. No commercial recommendation is made for the default stack.base-model-card: MIT on the Hugging Face card snapshot of zai-org/GLM-5.3-Flash. SPDX MIT. Commercial use unknown. Snapshot sha eb9eb208 card says MIT for the base checkpoint. That card is not the EXL3 file and not the DFlash2 file.
Official snapshot config has 288 routed experts, 8 per token, and moe intermediate 2048. The TP2 recipe does not say whether each rank holds whole experts or a slice, so full experts per rank stay unknown. The official 288 count is not an EXL3 byte count. Official snapshot eb9eb208 reports 321,323,031,390 parameters and 328,326,771,576 safetensors file bytes for zai-org/GLM-5.3-Flash. An active-parameter count, a packed FP4 parameter count, and an EXL3 parameter count were not printed in the pinned caches, so each stays unknown. The EXL3 recipe's ~176 GB and 164 GiB are one disk quantity in two units, not a second parameter count and not the official file size. Checkpoint file 328326771576 bytes (305.778 GiB).
Preflight
Confirm about 110 GiB free on each Spark before start. The recipe warns below that line. The warning is not a fit.Keep TensorFold v0.6.0, the 53 patches, and recipe v1.3.2. Do not substitute a different TensorFold kit.Read MODEL_REVISION and DFLASH2_REVISION from the pinned tree. The README names the repos and does not print those hashes.Default PARALLEL is 4 only with DRAFTER=dflash2. DRAFTER=mtp forces one request.Decide the drafter license before use. The default download is CC-BY-NC-ND-4.0.Confirm the EXL3 license text on the checkpoint you actually downloaded. ShapleyMcg is not MIT.
Validation
Maintainer reports a 981,841-token needle found, and GSM8K 98.0% on 250 problems plus HumanEval 97.6% on 164 problems, on the lossy FP8 KV and q4 dense path.Maintainer reports replies at concurrency 4 matched the same requests alone in 11 of 11 staggered and 11 of 11 burst cases.This repository did not rerun those checks.
Rollback
stop.sh stops both ranks. DRAFTER=mtp avoids the DFlash2 download and serves one request. DENSE=fp8 or bf16 is the documented response to the French end-of-turn misses. KV=bf16 shortens the window. These are source settings, not a deployment completed by this module.
Sources
- TP2 settings, disk units, KV pool, licenses
mia-glm-readme 92bf731c3aac61927726ef0c422b21b42111f2c4 checked 2026-10-02 - v1.3.2, 53 patches, French 12/48 note, image pin
mia-glm-changelog 92bf731c3aac61927726ef0c422b21b42111f2c4 checked 2026-10-02 - Apache-2.0 for the recipe repository
mia-glm-license 92bf731c3aac61927726ef0c422b21b42111f2c4 checked 2026-10-02 - Official parameter total and file bytes. Not the EXL3 file.
hf-glm-snapshot eb9eb208eb0d988989d07a6a12d0fdeb5f52574a checked 2026-10-02
Reported observations
Maintainer observations, not a measurement by this site, and not a forecast.
Prose, 1 request: 60.4 tok/s. Default config, GPU clocks capped at 2,200 MHz, sparkDash. Not a forecast.Prose, 4 requests: 108.8 tok/s aggregate. Same default config. Per request 27.9 tok/s. Not a forecast and not a 4-node result.
- Do not blend with glm-5.3-flash-lime-tensorfold-c8
EXL3 checkpoint: resident bytes are unknown.Routed experts: resident bytes are unknown.Dense weights: resident bytes are unknown.Engram: resident bytes are unknown.KV cache: resident bytes are unknown.Drafter: resident bytes are unknown.Workspace: resident bytes are unknown.PLE: resident bytes are unknown.Vision tower: resident bytes are unknown.Recurrent state: resident bytes are unknown.- Runtime claimed by this site: false. Summing components creates a fit: false. The generic calculator verdict is unchanged.