Ling-3.1-flash local LLM VRAM requirements
Local LLM VRAM requirements for Ling-3.1-flash (Ant Group / inclusionAI): the official mixed-FP8 checkpoint is 573,753,281,152 bytes (534.35 GiB, 573.75 GB). Hugging Face lists 572,120,680,768 parameters; the model card says 560B total and 25B active per token. Our formula estimates 535.04 GiB–616.06 GiB at 8,192 tokens, one stream. The official serving recipes are 8 × H200 (TP8+EP8) or 4 × GB300 (TP4+EP4). These are estimates, not measurements.
Ling-3.1-flash GPU memory: official files versus our estimate
inclusionAI's model card (fetched 2026-10-16) publishes a hybrid-linear MoE of about 560B total / 25B active with a 1M-token window cap, and serving recipes for 8 × H200 (TP8+EP8) and 4 × GB300 (TP4+EP4). The official mixed-FP8 checkpoint's shard index is 573,753,281,152 bytes (534.35 GiB / 573.75 GB). Our calculator estimates 535.04 GiB–616.06 GiB at 8,192 tokens, one stream. Neither number is a measurement of peak usage on a specific GPU.
| Device | Usable | Official FP8 (534.35 GiB) |
|---|---|---|
| GeForce RTX 4090 24GB (desktop) 24 GB GDDR6X | 22.0 GiB | Does not fit 535.04 GiB–616.06 GiB |
| GeForce RTX 5090 32GB (desktop) 32 GB GDDR7 | 30.0 GiB | Does not fit 535.04 GiB–616.06 GiB |
| NVIDIA L40S 48GB 48GB GDDR6 with ECC | 46.0 GiB | Does not fit 535.04 GiB–616.06 GiB |
| NVIDIA A100 80GB SXM 80GB HBM2e | 78.0 GiB | Does not fit 535.04 GiB–616.06 GiB |
| NVIDIA H100 80GB SXM 80GB | 78.0 GiB | Does not fit 535.04 GiB–616.06 GiB |
| NVIDIA DGX Spark (GB10, 128 GB unified) 128 GB LPDDR5x coherent unified system memory | 112.0 GiB | Does not fit 535.04 GiB–616.06 GiB |
| Apple M4 Max 128GB unified (MacBook Pro) 128 GB unified memory | 116.0 GiB | Does not fit 535.04 GiB–616.06 GiB |
"Does not fit" is a formula verdict for one unsplit replica. Ant's 8-GPU and 4-GPU recipes are publisher serving plans. This catalog has no H200, GB300, or B300 SKU.
Memory by context (official FP8)
| Checkpoint | 2,048 tokens | 8,192 tokens | 32,768 tokens |
|---|---|---|---|
| Official FP8 safetensors (F8_E4M3 experts + BF16 remainder) | 534.57 GiB–615.43 GiB | 535.04 GiB–616.06 GiB | 536.94 GiB–618.57 GiB |
Verified facts
- ModelLing-3.1-flash,
inclusionAI/Ling-3.1-flash - PublisherAnt Group inclusionAI. Weights open 2026-10-13/14 (HF createdAt 2026-10-13; lastModified 2026-10-15).
- LicenseMIT
- ParametersCard: 560B total, 25B active/token. HF safetensors.total: 572,120,680,768 (F8_E4M3 570,628,505,600 + BF16 1,491,619,712 + F32 555,456)
- ArchitectureHybrid-linear MoE: 63 KDA + 9 gated MLA (7:1), 4 dense layers, 512 routed experts, 8 activated, 1 shared, 64 heads, hidden 4096
- config.json72 layers, head_dim 128, max_position_embeddings 262,144, kv_lora_rank 512, q_lora_rank 1536, MTP layers=1
- Files54 safetensors shards, index total_size 573,753,281,152 bytes
- Official serving8 × H200 TP8+EP8; 4 × GB300 TP4+EP4 (model card). vLLM recipe uses tensor-parallel-size 8
- Unverified hereCommunity GGUF sizes, MXFP4 packed serving (~292 GB in an Ant social post), 1M-token KV bytes, any peak-memory trace
How this page estimates a hybrid MoE
- Weights are the official shard-index bytes (534.35 GiB). All 512 experts are resident. The 25B active count is compute, not memory.
- KV / state is partial. Nine gated MLA layers have published LoRA ranks; 63 KDA linear-attention layers are not a GQA cache. The calculator marks coverage partial and does not invent a KDA state size.
- Context in config.json is 262,144 tokens. The 1M figure is the card/docs window cap, not the config native length.
- Multi-GPU is not additive VRAM. 8 × H200 is a tensor/expert-parallel plan, not 8 × 141 GB pooled.
Ling-3.1-flash FAQ
How much VRAM does Ling-3.1-flash need?
The official FP8 shard index is 573,753,281,152 bytes (534.35 GiB). Our calculator estimates 535.04 GiB–616.06 GiB at 8,192 tokens, one stream, SGLang typical extras. The model card lists 8 × H200 or 4 × GB300. No one at vramcalculator.io has measured peak memory on those GPUs.
Can Ling-3.1-flash run on one RTX 4090, RTX 5090, or a Mac?
No, not as a single unsplit replica of the official checkpoint. The weight files alone are 534.35 GiB. Catalog 24 GB / 32 GB / 80 GB / 128 GB SKUs do not hold a full resident copy. Two Mac Studios are not in Ant's published Ling-3.1-flash recipes.
Is 560B or 572B the parameter count?
The model card says about 560 billion total and 25 billion active per token. Hugging Face safetensors metadata on 2026-10-16 lists 572,120,680,768 parameters (mostly F8_E4M3). This catalog stores the safetensors total for weight math and the card's 25B as active compute. All experts stay resident unless a recipe offloads them.
Does the 1M context change VRAM?
config.json sets max_position_embeddings to 262,144. The model card and Ant docs also mention a 1M-token window cap. KV/state coverage here is partial (hybrid KDA + gated MLA), so long-context figures are not a full cache model. Weights dominate.
Is there an official INT4 or GGUF build?
Not in inclusionAI/Ling-3.1-flash. The INT4 row in the catalog is theoretical arithmetic, labelled as such. Community GGUF files were not added.
Publisher and provenance
- PublisherAnt Group / inclusionAI
- Hugging Face
inclusionAI/Ling-3.1-flash - Revision
a4ccd6b45c002ddb236f9cb77d52cc427482bf19 - Total parameters572,120,680,768
- Active parameters25,000,000,000
- Layers72
- Query heads64
- KV heads64 (hybrid)
- Head dimension128
- Recorded context capability262,144
- Completenesspartial · partial
Active parameters describe tokens' routed compute, not the full resident model weight footprint. Full-resident weight estimates use total parameters.
Memory layers in this estimate
Artifact bytes, raw KV, engine extras, and total demand are separate. This is not a runtime trace.
- Weight artifact
- 534.35 GiB (573,753,281,152 bytes)
- Raw KV cache
- 0.63 GiB (679,477,248 bytes)
- Engine extras / workspace
- 0.63 GiB–83.26 GiB
- Total estimated demand
- 535.61 GiB–618.24 GiB
Published artifacts
| Quantization | Selected files | Bytes | Engine | Evidence |
|---|---|---|---|---|
| Official FP8 safetensors (F8_E4M3 experts + BF16 remainder) fp8 | 573,753,281,152 (534.35 GiB / 573.75 GB) | sglang, vllm, unspecified | file-metadata | |
| INT4 theoretical | No published file | 286,060,340,384 (theoretical) | — | theoretical |
Memory by quantization and context
Concurrent sequences = 1. KV defaults to FP16. The range includes documented runtime extras for the listed engine. These figures use the same core as the interactive calculator.
| Quantization | 2,048 tokens | 8,192 tokens | 32,768 tokens |
|---|---|---|---|
| Official FP8 safetensors (F8_E4M3 experts + BF16 remainder) | 535.13 GiB–617.59 GiB | 535.61 GiB–618.24 GiB | 537.51 GiB–620.82 GiB |
| INT4 theoretical Theoretical rows are not downloadable artifacts. | 266.70 GiB–343.93 GiB | 267.17 GiB–344.61 GiB | 269.07 GiB–347.30 GiB |
Raw KV contribution
Related pages
Sources
- Ling-3.1-flash model card — MIT license. Card: 560B total / 25B active, 1M-token context, hybrid-linear MoE, 63 KDA + 9 Gated MLA (7:1), 4 dense layers, 512 routed experts, 8 activated, 1 shared, 64 heads, hidden 4096. Official serving: 8× H200 TP8+EP8; 4× GB300 TP4+EP4.
- config.json — revision a4ccd6b45c002ddb236f9cb77d52cc427482bf19; model_type bailing_hybrid; num_hidden_layers 72; num_attention_heads 64; num_key_value_heads 64; head_dim 128; hidden_size 4096; max_position_embeddings 262144; num_experts 512; num_experts_per_tok 8; num_shared_experts 1; kv_lora_rank 512; q_lora_rank 1536; num_nextn_predict_layers 1; quantization_config fp8 e4m3 block 128×128.
- safetensors index — metadata.total_size=573753281152
- Hugging Face API — sha a4ccd6b45c002ddb236f9cb77d52cc427482bf19; gated false; safetensors.total=572120680768; sibling safetensors sum 573,781,078,249 bytes.
- SGLang Ling-3.1-flash cookbook — Linked from the official model card for H200, GB300, H800, H100, and B200 recipes.
- Ant Ling developer docs — Table: 560B parameters, 25B activated, 256K served context, 1M window cap.
Assumptions that affect this result
- Requested context is within the recorded native window of 262144 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- This MoE lists 572120680768 total parameters and 25000000000 active parameters. Active-parameter compute is not used as the weight-residency figure.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Weight bytes come from 54 published safetensors shards totaling 573753281152 bytes.
- Multi-token prediction / draft weights exist and are not included unless enabled.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
Calculator
Loading catalog snapshot…
Search or pick a catalog checkpoint to see memory demand.