MoE models and memory: why every expert has to be loaded

Qwen3-30B-A3B computes each token with about 3.3 billion parameters. It still needs 18.16 to 18.93 GiB of memory at 8,192 tokens, because its official Q4_K_M file is 17.28 GiB and all of it has to be loaded. A dense Qwen3-32B in the same format needs 20.53 to 21.41 GiB. The gap of about 2 GiB between them is much smaller than the ten-fold gap in active parameters would suggest. The figures are Formula Estimates from calculator 0.3.0, dataset 2026-10-02.1, with llama.cpp, FP16 KV and 1 sequence.

Total parameters set memory, active parameters set compute

A mixture-of-experts layer holds many feed-forward blocks (experts) and a small router. For each token and each layer, the router picks a few experts, and only those are multiplied. Qwen3-30B-A3B has 128 experts per layer and routes to 8. Mixtral 8x7B has 8 and routes to 2. gpt-oss-20b has 32 and routes to 4, and gpt-oss-120b has 128 and routes to 4.

The router decides again at every token and every layer, and the next token may need any expert. If an expert were not in fast memory, the engine would have to fetch it from somewhere slower in the middle of the forward pass. The repository’s DATA_SOURCES.md therefore records full residency for MoE models, with total parameters used for weights, unless a verified recipe offloads experts.

Sources: total parameters from Hugging Face safetensors.total for each repository, except DeepSeek-V3, where the model card’s 671B is used because the index total also counts multi-token-prediction tensors. Active parameters from the Qwen3 technical report (arXiv 2505.09388), the Mixtral and gpt-oss model cards, and the DeepSeek-V3 model card. File sizes are byte counts of official artifacts: Qwen’s own GGUF repositories, OpenAI’s and Mistral’s safetensors shards, and DeepSeek’s FP8 shards (1,369,062,772,000 bytes). Expert counts come from each config.json.

Same compute class, different memory bill

Three Qwen3 models in official Q4_K_M files, 8,192 tokens:

The 30B-A3B model per token does less arithmetic than the 8B model, and needs about 3.7 times its weight memory. A dense model of 3.3B parameters at the same bits per weight would be roughly 1.87 GiB (17.28 GiB × 3.3 ÷ 30.5). That arithmetic shows how far the file is from the compute-sized figure; it is an illustration, not a catalog value.

The smaller KV cache of 30B-A3B has a different cause. It uses 4 KV heads in 48 layers, against 8 KV heads in 64 layers for Qwen3-32B. The attention layers are separate from the experts, and MoE does not shrink them.

What this means when choosing hardware

• A MoE model buys speed per token, because fewer parameters are multiplied. It does not buy a smaller card. gpt-oss-120b is the clearest example: 5.1B active parameters, 60.77 GiB of weights, and it fits an 80 GB card but no consumer GPU in the catalog.
• Capacity planning uses the file size. Active-parameter counts belong in throughput discussions, which this site does not measure.
• A model whose file fits in unified memory but not in VRAM may run on a machine with enough unified memory, with the speed behavior of that platform. The two GPUs and unified memory guide covers the capacity side.
• Expert offload to system RAM exists in some runtimes. The catalog marks expert offload as “unspecified” for every MoE entry, the calculator never assumes it, and no measurement of its speed cost has been made here.

DeepSeek-V3 is a limit case. Its FP8 files are 1,275.04 GiB, which is far above any single machine in the catalog, and its attention uses multi-head latent attention, which the calculator only partly supports, so this site gives no KV figure for it.

Where to go next

For the formula that turns a file into a total, read how much VRAM a local LLM needs. Per-model architecture and files are on the Qwen3-30B-A3B, gpt-oss-120b and Qwen3-235B-A22B pages. The 24 GB tier numbers appear in what fits on 12, 16 and 24 GB.