gpt-oss-20b and gpt-oss-120b memory: file size, KV cache and which GPUs fit

gpt-oss-20b needs 13.07 to 13.63 GiB at 8,192 tokens, and gpt-oss-120b needs 61.12 to 61.68 GiB. Both are Formula Estimates from calculator 0.3.0, dataset 2026-10-02.1, for one sequence with an FP16 KV cache. Neither model has been run on hardware for this site. The estimates start from official byte counts, and the architecture values come from OpenAI’s config.json files.

What is in the files

Byte counts are the sums of the official safetensors shards (3 files for the 20B model and 15 for the 120B model) at the revisions recorded in the model pages. The quantization_config in both configs applies MXFP4 to the mixture-of-experts weights and keeps attention, the router, token embeddings and the output head out of the quantized set. That is why the effective size is 5.26 bits per parameter for the 20B model and 4.47 for the 120B model (bytes × 8 ÷ parameters), above the nominal 4.25 bits of MXFP4.

The weights stay resident in full even though only 3.6B or 5.1B parameters are active per token. Why that is has its own guide.

KV cache with a 128-token window

Both configs list sliding_window: 128 and a layer_types list that alternates sliding and full attention. Half of the layers therefore cache the whole context, and the other half cache at most 128 tokens. With 8 KV heads of width 64, one global layer costs 2,048 bytes per token in FP16.

If every layer kept the full length, gpt-oss-20b would need 6.00 GiB at 131,072 tokens instead of 3.00 GiB. The calculator assumes the engine stores the sliding layers at window length. That assumption has not been checked on a real engine, and the KV cache guide lists the full-length alternative for other models.

Fit by device at 8,192 tokens

Net budget is capacity minus the catalog’s default reserve. Verdicts use the “unspecified” engine, with typical extras.

Reading the table:

• gpt-oss-20b needs a 16 GB class card at minimum. On the RTX 5060 Ti 16GB the high end of 13.63 GiB leaves 0.37 GiB below the 14 GiB net budget, so the verdict is tight. OpenAI’s model card says the 20B model runs within 16 GB of memory, and the estimate agrees with that statement while showing how little slack remains with a desktop attached.
• On a 24 GB card, longer prompts still fit. At 32,768 tokens the RTX 4090 estimate is 13.63 to 14.19 GiB (comfortable), and at 131,072 tokens it is 15.88 to 16.44 GiB (fits).
• gpt-oss-120b fits an 80 GB card with room for long contexts. The H100 and A100 estimates are 61.96 to 62.52 GiB at 32,768 tokens and 65.33 to 65.90 GiB at 131,072 tokens, both inside 78 GiB net. OpenAI’s model card also states that the model runs on a single 80 GB GPU such as an H100 or MI300X.
• No consumer card in the catalog holds the 120B model. The RTX 5090 has 30 GiB net, and the L40S 46 GiB. The DGX Spark leaves 112 GiB and has more than enough capacity.

Capacity is only half of “will it run”

The catalog lists gpt-oss with the engines vllm and unspecified, and no GGUF row exists for either model, so there is no llama.cpp verdict. MXFP4 needs kernels in the inference engine and support from the GPU generation. The verdicts above say that the bytes fit; they do not say that a given engine on a given card loads the format. Check the model card and the engine’s release notes for your hardware.

Speed is not covered. The calculator estimates memory only, and no tokens-per-second figure for either model has been measured here.

Model records

Open the gpt-oss-20b and gpt-oss-120b pages for the file hashes and revisions, or the RTX 4090 page and H100 page for the net budgets used above. For which 12, 16 and 24 GB cards suit other models, read what fits on 12, 16 and 24 GB.