Llama 3.3 70B Instruct VRAM & Memory Requirements
Specific checkpoints with sourced architecture and weight files. Official BF16 safetensors: 134.54 GiB–136.49 GiB at 8,192 tokens, 1 sequence, FP16 KV, vllm.
This result is an estimate, not a measured trace.Not verifiedFile-backed weights
Publisher and provenance
- PublisherMeta
- Hugging Face
meta-llama/Llama-3.3-70B-Instruct - Revision
6f6073b423013f6a7d4d9f39144961bfbfbc386b - Total parameters70,553,706,496
- Layers80
- Query heads64
- KV heads8 (gqa)
- Head dimension128
- Recorded context capability131,072
- Completenesscomplete · full
Memory layers in this estimate
Artifact bytes, raw KV, engine extras, and total demand are separate. This is not a runtime trace.
- Weight artifact
- 131.42 GiB (141,107,497,872 bytes)
- Raw KV cache
- 2.50 GiB (2,684,354,560 bytes)
- Engine extras / workspace
- 0.63 GiB–2.58 GiB
- Total estimated demand
- 134.54 GiB–136.49 GiB
Published artifacts
| Quantization | Selected files | Bytes | Engine | Evidence |
|---|---|---|---|---|
| Official BF16 safetensors bf16 | 141,107,497,872 (131.42 GiB / 141.11 GB) | vllm, sglang, unspecified | file-metadata | |
| INT4 theoretical | No published file | 35,276,853,248 (theoretical) | — | theoretical |
Memory by quantization and context
Concurrent sequences = 1. KV defaults to FP16. The range includes documented runtime extras for the listed engine. These figures use the same core as the interactive calculator.
| Quantization | 2,048 tokens | 8,192 tokens | 32,768 tokens |
|---|---|---|---|
| Official BF16 safetensors | 132.67 GiB–134.56 GiB | 134.54 GiB–136.49 GiB | 142.04 GiB–144.22 GiB |
| INT4 theoretical Theoretical rows are not downloadable artifacts. | 33.60 GiB–43.62 GiB | 35.48 GiB–45.97 GiB | 42.98 GiB–55.20 GiB |
Raw KV contribution
Standard GQA/MHA raw KV is 2 × layers × kvHeads × headDim × tokens × sequences × bytes per element. This page uses KV heads (8), not query heads (64). Using query heads would overstate this cache by 8×.
- 8,192 tokens, 1 sequence, FP16: 2,684,354,560 bytes (2.5000 GiB)
- 32,768 tokens: 10,737,418,240 bytes (10.00 GiB)
Related pages
Sources
- Llama 3.3 70B Instruct model card (Meta) — Context length 128k, GQA, Llama 3.3 Community License. Parameters as printed on the card.
- Meta llama-models sku_list.py (architecture arguments) — arch_args dim=8192, n_layers=80, n_heads=64, n_kv_heads=8. head_dim is dim ÷ n_heads = 128 (Meta's reference model derives it that way; sku_list.py has no head_dim key). The Hugging Face config.json is gated (HTTP 401), so it was not read.
- Hugging Face API — sha 6f6073b423013f6a7d4d9f39144961bfbfbc386b; gated manual; safetensors.total=70553706496; 30 safetensors siblings.
- Cross-check only: GGUF header of a community conversion (not an official source) — Header read with an HTTP range request on 2026-10-18; it agrees with Meta's numbers: block_count 80, head_count 64, head_count_kv 8, embedding_length 8192, context_length 131072, key_length 128. The catalog values above come from Meta, not from this file.
- Hugging Face model page — Gated (manual). Llama 3.3 Community License applies; this page states facts only and does not redistribute the weights.
Assumptions that affect this result
- Requested context is within the recorded native window of 131072 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Weight bytes come from 30 published safetensors shards totaling 141107497872 bytes.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
- Concurrent sequences share one model replica; weights are not multiplied by sequence count.
- KV dtype is independent of weight quantization unless you change it.
Calculator
Loading catalog snapshot…
Search or pick a catalog checkpoint to see memory demand.
Search or pick a catalog checkpoint to see memory demand.