Best GPU for local LLM: start from the model's memory
For local inference the first filter is memory. Qwen3-8B runs cleanly from 9 GiB net, and gpt-oss-120b needs 78 GiB net. Net is capacity minus the catalog's default reserve (a 24 GB card is 22 GiB, an 80 GB card is 78 GiB). The tables below give the smallest budget for ten models and the models each budget covers, at 8,192 tokens with official file sizes. They rank by memory only: speed and price are not measured or sourced here.
Smallest memory budget per model
| Model | Smallest budget | File at that budget | Cards at that budget | Comfortable file from |
|---|---|---|---|---|
| Qwen3-8B | 9 GiB | Q6_K | GeForce RTX 3080 10GB | 30 GiB (BF16) |
| Qwen3-14B | 14 GiB | Q5_K_M | GeForce RTX 4080 16GB, GeForce RTX 4080 SUPER 16GB, GeForce RTX 5060 Ti 16GB, GeForce RTX 5070 Ti 16GB, GeForce RTX 5080 16GB, Radeon RX 7800 XT 16GB, Radeon RX 9070 16GB, Radeon RX 9070 XT 16GB | 46 GiB (BF16) |
| gpt-oss-20b | 18 GiB | MXFP4 | Radeon RX 7900 XT 20GB | 30 GiB (MXFP4) |
| Mistral-Small-24B-Instruct-2501 | 18 GiB | Q4_K_M | Radeon RX 7900 XT 20GB | 78 GiB (BF16) |
| Llama 3.1 8B Instruct | 22 GiB | BF16 | GeForce RTX 3090 24GB, GeForce RTX 4090 24GB, Radeon RX 7900 XTX 24GB | 30 GiB (BF16) |
| Gemma 3 27B IT | 22 GiB | Q4_0 | GeForce RTX 3090 24GB, GeForce RTX 4090 24GB, Radeon RX 7900 XTX 24GB | 96 GiB (BF16) |
| Qwen3-30B-A3B | 22 GiB | Q4_K_M | GeForce RTX 3090 24GB, GeForce RTX 4090 24GB, Radeon RX 7900 XTX 24GB | 96 GiB (BF16) |
| Qwen3-32B | 30 GiB | Q5_K_M | GeForce RTX 5090 32GB | 96 GiB (BF16) |
| Qwen2.5-72B-Instruct | 54 GiB | Q4_K_M | Apple M2 Max 64GB unified (MacBook Pro / Mac Studio) | no device |
| gpt-oss-120b | 78 GiB | MXFP4 | NVIDIA A100 80GB SXM, NVIDIA H100 80GB SXM | 96 GiB (MXFP4) |
"Cleanly" means the calculator's fits or comfortable verdict on the high end of the estimate range. The last column is the first budget where the model's largest file is comfortable. Estimates come from catalog formulas (calculator 0.3.0, dataset 2026-10-02.1); none is a measurement on these cards.
What each memory budget covers
Devices with the same net budget get identical verdicts, so a 16 GB card and another 16 GB card tie here. Use bandwidth and software support from the vendor pages to break ties. A tight verdict is not counted as running.
How to read the memory ranking
- Going from 12 to 16 GB raises the net budget from 11 to 14 GiB and moves 14B-class models from tight to fits; it does not move 24B-class models.
- Mixture-of-experts files are counted at full size, so Qwen3-30B-A3B needs about as much as a 32B dense model. See why every expert is loaded.
- Long contexts raise the bill at different rates per model, see KV cache size by model. Each model page lists the verdict at 32,768 tokens.
- Unified-memory machines share one pool with the OS. The reserve in each record is an assumption; read its page.
Data sources
- Catalog model files and architectures — dataset 2026-10-02.1; byte counts from Hugging Face repository listings, listed on each model page
- Hardware capacities — Vendor specification pages, listed on each device page