What LLM can I run on my GPU?

Pick your card to see which models fit it. All 32 GPUs and unified-memory machines are grouped by memory size. “Net budget” is what is left after the reserve for the OS and display.

8 GB 4

8 GB VRAM

GeForce RTX 3060 8GB (desktop)8 GB GDDR6Net budget: 7 GiBGeForce RTX 3070 8GB (desktop)8 GB GDDR6Net budget: 7 GiBGeForce RTX 4060 8GB (desktop)8 GB GDDR6Net budget: 7 GiBGeForce RTX 5060 Ti 8GB (desktop)8 GB GDDR7Net budget: 7 GiB

10–12 GB 6

12 GB VRAM

GeForce RTX 3080 10GB (desktop)10 GB GDDR6XNet budget: 9 GiBGeForce RTX 3060 12GB (desktop)12 GB GDDR6Net budget: 11 GiBGeForce RTX 3080 12GB (desktop)12 GB GDDR6XNet budget: 11 GiBGeForce RTX 4070 12GB (desktop, GDDR6X)12 GB GDDR6XNet budget: 11 GiBGeForce RTX 4070 SUPER 12GB (desktop)12 GB GDDR6XNet budget: 11 GiBGeForce RTX 5070 12GB (desktop)12 GB GDDR7Net budget: 11 GiB

16 GB 10

16 GB VRAM

GeForce RTX 4060 Ti 16GB (desktop)16 GB GDDR6Net budget: 15 GiBGeForce RTX 4070 Ti SUPER 16GB (desktop)16 GB GDDR6XNet budget: 15 GiBGeForce RTX 4080 16GB (desktop)16 GB GDDR6XNet budget: 14 GiBGeForce RTX 4080 SUPER 16GB (desktop)16 GB GDDR6XNet budget: 14 GiBGeForce RTX 5060 Ti 16GB (desktop)16 GB GDDR7Net budget: 14 GiBGeForce RTX 5070 Ti 16GB (desktop)16 GB GDDR7Net budget: 14 GiBGeForce RTX 5080 16GB (desktop)16 GB GDDR7Net budget: 14 GiBRadeon RX 7800 XT 16GB (desktop)16 GB GDDR6Net budget: 14 GiBRadeon RX 9070 16GB (desktop)16 GB GDDR6Net budget: 14 GiBRadeon RX 9070 XT 16GB (desktop)16 GB GDDR6Net budget: 14 GiB

20–24 GB 4

24 GB VRAM

Radeon RX 7900 XT 20GB (desktop)20 GB GDDR6Net budget: 18 GiBGeForce RTX 3090 24GB (desktop)24 GB GDDR6XNet budget: 22 GiBGeForce RTX 4090 24GB (desktop)24 GB GDDR6XNet budget: 22 GiBRadeon RX 7900 XTX 24GB (desktop)24 GB GDDR6Net budget: 22 GiB

32–80 GB 4

32 GB VRAM · 48 GB VRAM

GeForce RTX 5090 32GB (desktop)32 GB GDDR7Net budget: 30 GiBNVIDIA L40S 48GB48GB GDDR6 with ECCNet budget: 46 GiBNVIDIA A100 80GB SXM80GB HBM2eNet budget: 78 GiBNVIDIA H100 80GB SXM80GBNet budget: 78 GiB

Unified memory 4

Apple M2 Max 64GB unified (MacBook Pro / Mac Studio)64 GB unified memoryNet budget: 54 GiBApple M4 Max 128GB unified (MacBook Pro)128 GB unified memoryNet budget: 116 GiBNVIDIA DGX Spark (GB10, 128 GB unified)128 GB LPDDR5x coherent unified system memoryNet budget: 112 GiBRyzen AI Max+ 395128 GB LPDDR5x-8000 unified memoryNet budget: 96 GiB

Related pages