Can I run this model on my GPU?

78 pairs, grouped by card. Each page gives the verdict, the memory estimate, the quant to download and the longest context that fits. A pair only gets a page when the quant choice changes the answer.

GeForce RTX 4090 24GB (desktop) (11)

GeForce RTX 4060 8GB (desktop) (8)

GeForce RTX 4060 Ti 16GB (desktop) (7)

GeForce RTX 4070 12GB (desktop, GDDR6X) (6)

GeForce RTX 4070 Ti SUPER 16GB (desktop) (6)

GeForce RTX 5090 32GB (desktop) (6)

NVIDIA DGX Spark (GB10, 128 GB unified) (6)

Apple M2 Max 64GB unified (MacBook Pro / Mac Studio) (5)

GeForce RTX 3060 12GB (desktop) (5)

GeForce RTX 5080 16GB (desktop) (5)

GeForce RTX 3090 24GB (desktop) (3)

Radeon RX 7900 XTX 24GB (desktop) (3)

GeForce RTX 4080 SUPER 16GB (desktop) (2)

Apple M4 Max 128GB unified (MacBook Pro) (1)

NVIDIA A100 80GB SXM (1)

NVIDIA H100 80GB SXM (1)

Radeon RX 7800 XT 16GB (desktop) (1)

Radeon RX 7900 XT 20GB (desktop) (1)

Related pages