Which local LLMs fit on an L40S (48GB GDDR6 with ECC)?
The L40S has 48GB GDDR6 with ECC of discrete VRAM. After a 2 GiB reserve for the OS and display, about 46 GiB is usable for a model. At an 8K context the largest model in our list that fits is Mixtral-8x7B-Instruct-v0.1 at Q6_K (about 38 GiB). Qwen3-Next-80B-A3B-Instruct at Q4_K_M is tight (about 46 GiB–47.3 GiB).
These are estimates from the model's config and file sizes, not measurements. How we calculate
Which models fit at 8K
Each model uses the highest-precision file that fits, one chat at a time.
Fits (60)
| Model | Quant | Total |
|---|---|---|
| Mixtral-8x7B-Instruct-v0.1 | GGUF Q6_K (mradermacher) | 36.9 GiB–38.0 GiB |
| Qwen2.5-32B-Instruct | GGUF Q8_0 (Qwen) | 34.6 GiB–35.7 GiB |
| Qwen2.5-VL-32B-Instruct | GGUF Q8_0 (mradermacher) | 34.6 GiB–35.7 GiB |
| Qwen2.5-Coder-32B-Instruct | GGUF Q8_0 (Qwen) | 34.6 GiB–35.7 GiB |
| QwQ-32B | GGUF Q8_0 (Qwen) | 34.6 GiB–35.7 GiB |
| DeepSeek-R1-Distill-Qwen-32B | GGUF Q8_0 (Unsloth) | 34.6 GiB–35.7 GiB |
| OpenMath-Nemotron-32B | GGUF Q8_0 (mradermacher) | 34.6 GiB–35.7 GiB |
| Qwen3-32B | GGUF Q8_0 (Qwen) | 34.6 GiB–35.7 GiB |
| Qwen3.8-27B | GGUF Q8_0 (Unsloth) | 27.8 GiB–33.1 GiB |
| Qwen3-VL-30B-A3B-Instruct | GGUF Q8_0 (Qwen) | 31.1 GiB–32.2 GiB |
| Qwen3-30B-A3B-Instruct-2507 | GGUF Q8_0 (Unsloth) | 31.1 GiB–32.2 GiB |
| Qwen3-30B-A3B | GGUF Q8_0 | 31.1 GiB–32.2 GiB |
| Drama 3 | Unofficial int4 W4A16 (GPTQ-Marlin, with codec) | 22.7 GiB–27.0 GiB |
| Mistral-Small-24B-Instruct-2501 | GGUF Q8_0 (Unsloth) | 24.7 GiB–25.6 GiB |
| DeepSeek-V2-Lite-Chat | GGUF Q8_0 (mradermacher) | 15.9 GiB–19.2 GiB |
| Gemma 3 27B IT | GGUF Q4_0 (Google QAT) | 17.2 GiB–18.0 GiB |
| Qwen2.5-14B-Instruct | GGUF Q8_0 | 16.2 GiB–17.0 GiB |
| Qwen2.5-Coder-14B-Instruct | GGUF Q8_0 (Qwen) | 16.2 GiB–17.0 GiB |
| OpenMath-Nemotron-14B | GGUF Q8_0 (mradermacher) | 16.2 GiB–17.0 GiB |
| Phi-4-reasoning | GGUF Q8_0 (Unsloth) | 16.2 GiB–17.0 GiB |
| phi-4 | GGUF Q8_0 (Microsoft) | 16.2 GiB–17.0 GiB |
| Llama 3.1 8B Instruct | Official BF16 safetensors | 16.1 GiB–16.8 GiB |
| Qwen3-14B | GGUF Q8_0 (Qwen) | 16.0 GiB–16.8 GiB |
| Mistral-Nemo-Instruct-2407 | GGUF Q8_0 (Unsloth) | 13.5 GiB–14.2 GiB |
| gpt-oss-20b | MXFP4 safetensors | 13.1 GiB–13.8 GiB |
| GLM-4-9B-0414 | GGUF Q8_0 (Unsloth) | 9.8 GiB–10.3 GiB |
| granite-3.3-8b-instruct | GGUF Q8_0 (IBM) | 9.5 GiB–10.1 GiB |
| Qwen3-VL-8B-Instruct | GGUF Q8_0 (Qwen) | 9.4 GiB–10.0 GiB |
| Qwen3-8B-Base | GGUF Q8_0 (mradermacher) | 9.4 GiB–10.0 GiB |
| Qwen3-8B | GGUF Q8_0 | 9.4 GiB–10.0 GiB |
| Qwen2.5-Math-7B-Instruct | GGUF Q8_0 (bartowski) | 8.1 GiB–10.0 GiB |
| DeepSeek-R1-Distill-Llama-8B | GGUF Q8_0 (Unsloth) | 9.1 GiB–9.7 GiB |
| Gemma 3 12B IT | GGUF Q4_0 (Google QAT) | 8.5 GiB–9.0 GiB |
| Mistral-7B-Instruct-v0.3 | GGUF Q8_0 (bartowski) | 8.3 GiB–8.9 GiB |
| Qwen2.5-Coder-7B | GGUF Q8_0 (mradermacher) | 8.1 GiB–8.7 GiB |
| Qwen2.5-Coder-7B-Instruct | GGUF Q8_0 (Qwen) | 8.1 GiB–8.7 GiB |
| Qwen2.5-7B-Instruct | GGUF Q8_0 | 8.1 GiB–8.7 GiB |
| DeepSeek-R1-Distill-Qwen-7B | GGUF Q8_0 (Unsloth) | 8.1 GiB–8.7 GiB |
| OpenMath-Nemotron-7B | GGUF Q8_0 (mradermacher) | 8.1 GiB–8.7 GiB |
| Qwen2.5-VL-7B-Instruct | GGUF Q8_0 (Unsloth) | 8.1 GiB–8.7 GiB |
| Qwen2-7B-Instruct | GGUF Q8_0 (Qwen) | 8.1 GiB–8.7 GiB |
| Phi-3.5-mini-instruct | GGUF Q8_0 (bartowski) | 6.9 GiB–7.6 GiB |
| Llama 3.2 3B Instruct | Official BF16 safetensors | 7.0 GiB–7.5 GiB |
| Qwen3-VL-4B-Instruct | GGUF Q8_0 (Qwen) | 5.2 GiB–5.8 GiB |
| Qwen3-4B-Instruct-2507 | GGUF Q8_0 (Unsloth) | 5.2 GiB–5.8 GiB |
| Qwen3-4B | GGUF Q8_0 | 5.2 GiB–5.8 GiB |
| Phi-4-mini-instruct | GGUF Q8_0 (Unsloth) | 4.9 GiB–5.5 GiB |
| Qwen2.5-Coder-3B-Instruct | GGUF Q8_0 (Qwen) | 3.8 GiB–4.2 GiB |
| Qwen2.5-3B-Instruct | GGUF Q8_0 | 3.8 GiB–4.2 GiB |
| Qwen2.5-VL-3B-Instruct | GGUF Q8_0 (Unsloth) | 3.5 GiB–3.9 GiB |
| SmolLM2-1.7B-Instruct | GGUF Q8_0 (Unsloth) | 3.3 GiB–3.8 GiB |
| Gemma 3 4B IT | GGUF Q4_0 (Google QAT) | 3.3 GiB–3.8 GiB |
| Qwen3-1.7B-Base | GGUF Q8_0 (mradermacher) | 2.7 GiB–3.2 GiB |
| Qwen3-1.7B | GGUF Q8_0 | 2.7 GiB–3.2 GiB |
| Llama 3.2 1B Instruct | Official BF16 safetensors | 2.7 GiB–3.1 GiB |
| Qwen2.5-Coder-1.5B-Instruct | GGUF Q8_0 (Qwen) | 2.1 GiB–2.5 GiB |
| Qwen2.5-1.5B-Instruct | GGUF Q8_0 (Qwen) | 2.1 GiB–2.5 GiB |
| Qwen3-0.6B | GGUF Q8_0 (Qwen) | 1.6 GiB–2.0 GiB |
| Gemma 3 1B IT | GGUF Q4_0 (Google QAT) | 1.1 GiB–1.5 GiB |
| Qwen2.5-0.5B-Instruct | GGUF Q8_0 (Qwen) | 0.8 GiB–1.2 GiB |
Tight fit (3)
| Model | Quant | Total |
|---|---|---|
| Qwen3-Next-80B-A3B-Instruct | GGUF Q4_K_M (Qwen) | 46.0 GiB–47.3 GiB |
| Qwen2.5-72B-Instruct | GGUF Q4_K_M | 43.6 GiB–45.0 GiB |
| Llama-3.1-Nemotron-70B-Instruct-HF | GGUF Q4_K_M (bartowski) | 42.2 GiB–43.6 GiB |
Won't fit (23)
- gpt-oss-120b · MXFP4 · 62.8 GiB
- GLM-4.5-Air · Q4_K_M · 71.4 GiB
- Qwen3.8-Flash-Next · Q4_K_M · 131.5 GiB
- MiniMax-M2 · Q4_K_M · 134 GiB
- Qwen3-235B-A22B · Q4_K_M · 137.1 GiB
- Qwen3-235B-A22B-Instruct-2507 · Q4_K_M · 137.1 GiB
- Llama 3.3 70B Instruct · BF16 · 137.2 GiB
- DeepSeek-V4-Flash-0731 · FP8 · 183.7 GiB
- GLM-4.6 · Q4_K_M · 208.4 GiB
- GLM-4.5 · Q4_K_M · 209.2 GiB
- MiniMax-M3 · Q4_K_M · 249.7 GiB
- GLM-5.3-Flash · Q8_0 · 373.6 GiB
- DeepSeek-V3 · Q4_K_M · 443.1 GiB
- DeepSeek-R1 · Q4_K_M · 443.1 GiB
- DeepSeek-V3-0324 · Q4_K_M · 443.1 GiB
- DeepSeek-R1-0528 · Q4_K_M · 443.7 GiB
- DeepSeek-V3.1 · Q4_K_M · 444.2 GiB
- Muse Spark Open · NVFP4 · 508.9 GiB
- DeepSeek-V4.1-Flash · FP8 · 558.8 GiB
- Ling-3.1-flash · FP8 · 628.3 GiB
- Kimi-K2-Instruct · Q4_K_M · 679.5 GiB
- Kimi-K2-Thinking · Q4_K_M · 680 GiB
- GLM-5.3 · Q8_0 · 877 GiB
Memory
- NVIDIAL40S 48GB GDDR6 ECC
- Memory kindDiscrete VRAM
- Marketed48GB GDDR6 with ECC
- Gross48.00 GiB (51,539,607,552 bytes)
- Default reserve2.00 GiB (2,147,483,648 bytes)
- Net budget46.00 GiB (49,392,123,904 bytes)
The reserve is an adjustable assumption. On a headless machine you can lower it in the calculator.
Two of these cards do not add up to one bigger card. Each copy of a model has to fit on one card unless the engine splits it with tensor parallelism.
Related pages
Sources
- NVIDIA L40S product page — GPU Memory 48GB GDDR6 with ECC
Calculator
Loading catalog snapshot…
Pick a GPU or Mac to see which models fit, or pick a model to see how much memory it needs.
Pick a GPU to see which models fit.