Qwen3.8-27B on RTX 5090: Runs comfortably, Rough estimate
Conditional memory judgement for one configuration. Change context, quant, or engine and the conclusion can change. Configuration used on this page: GGUF UD-Q4_K_M (Unsloth), 8,192 cached tokens, 1 sequence, engine llama.cpp. Status: Likely fits. Software: Not verified..
The generic calculator judges the canonical checkpoint. 1 community-tested deployment recipes are also listed for this setup.
Verdict on GeForce RTX 5090 32GB (desktop)
Yes, run Q6_K. The calculator puts it at 21.7 GiB–25.9 GiB at 8,192 tokens, inside 30.0 GiB of usable VRAM (after the driver reserve). Q8_0 is the next step up and does not clear the bar (27.8 GiB–33.1 GiB).
With Q6_K, the longest context that still counts as a fit is about 23,146 tokens of a 262,144-token native window, one sequence, fp16 KV cache.
| GGUF quant | File weights | KV cache | Estimated total | Verdict |
|---|---|---|---|---|
| Q8_0 | 27.1 GiB | 0.5 GiB | 27.8 GiB–33.1 GiB | Tight |
| Q6_K (recommended) | 20.9 GiB | 0.5 GiB | 21.7 GiB–25.9 GiB | Fits |
| Q5_K_M | 18.2 GiB | 0.5 GiB | 19.0 GiB–22.7 GiB | Fits |
| Q4_K_M | 15.7 GiB | 0.5 GiB | 16.4 GiB–19.7 GiB | Comfortable |
Engine llama.cpp, one sequence, fp16 KV. File weights are the published GGUF sizes in the catalog. Totals come from the same calculator that powers the tool below; they are estimates, not a measurement of this setup.
Known working setups
1 known setupsCommunity and maintainer-tested deployment recipes for this model on this hardware platform.
- ✓ generation
- ✓ 262,144 context
- ✓ tool calling
- ✓ speculative decoding
Single NVIDIA RTX 5090 (32 GB, sm_120) deployment serving RadixArk/Qwen3.8-27B-NVFP4 with vLLM 0.27.1, native 256K context, TurboQuant 4-bit KV cache, and MTP-3 speculative decoding.
Fixed page configuration
Qwen3.8-27B GGUF UD-Q4_K_M on the catalog RTX 5090 32 GB SKU. Same published quant as the 4090 allowlisted combo. The NVFP4 5090 recipe stays a separate sourced page.
- ModelQwen3.8-27B VRAM
- HardwareGeForce RTX 5090 32GB (desktop)
- QuantizationGGUF UD-Q4_K_M (Unsloth)
- Context length (cached tokens)8,192
- Engine
llama.cpp - Concurrent sequences1
- KV cache typefp16
Technical identifiers and share data
- Model
qwen3.8-27b - Hardware
nvidia-geforce-rtx-5090-32gb-desktop - Quantization
gguf-ud-q4_k_m - Share fragment for this identity
v2.eyJzY2hlbWFWZXJzaW9uIjoyLCJjYWxjdWxhdG9yVmVyc2lvbiI6IjAuMy4wIiwiZGF0YXNldFZlcnNpb24iOiIyMDI2LTEwLTAyLjEiLCJjb25maWciOnsibW9kZWxJZCI6InF3ZW4zLjgtMjdiIiwiaGFyZHdhcmVJZCI6Im52aWRpYS1nZWZvcmNlLXJ0eC01MDkwLTMyZ2ItZGVza3RvcCIsImRldmljZUNvdW50IjoxLCJxdWFudGl6YXRpb25JZCI6ImdndWYtdWQtcTRfa19tIiwiY29udGV4dFRva2VucyI6ODE5MiwiY29uY3VycmVudFNlcXVlbmNlcyI6MSwiZW5naW5lIjoibGxhbWEuY3BwIiwia3ZUeXBlIjoiZnAxNiJ9fQ
Memory breakdown
- Weight artifact
- 15.33 GiB (16,464,440,224 bytes)
- Raw KV cache
- 0.50 GiB (536,870,912 bytes)
- Engine extras / workspace
- 0.27 GiB–3.53 GiB
- Total estimated demand
- 16.11 GiB–19.36 GiB
- Net calculator budget
- 30.00 GiB
- OS / display reserve
- 2.00 GiB
- Remaining vs net budget
- 10.64 GiB–13.89 GiB
19.36 GiB high estimate vs 30.00 GiB net
Enough memory does not mean this engine, quant, and OS will run. Support does not mean it will be fast.
Other published files on this device
Same model, device and context (8,192 tokens, 1 sequence, FP16 KV). Each row is one published weight-file set with its usual engine, sorted by memory.
Show all 28 published files
| Quantization | Engine | Estimated memory | Fit |
|---|---|---|---|
| BF16 safetensors | vllm | 53.02 GiB–63.15 GiB | Does not fit · Rough estimate |
| GGUF BF16 (Unsloth) | llama.cpp | 51.68 GiB–61.08 GiB | Does not fit · Rough estimate |
| GGUF UD-Q8_K_XL (Unsloth) | llama.cpp | 30.07 GiB–35.74 GiB | Does not fit · Rough estimate |
| GGUF Q8_0 (Unsloth) | llama.cpp | 27.83 GiB–33.10 GiB | Tight fit · Rough estimate |
| GGUF Q8_0 (mradermacher) | llama.cpp | 27.83 GiB–33.10 GiB | Tight fit · Rough estimate |
| GGUF UD-Q6_K_XL (Unsloth) | llama.cpp | 24.33 GiB–29.01 GiB | Tight fit · Rough estimate |
| GGUF Q6_K (mradermacher) | llama.cpp | 21.66 GiB–25.88 GiB | Fits · Rough estimate |
| GGUF UD-Q5_K_M (Unsloth) | llama.cpp | 19.19 GiB–22.97 GiB | Fits · Rough estimate |
| GGUF Q5_K_M (mradermacher) | llama.cpp | 18.97 GiB–22.71 GiB | Fits · Rough estimate |
| GGUF Q5_K_S (mradermacher) | llama.cpp | 18.44 GiB–22.10 GiB | Fits · Rough estimate |
| GGUF UD-Q4_K_XL (Unsloth) | llama.cpp | 17.13 GiB–20.55 GiB | Runs comfortably · Rough estimate |
| GGUF Q4_1 (Unsloth) | llama.cpp | 17.11 GiB–20.53 GiB | Runs comfortably · Rough estimate |
| GGUF Q4_K_M (mradermacher) | llama.cpp | 16.43 GiB–19.74 GiB | Runs comfortably · Rough estimate |
| GGUF UD-Q4_K_M (Unsloth) This page | llama.cpp | 16.11 GiB–19.36 GiB | Runs comfortably · Rough estimate |
| GGUF Q4_0 (Unsloth) | llama.cpp | 15.73 GiB–18.91 GiB | Runs comfortably · Rough estimate |
| GGUF Q4_K_S (mradermacher) | llama.cpp | 15.51 GiB–18.66 GiB | Runs comfortably · Rough estimate |
| GGUF IQ4_XS (mradermacher) | llama.cpp | 15.13 GiB–18.22 GiB | Runs comfortably · Rough estimate |
| GGUF Q3_K_L (mradermacher) | llama.cpp | 14.33 GiB–17.28 GiB | Runs comfortably · Rough estimate |
| GGUF UD-IQ4_XS (Unsloth) | llama.cpp | 14.05 GiB–16.94 GiB | Runs comfortably · Rough estimate |
| GGUF Q3_K_M (mradermacher) | llama.cpp | 13.35 GiB–16.12 GiB | Runs comfortably · Rough estimate |
| GGUF Q3_K_S (mradermacher) | llama.cpp | 12.19 GiB–14.76 GiB | Runs comfortably · Rough estimate |
| GGUF UD-IQ3_S (Unsloth) | llama.cpp | 11.99 GiB–14.53 GiB | Runs comfortably · Rough estimate |
| GGUF UD-IQ3_XXS (Unsloth) | llama.cpp | 10.96 GiB–13.32 GiB | Runs comfortably · Rough estimate |
| GGUF Q2_K (mradermacher) | llama.cpp | 10.89 GiB–13.24 GiB | Runs comfortably · Rough estimate |
| GGUF UD-IQ2_S (Unsloth) | llama.cpp | 8.57 GiB–10.52 GiB | Runs comfortably · Rough estimate |
| GGUF UD-IQ2_XXS (Unsloth) | llama.cpp | 7.54 GiB–9.31 GiB | Runs comfortably · Rough estimate |
| GGUF UD-IQ1_M (Unsloth) | llama.cpp | 7.04 GiB–8.72 GiB | Runs comfortably · Rough estimate |
| GGUF UD-IQ1_S (Unsloth) | llama.cpp | 6.54 GiB–8.14 GiB | Runs comfortably · Rough estimate |
Assumptions that affect this result
- Requested context is within the recorded native window of 262144 tokens.
- Full-resident weight estimates use total parameters, including inactive MoE experts.
- Weight bytes come from the selected weight files only, not from every file in the repository.
- Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
- Counted files: Qwen3.8-27B-UD-Q4_K_M.gguf.
- This checkpoint has a vision component. Text-only inference does not include vision encoder or projector bytes.
- Multi-token prediction / draft weights exist and are not included unless enabled.
- Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
Related pages
Sources
- Qwen3.8-27B model card
- config.json — revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
- safetensors index — metadata.total_size=55562855904
- Artifact repo unsloth/Qwen3.8-27B-GGUF
- Artifact repo mradermacher/Qwen3.8-27B-GGUF
- NVIDIA GeForce RTX 5090 product page — Standard Memory Config 32 GB GDDR7
Calculator
This combo page keeps the route model and device. Customize to change them on the main calculator; the share then keeps that identity.
Loading catalog snapshot…
Search or pick a catalog checkpoint to see memory demand.