Fits: Qwen3.8-27B needs 19.36 GiB on RTX 4090 (Rough estimate)

How this estimate is calculated

Catalog config for Qwen/Qwen3.8-27B (revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0) is the Hugging Face config.json linked below. text_config records 64 layers, hidden size 5,120, 24 attention heads, 4 KV heads, head_dim 256, and max_position_embeddings 262,144. The hybrid spec is 16 groups of 3 Gated DeltaNet layers and 1 gated-attention layer (16 attention layers, 48 linear layers). KV uses those attention layers only. The all-layer GQA formula is not used. This page pins GGUF UD-Q4_K_M (Unsloth), file Qwen3.8-27B-UD-Q4_K_M.gguf from unsloth/Qwen3.8-27B-GGUF, 16,464,440,224 bytes. The model page auto-select is GGUF Q4_K_M (mradermacher), file Qwen3.8-27B.Q4_K_M.gguf, a different published file. At 8,192 tokens, 1 sequence, FP16 KV, llama.cpp, the calculator estimates 16.11 GiB–19.36 GiB. This catalog has no calibration and no planned benchmark for this model. The range is a calculator estimate, not a measured run.

On RTX 4090 the catalog net budget is 22.00 GiB after the default reserve. 24 GB GDDR6X is discrete VRAM. Host RAM is not added to that budget.

• config.json num_hidden_layers64
• config.json hidden_size5,120
• config.json num_attention_heads24
• config.json num_key_value_heads4
• config.json head_dim256
• config.json max_position_embeddings262,144
• catalog hybrid groups16
• Gated DeltaNet layers48
• gated-attention layers16
• linear_num_key_heads16
• linear_num_value_heads48
• linear_key_head_dim128
• linear_value_head_dim128
• linear_conv_kernel_dim4
• recurrent state dtypefp32
• total parameters27,781,427,952
• weight fileQwen3.8-27B-UD-Q4_K_M.gguf (16,464,440,224 bytes)

Calculator notes for this setup

• Weight bytes come from the selected weight files only, not from every file in the repository.
• Counted files: Qwen3.8-27B-UD-Q4_K_M.gguf.
• Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
• This checkpoint has a vision component. Text-only inference does not include vision encoder or projector bytes.
• Multi-token prediction / draft weights exist and are not included unless enabled.
• Hybrid layout: 16 groups of Gated DeltaNet plus gated attention (16 attention layers, 48 linear layers).
• Attention KV uses 16 gated-attention layers, 4 KV heads, head dim 256. Linear layers are not counted as extra KV layers.
• Gated DeltaNet recurrent state uses 1 sequences × 48 layers × 48 value heads × 128×128 × fp32 element size. It does not grow with context the way KV does, and is not treated as zero.
• Gated DeltaNet conv state uses 1 sequences × 48 layers × recorded channels × kernel 4 × fp32. Layout follows the published kernel dim, not a runtime dump.
• Using 4 KV heads, not 24 query heads.
• KV dtype is independent of weight quantization unless you change it.
• Recurrent and conv state dtype is independent of KV cache dtype. Changing KV precision does not silently change state precision.
• Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
• Concurrent sequences share one model replica; weights are not multiplied by sequence count.
• Hybrid cache packing differs by engine (Transformers recurrent/conv vs vLLM mamba_cache_dtype / mamba_ssm_cache_dtype). This is a partial estimate, not a complete green result.
• Host RAM is not added to discrete GPU VRAM.
• llama.cpp compute buffers grow with context and batch. Values here are user-adjustable scenarios, not measured traces — unless a matching calibration is shown.

• config.json — revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
• Qwen3.8-27B model card
• unsloth/Qwen3.8-27B-GGUF weight repo
• Methodology
• Qwen3.8-27B VRAM
• NVIDIA GeForce RTX 4090 product page — Standard Memory Config 24 GB GDDR6X

Memory breakdown

Weight artifact

15.33 GiB (16,464,440,224 bytes)

Raw KV cache

0.50 GiB (536,870,912 bytes)

Engine extras / workspace

0.27 GiB–3.53 GiB

Total estimated demand

16.11 GiB–19.36 GiB

Net calculator budget

22.00 GiB

OS / display reserve

2.00 GiB

Remaining vs net budget

2.64 GiB–5.89 GiB

Enough memory does not mean this engine, quant, and OS will run. Support does not mean it will be fast.

Other published files on this device

Same model, device and context (8,192 tokens, 1 sequence, FP16 KV). Each row is one published weight-file set with its usual engine, sorted by memory.

Assumptions that affect this result

• Requested context is within the recorded native window of 262144 tokens.
• Full-resident weight estimates use total parameters, including inactive MoE experts.
• Weight bytes come from the selected weight files only, not from every file in the repository.
• Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
• Counted files: Qwen3.8-27B-UD-Q4_K_M.gguf.
• This checkpoint has a vision component. Text-only inference does not include vision encoder or projector bytes.
• Multi-token prediction / draft weights exist and are not included unless enabled.
• Context budget is the number of cached tokens per sequence (prompt plus reserved generation).

Related pages

• Qwen3.8-27B VRAM
• GeForce RTX 4090 24GB (desktop)
• Can Qwen3.8-27B run on DGX Spark?
• Can Qwen3.8-27B run on Apple M2 Max 64GB unified (MacBook Pro / Mac Studio)?
• KV cache calculator
• File size is not runtime VRAM
• Out-of-memory troubleshooting
• Quantization and memory

Sources

• Qwen3.8-27B model card
• config.json — revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
• safetensors index — metadata.total_size=55562855904
• Artifact repo unsloth/Qwen3.8-27B-GGUF
• Artifact repo mradermacher/Qwen3.8-27B-GGUF
• NVIDIA GeForce RTX 4090 product page — Standard Memory Config 24 GB GDDR6X

Calculator