Qwen3.8-27B on GeForce RTX 5090 32GB (desktop) — vLLM Deployment

Single NVIDIA RTX 5090 (32 GB, sm_120) deployment serving RadixArk/Qwen3.8-27B-NVFP4 with vLLM 0.27.1, native 256K context, TurboQuant 4-bit KV cache, and MTP-3 speculative decoding.

量化 NVFP4 safetensors
仓库 RadixArk/Qwen3.8-27B-NVFP4
已核上下文262,144
设备数1 × GeForce RTX 5090 32GB (desktop)

来源报告 https://github.com/MiaAI-Lab/Qwen3.8-27B-NVFP4-RTX-5090

160 tok/s (decode-tok-s)

This deployment recipe is currently maintained in English because commands and technical instructions are source-aligned.

Qwen3.8-27B-NVFP4 on GeForce RTX 5090 32GB (vLLM MTP-3) is a working community deployment for Qwen3.8-27B running on 1 × GeForce RTX 5090 32GB (desktop) using vLLM and NVFP4 safetensors. Verified up to 262,144 tokens context.

Measured by upstream maintainerworking

Tested Setup & Specifications

Model CheckpointQwen3.8-27B
Hardware SKUGeForce RTX 5090 32GB (desktop)
Device Count1 device
Runtime EnginevLLM (0.27.1)
Patches RequiredCustom kernel patch
Artifact FormatNVFP4 · 4 bpw
Active Weights22.00 GiB
Tensor ParallelismTP = 1
PlacementSingle desktop GPU
Verified Context262,144 tokens
Last Verified2026-09-23

Tested & Reported Capabilities

✓Text generation verified
✓Clean model loading & initialisation
✓Context scaling verified up to 262,144 tokens
✓Function & Tool calling (Verified)
—Multimodal vision input (Not tested)
✓Multi-Token Prediction / Speculative draft (Verified)

Source-Backed Benchmarks

Measured performance on actual physical machines recorded by the upstream maintainer or community report. No linear extrapolations or simulated numbers.

Metric	Measured Value	Context	Concurrency	Details & Source
Decode Throughput	160 tok/s	—	1 stream	single-stream generation (MTP-3, TurboQuant 4-bit KV)Measured generation speed on RTX 5090 32GB with MTP-3 and TurboQuant 4-bit KV cache [source ↗]

Quick Start & Deployment Commands

Condensed deployment procedure extracted directly from the upstream repository.

Prerequisites

1× NVIDIA GeForce RTX 5090 (32GB VRAM, sm_120)
Linux with driver supporting sm_120 (570+)
Python 3.11+, gcc, curl, nvidia-smi

Review these commands, then run them yourself. This page does not run them. This is not the deployment brief.

1One-click start

Creates venv, installs dependencies, patches vLLM, downloads model, and launches on port 8888

./start.sh

Caveats & Operational Constraints

Open Calculator With This Exact Setup

Source & Provenance

Mia's AI Lab: MiaAI-Lab/Qwen3.8-27B-NVFP4-RTX-5090 — Commit: a5f9bff97796989b2f3629a2cf0d8335eca7a7e8
Artifact: RadixArk/Qwen3.8-27B-NVFP4 — NVFP4 safetensors