Qwen3.8-27B on GeForce RTX 4090 24GB (desktop)

kelnei llama.cpp on one RTX 4090

Qwen3.8-27B on GeForce RTX 4090 24GB (desktop). One headless Linux report for Qwen3.8-27B on a 24 GB RTX 4090: llama.cpp pin b10448, CUDA 13.2, sm89, Unsloth UD-Q4_K_XL main, F16 projector, and the MTP inside that main file. Context in the report is 131072. This site did not run it.

The catalog VRAM estimate for this checkpoint is on the Qwen3.8-27B VRAM page.

Community component placement

Copies text for another assistant or a person to review. Nothing is deployed.

This is not a fit and not a finished deployment. Source reported; not reproduced by this site.

Hardware matches this pin. 1 nodes. TP 1. EP unknown.

Weight revision is unresolved.

As-pinned recipe.

  • manifest-gguf: Apache-2.0 on the current Unsloth GGUF manifest card. Commercial use unknown. The manifest card says apache-2.0. The measured file's revision was not named, so this is not proof about the bytes that were timed.
  • official-checkpoint: Apache-2.0 on the current Qwen/Qwen3.8-27B manifest card. Commercial use unknown. The official card is not the GGUF file used in the report.
  • recipe-repo: Unknown. Commercial use unknown. The cached tree at e99d7d05e3e555d556bd8386017a10e27c3f79d8 has the README and setup scripts and no license file. The recipe license stays unknown. It is not inferred from llama.cpp or from the model card.
  • One headless Linux report. It is not a Windows result and not a 32 GB Blackwell recipe.
  • The report's speed figures are not a forecast and were not reproduced here.
  • Old and new quant file sizes differ. Until the measured GGUF revision is known, the manifest bytes can be the wrong generation.
  • Built-in MTP still occupies space inside the main file. Speculation off does not delete it.
  • The report is one headless Linux process. A display-attached or different CUDA build is outside the pin.

Do not add these rows together. Main weights overlap subcomponents. CPU and a unified pool are one residence, not extra capacity.

Placements are not applied to the calculator memory verdict.

Logical checkpoint

File ledger. Not a per-rank residence.

Main GGUF

File ledger.

17559178144 bytes (16.353 GiB). cluster total, not one rank.

GPU

Per-rank GPU residence.

Dense backbone

Discrete GPU. Resident matmul.

File size unknown.

F16 projector

Discrete GPU. Resident matmul.

927607488 bytes. cluster total, not one rank.

Internal MTP

Discrete GPU. Resident matmul.

File size unknown.

KV cache

Discrete GPU. Attention cache.

File size unknown.

Recurrent state

Discrete GPU. Recurrent state.

File size unknown.

Process and workspace

Discrete GPU. Runtime scratch.

File size unknown.

Absent: Routed experts, Engram, PLE. Absence is not zero bytes.

Resident bytes: unknown.

Placement

Local disk path

Not a drag-to-fit control.

Other rejected paths

Other residences

Full config and source evidence
  • Main GGUF
    The main UD-Q4_K_XL file is the inclusive ledger for this recipe, including the built-in MTP. The projector is separate. The MTP sidecar is not added.
    Discrete GPU memory after load. Manifest file bytes are not the measured resident split.
    17559178144 bytes (16.353 GiB). cluster total, not one rank.
    Per-rank storage: unknown. File ledger.
    Subcomponents can overlap this main-weight row. Do not add them into the checkpoint.
    Access: Resident matmul.
    The research brief does not tie this current manifest revision to the measured run. These file bytes are not resident memory and are not a fit.
    Process memory was not split into this file.
  • Routed experts
    There is no routed-expert stack on Qwen3.8-27B. The 6B active figure belongs to Qwen3.8-Flash-Next and is not a weight size for this model.
    Not present. Do not substitute a 6B active count for these weights.
    File size unknown.
    Per-rank storage: unknown. Not in this recipe.
    This row can overlap main weights. Do not add it again.
    Access: Not in this recipe.
    No routed-expert bytes exist on Qwen3.8-27B.
  • Dense backbone
    Qwen3.8-27B is a dense vision-language checkpoint. Its language stack is resident matmul. This row overlaps the main file and is not a second copy.
    Inside the main file. 64 layers: 48 Gated DeltaNet plus 16 attention layers.
    File size unknown.
    Per-rank storage: unknown. Overlaps the file ledger.
    Subcomponents can overlap this main-weight row. Do not add them into the checkpoint.
    Access: Resident matmul.
    Dense bytes were not split out of the main file.
  • Engram
    This Qwen profile has no Engram table. A PLE lookup, when this model has one, is a different component.
    Not present.
    File size unknown.
    Per-rank storage: unknown. Not in this recipe.
    This row can overlap main weights. Do not add it again.
    Access: Not in this recipe.
    No Engram bytes exist on this profile.
  • PLE
    Qwen3.8-27B does not use a 51B n-gram PLE table. SSD offload for this model is not the Flash-Next file-backed PLE path.
    Not present.
    File size unknown.
    Per-rank storage: unknown. Not in this recipe.
    This row can overlap main weights. Do not add it again.
    Access: Not in this recipe.
    No PLE bytes exist on Qwen3.8-27B.
  • F16 projector
    The projector is an optional vision role and a separate file from the main GGUF. It is not the drafter and not an extra copy of the language weights.
    Separate mmproj-F16.gguf. Loaded when vision is used. Manifest bytes are not the measured resident size.
    927607488 bytes. cluster total, not one rank.
    Per-rank storage: unknown. Separate from the file ledger.
    This row can overlap main weights. Do not add it again.
    Access: Resident matmul.
    The research brief does not tie this current manifest revision to the measured run. These file bytes are not resident memory and are not a fit.
  • Internal MTP
    This recipe uses the MTP stored inside the main GGUF. The 1369590656-byte MTP sidecar exists in the manifest and is not part of this recipe.
    Inside the main file. Not a separate loaded drafter. Not a shared-embedding sidecar.
    File size unknown.
    Per-rank storage: unknown. Overlaps the file ledger.
    Subcomponents can overlap this main-weight row. Do not add them into the checkpoint.
    Access: Resident matmul.
    MTP bytes inside the main GGUF were not split out.
    Do not add the sidecar on top of the main file.
  • KV cache
    KV is stored for the 16 attention layers, not for all 64 layers. Recurrent state is the other row. A cookbook byte-per-token anchor is not multiplied into a fit here.
    Reported q4_0 KV. Attention cache only. Not a 64-layer cache.
    File size unknown.
    Per-rank storage: unknown. Runtime, not weights.
    This row can overlap main weights. Do not add it again.
    Access: Attention cache.
    This module does not multiply a per-token width by context.
    Native context 262144 and the 1M extension are not automatically verified by a shorter recipe.
  • Recurrent state
    48 Gated DeltaNet layers keep a fixed recurrent state. That state is present at short context. It is not the KV cache, and a cookbook's approximate megabyte label is not stored as bytes.
    Discrete GPU memory. Fixed state slots, separate from the 16 attention KV layers.
    File size unknown.
    Per-rank storage: unknown. Runtime, not weights.
    This row can overlap main weights. Do not add it again.
    Access: Recurrent state.
    Slot count depends on the runtime. One slot is not the whole state budget.
    Approximate megabytes were not converted.
  • Process and workspace
    The report gives one headless process reading. It is not a per-component resident allocation and it was not measured by this site.
    Discrete GPU process. Host RAM is not included in that GPU reading.
    File size unknown.
    Per-rank storage: unknown. Runtime, not weights.
    This row can overlap main weights. Do not add it again.
    Access: Runtime scratch.
    22648 MiB is one process total, not weights, KV, or a fit.
  • Pinned source Cached recipe README at this repository SHA. This SHA does not prove the historic llama.cpp binary or the measured GGUF revision. The reported build tag remains b10448. e99d7d05e3e555d556bd8386017a10e27c3f79d8
  • Pinned source Cached llama.cpp setup script at the same repository SHA. It is not the measured binary hash. e99d7d05e3e555d556bd8386017a10e27c3f79d8
  • Pinned source Cached model-setup script at the same repository SHA. It does not name the timed GGUF revision. e99d7d05e3e555d556bd8386017a10e27c3f79d8
  • Pinned source Cached serve script at the same repository SHA. It is not a measurement of the running binary. e99d7d05e3e555d556bd8386017a10e27c3f79d8
  • Pinned source Current GGUF manifest bytes and Apache-2.0 card. Not tied to the benchmark. 4ca720788d1e01f1bff70c033e0d0028fd02e502
  • Pinned source Official BF16 file-byte manifest. Not the GGUF that was timed. 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0

Speed figures in the copied brief are maintainer observations, not a measurement by this site.

Related

Hashes, preflight, and reported observations

Hardware nvidia-geforce-rtx-4090-24gb-desktop. Pool Discrete GPU memory on one RTX 4090. Host RAM is a separate pool. Adding host RAM to VRAM does not make this a unified Spark pool and does not create a fit.. Weight pin manifest-not-tied-to-report.

Checkpoint unsloth/Qwen3.8-27B-GGUF revision unknown. Manifest 4ca720788d1e01f1bff70c033e0d0028fd02e502. Runtime Reported llama.cpp short build tag b10448, CUDA 13.2, compute capability sm89. The recipe repository pin is separate. The full measured llama.cpp binary hash is unknown, and b10448 is not that hash.. Commit b10448.

Reported context is 131072 with q4_0 KV. Concurrency was not printed as an integer. Internal MTP is the reported speculative path. The sidecar MTP file is not this path.

Commercial recommendation: false. Memory fit claimed: false. Deployment done: false.

  • manifest-gguf: Apache-2.0 on the current Unsloth GGUF manifest card. SPDX Apache-2.0. Commercial use unknown. The manifest card says apache-2.0. The measured file's revision was not named, so this is not proof about the bytes that were timed.
  • official-checkpoint: Apache-2.0 on the current Qwen/Qwen3.8-27B manifest card. SPDX Apache-2.0. Commercial use unknown. The official card is not the GGUF file used in the report.
  • recipe-repo: Unknown. SPDX unknown. Commercial use unknown. The cached tree at e99d7d05e3e555d556bd8386017a10e27c3f79d8 has the README and setup scripts and no license file. The recipe license stays unknown. It is not inferred from llama.cpp or from the model card.

Dense model. No routed experts. Do not import Flash-Next's 6B active label or 512 experts. Official BF16 safetensors sum to 55,563,006,776 bytes at manifest revision 1d4bf0f2. The index payload 55,562,855,904 bytes is a different number and is not stored here. Served GGUF or NVFP4 bytes are not this official file. Checkpoint file 55563006776 bytes (51.747 GiB).

Preflight

  • Confirm llama.cpp pin b10448, CUDA 13.2, and sm89 before treating the report as the same binary.
  • Use the main GGUF plus the F16 projector. Do not also load the MTP sidecar.
  • The measured GGUF revision is unknown. Record the file revision you actually have and do not assume it is 4ca72078.
  • Context in the report is 131072. A longer context, including the 1M extension, is outside this report.
  • Host RAM is not added to the 24 GB class GPU.

Validation

  • The research brief reports 22648 MiB used of 24564 MiB on that headless run, at context 131072.
  • The same brief reports about 104 tok/s with MTP and 44.8 tok/s plain, and about 1800 tok/s prefill at 120K. Those are source reports.
  • This site did not run the recipe or confirm those readings.

Rollback

  • Keep the previous llama.cpp build and the previous GGUF pair. This module does not switch binaries or delete files.

Sources

Reported observations

Maintainer observations, not a measurement by this site, and not a forecast.

  • Reported decode: about 104 tok/s with MTP, 44.8 tok/s plain. kelnei headless Linux report, context 131072. Source report. Not a site measurement and not a forecast.
  • Do not blend with qwen3.8-27b-spark-sglang-dspark-c1
  • Main GGUF: resident bytes are unknown.
  • Routed experts: resident bytes are unknown.
  • Dense backbone: resident bytes are unknown.
  • Engram: resident bytes are unknown.
  • PLE: resident bytes are unknown.
  • F16 projector: resident bytes are unknown.
  • Internal MTP: resident bytes are unknown.
  • KV cache: resident bytes are unknown.
  • Recurrent state: resident bytes are unknown.
  • Process and workspace: resident bytes are unknown.
  • Runtime claimed by this site: false. Summing components creates a fit: false. The generic calculator verdict is unchanged.