Local LLM memory guides
Short, practical guides on fitting models into GPU memory. New here? Start with how much VRAM a local LLM needs.
Getting started
How much memory a local model needs, and why it is more than the file size.
Quantization and file size
What Q4, Q5, Q6, Q8 and FP16 files cost in memory.
Context length and KV cache
How longer prompts and parallel requests add memory on top of the weights.
Multi-GPU, Mac and DGX Spark
Why two cards are not one bigger card, and how unified memory is counted.
Specific models
Memory tables for popular model families.
Fine-tuning (LoRA and QLoRA)
Published minimums for training, next to the inference numbers.
Deployment reports
Write-ups of specific setups on real hardware.
New releases we checked
What the publisher has released so far: weights, file sizes, or only an API.