Model sizes and quantization
thecontinuitylayer.com/open/learn/model-sizesChecked 6 November 2026. US dollars. No affiliate links.
A model's name tells you its size: the “27B” in Qwen3.8 27B means 27 billion parameters, the numbers it learned. Bigger usually means smarter and always means more memory. Quantization is how a big model is squeezed to fit.
By the end
You can look at any model download and estimate, before you click, whether your machine can run it.
The rule of thumb
At 4-bit, the common download, a model needs about 0.6GB of memory per billion parameters, plus a few gigabytes for the conversation. So:
| Model size | Memory at 4-bit (about) | Real examples |
|---|---|---|
| 3–4B | 2–3GB | Llama 3.2 3B: 2.0GB download |
| 8–9B | 5–7GB | Llama 3.1 8B: 4.9GB; Qwen3.5 9B: 6.6–7.6GB |
| 12–14B | 7–9GB | Gemma 4 12B: 6.7GB (Google); Phi-4: 9.1GB |
| 24–32B | 15–20GB | Mistral Small 3.2: 15GB; Qwen3.8 27B: 18GB |
| 70B | about 40–43GB | Llama 3.3 70B: 43GB |
| 120B (mixture of experts) | about 65GB | gpt-oss-120b: 65GB |
Figures in the third column are the published download sizes on Ollama's library and Google's Gemma 4 card, read on 6 November 2026.
What quantization does
A model is stored as numbers. At full precision each number takes 16 bits; quantization rounds them to 8, 5 or 4 bits. That makes the file about a quarter of the size at 4-bit, and it runs faster. The cost is a small loss in quality, larger on small models and at 3 bits or below.
- Q4 (4-bit, e.g. Q4_K_M): the usual default. The best balance for most people.
- Q5 / Q6: a little better, a little bigger. Worth it if you have room.
- Q8 (8-bit): close to full quality, twice the size of Q4.
- Q2 / Q3: only to squeeze a big model onto a small machine. Expect mistakes.
Rule: a bigger model at 4-bit usually beats a smaller model at 8-bit for the same memory.
RAM, VRAM and Apple Silicon
- PC with a graphics card: the card's own memory (VRAM) is what makes it fast. A model that fits in VRAM is quick; one that spills into ordinary RAM runs, but much slower.
- PC without a graphics card: it runs on the processor from RAM. Fine for 3–8B models; slow beyond that.
- Mac with Apple Silicon: memory is shared, so the model can use most of it. A 32GB Mac fits models a 24GB card cannot, but runs them more slowly than the card would.
- NVIDIA vs AMD: NVIDIA has the widest software support. AMD cards work with llama.cpp, Ollama and LM Studio, with more setup on some systems.
Mixture of experts: big, but faster than its size
Some models, written like “26B A4B”, are mixtures of experts: all 26 billion parameters must fit in memory, but only about 4 billion work on each word. They need the memory of a big model and run at the speed of a small one. Gemma 4 26B A4B, gpt-oss and most of the giant open models work this way.