What you need to run AI at home
thecontinuitylayer.com/open/hardwareChecked 6 November 2026. US dollars. No affiliate links.
One number decides almost everything: how much memory the model can use. It decides which models fit at all. The type of memory, and the chip next to it, decides how fast they run. Start with the computer you already have; most people can run something useful tonight.
The one number: memory
A model at the usual 4-bit size needs about 0.6GB per billion parameters, plus a few gigabytes for the conversation. An 8B model wants about 5–6GB; a 27B model about 18–20GB; a 70B model about 43GB. Plan on using about 60% of your total memory for the model, because the system and your other apps need the rest. The full rule, with real download sizes, is in model sizes and quantization.
RAM, VRAM, and which chip
| Setup | Memory that counts | Speed | Notes |
|---|---|---|---|
| PC with an NVIDIA card | The card's VRAM | Fastest per dollar on models that fit | Widest software support (CUDA). A model that spills out of VRAM slows to a crawl. |
| PC with an AMD card | The card's VRAM | Fast once set up | Works with llama.cpp, Ollama and LM Studio; more setup on some systems. |
| PC with an Intel Arc card | The card's VRAM | Good at small and mid sizes | Cheap 24GB option; expect to do your own software setup. |
| Apple Silicon Mac | Most of the shared memory | Slower than a big card, but fits more | A 64–128GB Mac runs models no single home card can. Memory can't be upgraded later. |
| Processor only (no graphics card) | System RAM | Slow | Fine for 3–8B models on a recent computer; painful above that. |
| 128GB mini PC (Strix Halo, DGX Spark) | Shared memory, up to 128GB | Good on mixture-of-experts, slow on dense 70B | The cheapest way to fit 120B-class models on one desk. |
Five tiers, with example machines
| Tier | Runs well | Example machines |
|---|---|---|
| 8GB | 3–8B models: Gemma 4 E4B, Llama 3.1 8B | Any laptop or desktop with 8GB of memory; an 8GB Mac |
| 16GB | 8–14B models: Gemma 4 12B, Qwen3.5 9B; gpt-oss-20b just fits | A 16GB Mac or laptop; GeForce RTX 5060 Ti 16GB |
| 24–32GB | 24–32B models: Qwen3.8 27B, Gemma 4 31B, Devstral Small 2 | GeForce RTX 3090 (used), GeForce RTX 4090 (used), GeForce RTX 5090, Intel Arc Pro B60 |
| 64GB | 27–32B at higher quality, long context; 70B too tight | Mac mini (M5 Pro) with 64GB |
| 128GB | gpt-oss-120b, 70B at 4-bit | Mac Studio (M5 Max), Framework Desktop, GMKtec EVO-X2, NVIDIA DGX Spark |
Prices and specs for every machine are on the best hardware to run an LLM locally, read from each maker's page (last reviewed 31 October 2026). Measured speeds by machine are on Will it run on my machine?
Older and low-end computers
- A computer with 8GB and no graphics card can run 1–4B models. Answers are simple, but private and offline.
- Close the browser and other heavy apps before loading a model; that memory is what the model needs.
- Pick the smallest model that does the job, and keep conversations short: long chats use more memory.
- LM Studio does not support Intel Macs. llama.cpp still runs on them, on the processor, slowly.
- The best picks for these machines are in best local AI for low-end hardware.
When renting a GPU makes more sense
- You want a 70B-plus or frontier-size model now, without spending thousands.
- You need it for a project, a weekend or a burst of work, not every day.
- You want to try a model before buying hardware for it.
A 24GB card rents for well under a dollar an hour, and an 80GB H100 for about $2–4.50 an hour on demand. Compare GPU clouds by the hour. If you only want better answers, a $10–20 monthly plan for open models is cheaper than both; see a cheap monthly plan.