How much memory do I need to run an LLM locally?

Roughly the model's parameter count in gigabytes at 4-bit, plus room for context: 16GB runs 8–14B models, 24GB runs 27–32B, and 128GB runs 70B dense models and 120B mixture-of-experts models. If a model does not fit in memory it either fails to load or runs far too slowly to use.

Is a Mac or a PC with a graphics card better for local AI?

A PC with an NVIDIA card is faster per dollar on models that fit in the card's memory and has the widest software support. A Mac with 64GB or more fits much larger models in one quiet box, but runs them more slowly. Pick the PC for speed on mid-size models and the Mac for size and silence.

Are the 128GB mini PCs worth it?

They are the cheapest way to fit 70B-class and mixture-of-experts models on one machine. They are slow on dense 70B models and fast enough on mixture-of-experts models such as gpt-oss-120b. If you mostly want 30B-and-under models, a single 24GB card is cheaper and faster.

Tools · Local hardware

Best hardware to run an LLM locally (2026): graphics cards, Macs and 128GB desk boxes

thecontinuitylayer.com/tools/local-hardware

For most people the answer is a used 24GB graphics card for 27–32B models, or a 128GB Mac Studio or Strix Halo mini PC for 70B-class and mixture-of-experts models. Buy memory first and speed second: a model that does not fit will not run at all.

If you are shopping for the best hardware to run an LLM locally, start with memory: it decides which models fit at all. Speed decides whether you will enjoy using them. Each card below gives the price range, the size of model it runs, and who it suits.

14 entries · Last reviewed 31 October 2026 · US dollars · No affiliate links

Compared at a glance

Prices and fees are the ones each vendor published on the day in the table; check the vendor before buying. Price read from each vendor's own page on 2 November 2026; 7 of 14 carried a figure that day. Where a vendor showed none, the cell says “not published”.

Local hardware: name, what it is for, price, verdict and vendor link
Tool	Best for	Price	Verdict	Link
GeForce RTX 5060 Ti 16GBGPU	First local-AI card on a budget	$429 (16GB, NVIDIA starting price)	The cheapest new 16GB card: fine for 8–14B models, slow and tight above that.	nvidia.com ↗
GeForce RTX 5090GPU	Fastest single consumer card	not published	The fastest card you can put in a desktop, at a price and power draw most people cannot justify.	nvidia.com ↗
GeForce RTX 3090 (used)GPU (used)	Cheapest 24GB with CUDA	not published	Still the best value route to 24GB if you are happy buying used and checking the card.	nvidia.com ↗
GeForce RTX 4090 (used)GPU (used)	Fast 24GB without 5090 prices	not published	Faster than a 3090 on the same models, but it fits no bigger ones, so pay the difference only for speed.	nvidia.com ↗
AMD Radeon AI PRO R9700GPU	32GB on one new card, Linux	not published	The most memory on a single new card under the RTX 5090, if you can live without CUDA.	amd.com ↗
Intel Arc Pro B60GPU	Cheap new 24GB, more setup	not published	A low-cost 24GB card with a warranty, for people willing to do their own software setup.	intel.com ↗
Mac mini (M5 Pro)Desktop	Small, silent, mid-size models	From $1,699	The quietest way to run 27–32B models if you order it with 64GB; it cannot be upgraded later.	apple.com ↗
MacBook Pro (M5 Max)Laptop	Large models on a laptop	From $3,599 (14-inch) or $3,899 (16-inch)	The only laptop here that runs 70B-class models; you pay for the portability.	apple.com ↗
Mac Studio (M5 Max)Desktop	Fastest quiet 128GB box	From $2,499 (36GB); 128GB costs more	The best all-round 128GB desk machine for 70B–120B models, faster and quieter than the mini PCs.	apple.com ↗
Mac Studio (M5 Ultra)Desktop	Largest models on one desk	From $5,499 (96GB)	The one machine here that runs 200B-plus models on its own, and far more than most home users need.	apple.com ↗
Framework DesktopMini PC	Cheapest 128GB, repair-friendly	not published	The low-cost 128GB route for Linux tinkerers, strongest on mixture-of-experts models.	frame.work ↗
GMKtec EVO-X2Mini PC	Ready-made 128GB with Windows	not published	A plug-in 128GB Strix Halo box; good for mixture-of-experts models, slow on dense 70B.	gmktec.com ↗
MINISFORUM MS-S1 MAXMini PC	128GB workstation, fast networking	$3,919–$4,199 (128GB, sale prices shown; regular $4,899–$5,249)	The 128GB mini PC to pick if you want expansion and fast networking, with prices that move often.	store.minisforum.com ↗
NVIDIA DGX SparkMini PC	CUDA development on a desk	From $4,999 (64GB)	Buy it for the NVIDIA software stack, not for value: the 128GB mini PCs cost far less.	nvidia.com ↗

Every local hardware entry

GPU · NVIDIA
GeForce RTX 5060 Ti 16GB
Price
$429 (NVIDIA starting price; retail prices vary)
Runs
Up to ~14B models at 4-bit; ~24B fits with short context
People adding a first local-AI card to an existing desktop on a budget
Details nvidia.com ↗
GPU · NVIDIA
GeForce RTX 5090
Price
$1,999 (NVIDIA starting price, Founders Edition; partner cards cost more)
Runs
Up to ~32B models at 8-bit, ~70B only with heavy quantization or CPU offload
Enthusiasts who want the fastest single consumer card and can absorb the price
Details nvidia.com ↗
GPU (used) · NVIDIA
GeForce RTX 3090 (used)
Price
varies (used only)
Runs
Up to ~32B models at 4-bit; 70B too slow on one card
Tinkerers who want 24GB of memory for the least money and are fine buying used
Details nvidia.com ↗
GPU (used) · NVIDIA
GeForce RTX 4090 (used)
Price
varies (mostly used)
Runs
Up to ~32B models at 4–8-bit; 70B too slow on one card
People who want fast 24GB performance without paying RTX 5090 prices
Details nvidia.com ↗
GPU · AMD
AMD Radeon AI PRO R9700
Price
$1,299 (AMD MSRP)
Runs
Up to ~32B models at 4–8-bit; 70B only with heavy quantization
Linux-comfortable builders who want 32GB on one new card for less than an RTX 5090
Details amd.com ↗
GPU · Intel
Intel Arc Pro B60
Price
not published (sold through board makers; prices vary)
Runs
Up to ~30B models at 4-bit; best at 14B
Budget builders who want 24GB new with a warranty and can handle setup
Details intel.com ↗
Desktop · Apple
Mac mini (M5 Pro)
Price
From $1,699 (24GB; 48GB and 64GB cost more)
Runs
Up to ~32B models at 4-bit with 64GB; 70B too tight
Mac users who want a small, silent box for mid-size local models
Details apple.com ↗
Laptop · Apple
MacBook Pro (M5 Max)
Price
From $3,599 (14-inch, 36GB); 128GB costs more
Runs
With 128GB: ~120B mixture-of-experts models and 70B at 4-bit
People who need large local models on a laptop they can carry
Details apple.com ↗
Desktop · Apple
Mac Studio (M5 Max)
Price
From $2,499; 128GB configurations cost more
Runs
With 128GB: gpt-oss-120b and 70B models at 4-bit
People who want the fastest, quietest 128GB box for local models
Details apple.com ↗
Desktop · Apple
Mac Studio (M5 Ultra)
Price
From $5,499 (96GB); larger memory costs more
Runs
With 256–512GB: 200B–400B-plus models at 4-bit
Professionals who want the largest open models on one desk machine
Details apple.com ↗
Mini PC · Framework
Framework Desktop
Price
$3,149 (128GB mainboard only; full system costs more)
Runs
With 128GB: gpt-oss-120b and 70B models at 4-bit, slowly for 70B
Tinkerers who want a 128GB Strix Halo machine from a repair-friendly maker
Details frame.work ↗
Mini PC · GMKtec
GMKtec EVO-X2
Price
$3,499.99–$3,649.99 (128GB, GMKtec US store)
Runs
With 128GB: gpt-oss-120b and 70B models at 4-bit, slowly for 70B
Buyers who want a ready-made 128GB Strix Halo box with Windows installed
Details gmktec.com ↗
Mini PC · MINISFORUM
MINISFORUM MS-S1 MAX
Price
varies (Minisforum's own listings show different prices)
Runs
With 128GB: gpt-oss-120b and 70B models at 4-bit, slowly for 70B
Small-office users who want a Strix Halo workstation with fast networking
Details store.minisforum.com ↗
Mini PC · NVIDIA
NVIDIA DGX Spark
Price
$4,999 (64GB) or $6,950 (128GB)
Runs
With 128GB: gpt-oss-120b and 70B models at 4-bit
Developers who need NVIDIA's CUDA software stack in a desktop box
Details nvidia.com ↗

Questions people ask

How much memory do I need to run an LLM locally?: Roughly the model's parameter count in gigabytes at 4-bit, plus room for context: 16GB runs 8–14B models, 24GB runs 27–32B, and 128GB runs 70B dense models and 120B mixture-of-experts models. If a model does not fit in memory it either fails to load or runs far too slowly to use.
Is a Mac or a PC with a graphics card better for local AI?: A PC with an NVIDIA card is faster per dollar on models that fit in the card's memory and has the widest software support. A Mac with 64GB or more fits much larger models in one quiet box, but runs them more slowly. Pick the PC for speed on mid-size models and the Mac for size and silence.
Are the 128GB mini PCs worth it?: They are the cheapest way to fit 70B-class and mixture-of-experts models on one machine. They are slow on dense 70B models and fast enough on mixture-of-experts models such as gpt-oss-120b. If you mostly want 30B-and-under models, a single 24GB card is cheaper and faster.

How we pick: a tool is listed because people use it to run open models, never because it pays. No link on this page pays us today; if that changes it will be marked here. How links work on this site. New to open models? Start with Run your own AI, or read the hardware guide.