Run your own AI · Run your first local model

Run your first local model

thecontinuitylayer.com/open/learn/first-local-model

Checked 6 November 2026. US dollars. No affiliate links.

Two ways in. LM Studio is an app with buttons: the easiest start. Ollama is a small program you talk to in a terminal: the best base for building on later. Pick one; you can add the other any time.

By the end

A model running on your own computer, answering you with the internet switched off.

Before you start

Know how much memory you have. On Windows: Settings, System, About. On a Mac: Apple menu, About This Mac. That number decides which model to download; the model finder shows what fits. Leave about 10GB of free disk space.

Way 1: LM Studio (no terminal)

1
Download LM Studio from lmstudio.ai and install it. It needs an Apple Silicon Mac on macOS 14 or newer, or a Windows or Linux PC; 16GB of memory is recommended, and 8GB works with small models.
2
Open the Discover tab and search for gemma 4. On 8GB pick the E4B; on 16GB pick the 12B. LM Studio marks the downloads that are likely to fit.
3
Click Download and wait. A first model is about 4–8GB.
4
Go to the Chat tab, choose the model at the top, and type a question.
5
Turn off Wi-Fi and ask another. It still answers: that is the point.

Way 2: Ollama (a terminal, then anything)

1
Install Ollama from ollama.com/download (Mac, Windows or Linux).

Open a terminal (Terminal on a Mac, PowerShell on Windows) and run a model that fits your memory:

ollama run gemma4:e4b      # 8GB machines
ollama run gemma4:12b      # 16GB machines
ollama run qwen3.8:27b     # 24GB graphics card or 32GB Mac

3
The first run downloads the model, then gives you a prompt. Type a question and press Enter. Type /bye to leave.

See what you have, and switch:

ollama list                # models on this machine
ollama run gpt-oss:20b     # a different model
ollama rm gemma4:e4b       # free the disk space

Add a chat window to Ollama

The terminal works, but most people want a window with history and file uploads. The simplest is Jan, a desktop app that runs models itself and can also use Ollama. For a ChatGPT-style page in your browser that the whole house can use, Open WebUI connects to Ollama and adds document chat, memory and user accounts:

pip install open-webui
open-webui serve           # then open http://localhost:8080

Open WebUI recommends Python 3.12; a Docker image is also offered. Every app is compared in local AI apps.

If it is slow or will not load

Nine times in ten the model is too big for your memory. Pick the next size down. The full list of fixes is in Why is it slow or broken?

← Local or hosted: the honest trade-offs Next: Model sizes and quantization →