llama.cpp
thecontinuitylayer.com/tools/local-ai-apps/llama-cppTinkerers who want the most control and the newest features first
Inference engine · ggml-org
llama.cpp is the C/C++ engine that made running models on ordinary computers practical. It runs GGUF model files on processors and graphics cards from every major maker, with no other dependencies.
LM Studio, Jan, KoboldCpp and others build on it. Used directly, it gives you a server with an OpenAI-compatible API and a built-in web chat, and full control over every setting. The trade-off is more to learn: you pick the files and the flags yourself.
Specs
| Price | Free and open source (MIT) |
|---|---|
| Runs | GGUF models on almost any hardware: CPU, NVIDIA, Apple, AMD, Intel, Vulkan |
| Hardware | CPU (x86, ARM), CUDA, Metal, Vulkan, SYCL, HIP and more |
| Difficulty | Advanced: terminal and settings |
| Format | GGUF |
| API | OpenAI-compatible server with a web UI |
| License | MIT |
Good for
- Squeezing the most out of unusual or older hardware
- Trying new models the day they land
Not for
- Beginners (start with LM Studio or Ollama)
See also
We last reviewed llama.cpp on 6 November 2026. Prices and specs change; check github.com before you buy or sign up. Figures we could not confirm on the vendor's own page read “not published” or “varies”.
Vendor source: https://github.com/ggml-org/llama.cpp