vLLM
thecontinuitylayer.com/tools/local-ai-apps/vllmTeams serving a model to many users at once
Serving engine · vLLM project
vLLM is an engine for serving language models fast, to many people at once. It is what you run on a rented GPU server when a model has to answer a team, an app or a business, not one person.
It offers an OpenAI-compatible server, plus Anthropic Messages API and gRPC support, and runs on NVIDIA, AMD and Intel graphics cards, with plugins for Google TPUs, Apple Silicon and others. It is installed with pip and configured from the command line.
Specs
| Price | Free and open source (Apache 2.0) |
|---|---|
| Runs | Most open models on NVIDIA, AMD and Intel GPUs, plus TPUs and others by plugin |
| Hardware | NVIDIA, AMD, Intel GPUs; TPU, Apple Silicon and others by plugin |
| Difficulty | Advanced: Linux, Python, GPU drivers |
| API | OpenAI-compatible, Anthropic Messages, gRPC |
| License | Apache 2.0 |
Good for
- Many users on one model
- Self-hosting on a rented GPU
Not for
- Personal use on a laptop (use Ollama or LM Studio)
See also
We last reviewed vLLM on 6 November 2026. Prices and specs change; check docs.vllm.ai before you buy or sign up. Figures we could not confirm on the vendor's own page read “not published” or “varies”.
Vendor source: https://github.com/vllm-project/vllm