llama.cpp
Free, open-source (MIT) LLM inference in C/C++ on your own hardware.
AI APIs
What you get
Run open-weight GGUF models locally on CPU or GPU, with a CLI and an OpenAI-compatible HTTP server (llama-server).
How to get it
Install via a package manager or a prebuilt release from the GitHub repo.
What it requires
A computer with enough RAM or VRAM for the model you run.
The catch
No hosted service: speed and model size depend on your own hardware, and setup is more hands-on than Ollama or LM Studio.
From the official page · checked Nov 1, 2026