# llama.cpp

> A free engine that runs AI models on most PCs and laptops.

**Field:** Artificial Intelligence  
**Category:** Running Locally  
**Also known as:** llama-cpp, llama cpp

## Think of it like

Think of it like a small, tough motor that fits in just about any car. It runs on most things.

## Definition

llama.cpp is a free, open-source inference engine that runs language models on everyday machines. It first came out in March 2023. It is written in C and C++ and needs very few other parts. It runs on CPUs, on many kinds of graphics cards and on Apple chips. It reads models in the GGUF format, which it made. Many local AI apps use it inside, often without showing its name.

## Example

Lucy builds llama.cpp on an old office PC with no graphics card. A small 4-bit model writes about 8 tokens per second, which is enough for her notes tool.

## Example code

```bash
llama-cli -m model-Q4_K_M.gguf -p "Write a haiku about rain."
```

Run a GGUF model with one prompt from the command line.

## Why it matters

It made local AI work on normal machines. Much of the local AI world is built on it.

## Common mistake

Using a very old build. It changes fast, and new models often need a recent version to load at all.

## Three principles

- **Runs almost anywhere.** CPU, GPU, Apple.
- **Made GGUF.** Its own format.
- **Inside many apps.** Often unnamed.

## Related terms

- [Inference Engine](https://learn.benibix.com/term/ai/inference-engine)
- [GGUF](https://learn.benibix.com/term/ai/gguf)
- [Ollama](https://learn.benibix.com/term/ai/ollama)
- [LM Studio](https://learn.benibix.com/term/ai/lm-studio)

## Told apart from

- [Ollama vs llama.cpp](https://learn.benibix.com/compare/ollama-vs-llama-cpp): People think Ollama and llama.cpp are two rival engines, and they are not. llama.cpp is the engine. It loads a GGUF file and runs the maths, with a long list of flags to set. Ollama is a tool built on the same core. It finds a model by name, picks the settings for you and keeps a small server running. Pick llama.cpp for control, and Ollama for ease.

![llama.cpp reads a GGUF file and runs it on many kinds of hardware: a laptop CPU, a graphics card or an Apple chip.](https://learn.benibix.com/diagrams/lo-llama-cpp.svg)

---

Source: https://learn.benibix.com/term/ai/llama-cpp
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
