💻llama.cpp
Also known as llama-cpp, llama cpp
A free engine that runs AI models on most PCs and laptops.
Think of it like
Think of it like a small, tough motor that fits in just about any car. It runs on most things.
Example
Lucy builds llama.cpp on an old office PC with no graphics card. A small 4-bit model writes about 8 tokens per second, which is enough for her notes tool.
Why it matters
It made local AI work on normal machines. Much of the local AI world is built on it.
Definition
Example code
bash
llama-cli -m model-Q4_K_M.gguf -p "Write a haiku about rain."Common mistake
Key takeaways
- 1Runs almost anywhere. CPU, GPU, Apple.
- 2Made GGUF. Its own format.
- 3Inside many apps. Often unnamed.