# GGUF

> One file that holds a whole model. It is ready to run on your own PC.

**Field:** Artificial Intelligence  
**Category:** Running Locally  
**Also known as:** GGUF File, GGUF Format, .gguf

## Think of it like

Think of it like a lunch box with a label. Everything you need is packed inside one box.

## Definition

GGUF is a file format that keeps a whole model in one file, ready to run on your own machine. It came out in 2023. It took the place of an older format called GGML. A GGUF file holds the weights, often shrunk, plus all that is needed to run them: the model's settings, its tokenizer and its prompt format. Names like Q4_K_M or Q8_0 in the file name tell you how it was shrunk. Many local tools read GGUF files.

## Example

Emma finds a model in several GGUF files on a model hub. She picks the Q4_K_M one at about 5 GB, because it fits her laptop and keeps good quality.

## Example code

```text
model-8B-Instruct-Q4_K_M.gguf
```

A common file name. It gives the model, its size, and Q4_K_M, which means 4 bits per weight.

## Why it matters

One file with all you need in it is easy to share, download and run.

## Common mistake

Picking a GGUF file by size alone. The Q label tells you the quality trade, so read it before you download.

## Three principles

- **One file.** Everything inside.
- **Weights and settings.** Tokenizer too.
- **Q label.** Tells the quantization.

## Related terms

- [Quantization (AI)](https://learn.benibix.com/term/ai/quantization)
- [Model Weights File](https://learn.benibix.com/term/ai/model-weights)
- [llama.cpp](https://learn.benibix.com/term/ai/llama-cpp)
- [GGUF Quantization Type](https://learn.benibix.com/term/ai/gguf-quantization-type)
- [Safetensors](https://learn.benibix.com/term/ai/safetensors)

## Told apart from

- [Safetensors vs GGUF](https://learn.benibix.com/compare/safetensors-vs-gguf): People think safetensors and GGUF are just two names for the same kind of model file. They are made for different jobs. Safetensors holds the full weights, most often at 16 bits, and it is how most models are shared on hubs. GGUF packs a model for local apps, most often shrunk to fewer bits. You often turn the first into the second.

![One GGUF file opened up. A header holds the model's details and its tokenizer, and the rest holds the shrunken weights.](https://learn.benibix.com/diagrams/lo-gguf.svg)

---

Source: https://learn.benibix.com/term/ai/gguf
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
