💻GGUF
Also known as GGUF File, GGUF Format, .gguf
One file that holds a whole model. It is ready to run on your own PC.
Think of it like
Think of it like a lunch box with a label. Everything you need is packed inside one box.
Example
Emma finds a model in several GGUF files on a model hub. She picks the Q4_K_M one at about 5 GB, because it fits her laptop and keeps good quality.
Why it matters
One file with all you need in it is easy to share, download and run.
Definition
Example code
text
model-8B-Instruct-Q4_K_M.ggufCommon mistake
Key takeaways
- 1One file. Everything inside.
- 2Weights and settings. Tokenizer too.
- 3Q label. Tells the quantization.