# Chinchilla Scaling Laws

> Train on about 20 words of text for each weight.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Chinchilla, Compute-Optimal Training, Chinchilla Optimal

## Think of it like

It is like a new library. It is no good until you buy books to fill it.

## Definition

The Chinchilla scaling laws are a rule for splitting a training budget between model size and data. They were published in 2022. They say a model should read about 20 tokens of text for each parameter. A model trained this way beat one four times its size that had read far less. Since then, labs train smaller models on much more text.

## Example

DeepMind's model Chinchilla had 70 billion parameters and read 1.4 trillion tokens, which is 20 for each one. A model with 10 billion would need about 200 billion.

## Why it matters

It showed that many big models of the time had read too little. It changed how labs spend their money.

## Common mistake

Treating 20 tokens per parameter as the best model to ship. It is the cheapest way to train. Labs now go far past it, because a smaller model is cheaper to run.

## Three principles

- **Grow both together.** Size and data rise as a pair.
- **About 20 to 1.** Tokens of text for each parameter.
- **Cheapest to train.** Not the cheapest to run.

## Related terms

- [Scaling Laws](https://learn.benibix.com/term/ai/scaling-laws)
- [Pretraining](https://learn.benibix.com/term/ai/pretraining)
- [Training Data](https://learn.benibix.com/term/ai/training-data)
- [Parameter Count](https://learn.benibix.com/term/ai/parameter-count)
- [Google DeepMind](https://learn.benibix.com/term/ai/google-deepmind)

## Taught in

- [Pretraining an LLM](https://learn.benibix.com/learn/ai-llms/pretraining-an-llm): LLMs, 7/15

![A chart with compute on the bottom axis and error on the side axis, both on a scale where each step is ten times bigger. The error falls along a straight line as compute grows.](https://learn.benibix.com/diagrams/llm-scaling-laws.svg)

---

Source: https://learn.benibix.com/term/ai/chinchilla-scaling-laws
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
