# Scaling Laws

> Rules that say how much better a model gets as it grows.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Neural Scaling Laws, LLM Scaling Laws

## Think of it like

Think of it like a rule that says how much faster you run if you train more. You can plan ahead.

## Definition

Scaling laws are patterns that show how a model's error falls as it gets more weights, data and compute. On a chart where each step is ten times bigger, the error falls along a nearly straight line. So a lab can predict how good a big model will be from small test runs. A known 2022 result found about 20 training tokens per weight works best for a set budget.

## Example

A lab trains five small models and plots their error. The line tells them a 100B model trained on 2 trillion tokens should reach a set score, before they spend millions.

## Why it matters

They are why labs keep building bigger models. The gains were predictable enough to bet on.

## Common mistake

Thinking the line goes on forever with no cost. Each gain needs about ten times more compute than the last.

## Three principles

- **Error falls with scale.** Weights, data, compute.
- **A straight line.** On a ten-times scale.
- **Plan before spending.** Small runs predict big ones.

## Related terms

- [Pretraining](https://learn.benibix.com/term/ai/pretraining)
- [Parameter Count](https://learn.benibix.com/term/ai/parameter-count)
- [Loss Function](https://learn.benibix.com/term/ai/loss-function)
- [Test-Time Compute](https://learn.benibix.com/term/ai/test-time-compute)
- [Frontier Model](https://learn.benibix.com/term/ai/frontier-model)
- [AI Compute](https://learn.benibix.com/term/ai/ai-compute)
- [Emergent Abilities](https://learn.benibix.com/term/ai/emergent-abilities)

## Taught in

- [Pretraining an LLM](https://learn.benibix.com/learn/ai-llms/pretraining-an-llm): LLMs, 7/15

![A chart with compute on the bottom axis and error on the side axis, both on a scale where each step is ten times bigger. The error falls along a straight line as compute grows.](https://learn.benibix.com/diagrams/llm-scaling-laws.svg)

---

Source: https://learn.benibix.com/term/ai/scaling-laws
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
