# Model Collapse

> Models trained on other models' text get worse each round.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** AI Model Collapse, Recursive Training Collapse

## Think of it like

It is like a copy of a copy of a copy. Each one is a bit more blurry.

## Definition

Model collapse is the harm that builds up when models learn, round after round, from text made by earlier models. Each model copies the common patterns well but drops some rare ones. Over many rounds, the rare parts of the first data fade away. The output grows narrow and dull. A 2024 paper in Nature showed it in language models and in other kinds of model. Keeping real human data in the mix, and filtering made-up data, lowers the risk.

## Example

Researchers train each new model only on text from the one before. After a few rounds, the rare words and odd facts are gone, and all the text sounds alike.

## Why it matters

More and more of the web is written by AI. Future models trained on it could take in this slow decay.

## Common mistake

Thinking all made-up training data is harmful. Checked and mixed with real data, it is widely used.

## Three principles

- **Copies of copies.** Each round loses a little.
- **Rare things go first.** The edges of the data fade.
- **Keep real data.** Fresh human data slows the decay.

## Related terms

- [Synthetic Data](https://learn.benibix.com/term/ai/synthetic-data)
- [Pretraining](https://learn.benibix.com/term/ai/pretraining)
- [Catastrophic Forgetting](https://learn.benibix.com/term/ai/catastrophic-forgetting)
- [Knowledge Distillation](https://learn.benibix.com/term/ai/knowledge-distillation)
- [Hallucination (AI)](https://learn.benibix.com/term/ai/hallucination)

## Told apart from

- [Catastrophic Forgetting vs Model Collapse](https://learn.benibix.com/compare/catastrophic-forgetting-vs-model-collapse): People think catastrophic forgetting and model collapse are one problem, because both make a model worse after training. They are not. Forgetting happens inside one model: new training writes over skills it had. Collapse builds up across many models, each one trained on text made by the one before. The fix for the first is to test old skills. The fix for the second is to keep real human data.

## Taught in

- [Knowledge Distillation](https://learn.benibix.com/learn/ai-llms/knowledge-distillation): LLMs, 10/15

![Four curves in a row, one per round of training. The first is wide, with long thin edges for rare things. Each next curve is narrower, until the last is a tall thin spike.](https://learn.benibix.com/diagrams/ai9-model-collapse.svg)

---

Source: https://learn.benibix.com/term/ai/model-collapse
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
