Skip to content
BENIBIX Learn

Synthetic Data

AI💬 LLMsLanguage Model Training

Also known as AI-Generated Training Data, Generated Data

Data made by a computer to train a model. It is not taken from real life.

Think of it like

Think of it like you make up quiz cards for a friend. You check each card first.

Example

Jack needs 10,000 maths word problems with worked answers. A model writes 30,000, a checker keeps the 12,000 whose answers are right, and those go into training.

Why it matters

Good real data is running short for the biggest models. Made data is how labs keep feeding them.

Definition

Synthetic data is training data made by a computer, not collected from the real world. For language models, it is often text written by another model: questions, answers, worked steps or code. It is cheap and can fill gaps where real data is rare. But it must be checked. A model trained on its own mistakes can get worse over time.
A model writes lots of examples. A filter keeps only the good ones for training.

Common mistake

Using it unchecked. Errors and odd habits in the made data get baked into the new model.

Key takeaways

  1. 1
    Made, not collected. Often by another model.
  2. 2
    Fills gaps. Where real data is rare.
  3. 3
    Must be checked. Or mistakes compound.

Compare

Learn it properly