Data Augmentation vs Synthetic Data
People think data augmentation makes synthetic data, and it does not. Augmentation takes a real example and changes it a bit, like a flipped photo. Synthetic data is made from scratch, often by a model or a program. One stretches the data you have. The other makes new data you never collected.
Data Augmentation
Make more examples by changing the ones you have. Flip them, crop them, tint them.
Data augmentation makes more training examples by changing the ones you have. For a photo, you can flip it, turn it, crop it or make it darker. For text, you can swap in words that mean the same. The label stays the same, because a flipped cat is still a cat. The model sees more variety without new data being collected. It helps the model cope with real, messy inputs.
Synthetic Data
Data made by a computer to train a model. It is not taken from real life.
Synthetic data is training data made by a computer, not collected from the real world. For language models, it is often text written by another model: questions, answers, worked steps or code. It is cheap and can fill gaps where real data is rare. But it must be checked. A model trained on its own mistakes can get worse over time.
In one line
- Data Augmentation
- Make more examples by changing the ones you have. Flip them, crop them, tint them.
- Synthetic Data
- Data made by a computer to train a model. It is not taken from real life.