# Validation Set

> Data kept out of training. You use it to pick the best settings.

**Field:** Artificial Intelligence  
**Category:** Machine Learning  
**Also known as:** Dev Set, Development Set, Holdout Set

## Think of it like

Think of it like a practice quiz. You use it to see what to study more.

## Definition

The validation set is a part of the data kept out of training and used to tune the model. After each try, you check the model on it. Then you change a setting and try again. This lets you pick the best settings without touching the test set. Because you look at it many times, its score gets a little too hopeful.

## Example

Jack trains five versions of his model with different settings. He checks all five on the validation set and keeps the one that scores best.

## Why it matters

It lets you make the model better. The test set stays clean for one fair final check.

## Common mistake

Reporting the validation score as the final score. You chose the model by it, so it flatters the model a little.

## Three principles

- **Kept out of training.** The model does not learn from it.
- **Used for tuning.** Compare settings on it.
- **Checked many times.** So its score runs a bit high.

## Related terms

- [Training Set](https://learn.benibix.com/term/ai/training-set)
- [Test Set](https://learn.benibix.com/term/ai/test-set)
- [Hyperparameter](https://learn.benibix.com/term/ai/hyperparameter)
- [Data Leakage](https://learn.benibix.com/term/ai/data-leakage)
- [Cross-Validation](https://learn.benibix.com/term/ai/cross-validation)

## Told apart from

- [Validation Set vs Test Set](https://learn.benibix.com/compare/validation-set-vs-test-set): People often think these are two names for the same held-out data. Both are kept out of training, but they have different jobs. The validation set is checked again and again while you tune. The test set is opened once, at the end. Tuning on the test set makes your final score a lie.

## Taught in

- [Training and Test Sets](https://learn.benibix.com/learn/ai-machine-learning/training-and-test-sets): Machine Learning, 2/18

![One long bar of data cut into three parts: a big training set, a smaller validation set and a smaller test set. A dashed arrow from the test set back into the training set is marked data leakage.](https://learn.benibix.com/diagrams/ml-data-splits.svg)

---

Source: https://learn.benibix.com/term/ai/validation-set
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-17
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
