# Regularization

> Ways to stop a model from memorising. They keep it simpler.

**Field:** Artificial Intelligence  
**Category:** Machine Learning  
**Also known as:** Regularisation, L1 Regularization, L2 Regularization, Weight Decay

## Think of it like

Think of it like a rule that says keep your answer short. It stops you from rambling.

## Definition

Regularization is any method that stops a model from fitting its training data too closely. A common way adds a small cost to the loss when the weights grow big. This pushes the model toward simple answers. Other ways are dropout and stopping training early. The aim is a model that does better on new data. Its training score may drop a bit.

## Example

Emma's model overfits, with 98 percent on training and 75 on validation. She adds a penalty on big weights. Training drops to 92, but validation climbs to 86.

## Why it matters

It is the main tool against overfitting. Almost every real model uses some kind of it.

## Common mistake

Adding so much that the model underfits. Too strong a penalty makes the model too simple to learn.

## Three principles

- **Fights overfitting.** Stops too close a fit.
- **A penalty on size.** Big weights cost extra.
- **Simpler is safer.** Better on new data.

## Related terms

- [Overfitting](https://learn.benibix.com/term/ai/overfitting)
- [Dropout](https://learn.benibix.com/term/ai/dropout)
- [Weight (Neural Network)](https://learn.benibix.com/term/ai/weight)
- [Bias-Variance Tradeoff](https://learn.benibix.com/term/ai/bias-variance-tradeoff)
- [Early Stopping](https://learn.benibix.com/term/ai/early-stopping)

## Taught in

- [Regularization and Dropout](https://learn.benibix.com/learn/ai-machine-learning/regularization-and-dropout): Machine Learning, 17/18

![A chart of loss over epochs. Training loss keeps falling. Validation loss falls, then turns up again. The point where it turns is marked stop here.](https://learn.benibix.com/diagrams/nn-overfit-curve.svg)

---

Source: https://learn.benibix.com/term/ai/regularization
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-17
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
