Skip to content
BENIBIX Learn

Regularization

Also known as Regularisation, L1 Regularization, L2 Regularization, Weight Decay

Ways to stop a model from memorising. They keep it simpler.

Think of it like

Think of it like a rule that says keep your answer short. It stops you from rambling.

Example

Emma's model overfits, with 98 percent on training and 75 on validation. She adds a penalty on big weights. Training drops to 92, but validation climbs to 86.

Why it matters

It is the main tool against overfitting. Almost every real model uses some kind of it.

Definition

Regularization is any method that stops a model from fitting its training data too closely. A common way adds a small cost to the loss when the weights grow big. This pushes the model toward simple answers. Other ways are dropout and stopping training early. The aim is a model that does better on new data. Its training score may drop a bit.
Regularization keeps the two lines close. Then the validation loss does not turn back up so soon.

Common mistake

Adding so much that the model underfits. Too strong a penalty makes the model too simple to learn.

Key takeaways

  1. 1
    Fights overfitting. Stops too close a fit.
  2. 2
    A penalty on size. Big weights cost extra.
  3. 3
    Simpler is safer. Better on new data.

Learn it properly