Skip to content
BENIBIX Learn

Early Stopping

Also known as Early Stop, Stop Training Early

Stop the training when the score on new data stops going up.

Think of it like

It is like taking cookies out of the oven when they look right. Wait more and they burn.

Example

Jack plans 100 epochs. Validation loss is lowest at epoch 23 and rises for 5 more, so training stops at epoch 28 and keeps epoch 23.

Why it matters

It guards against overfitting for free. It also saves hours of wasted training time.

Definition

Early stopping ends training once the model stops getting better on new data. While training, you check the loss on the validation set after each epoch. At first it falls along with the training loss. Then it flattens or starts to rise, which means overfitting has begun. You stop there and keep the best version. A patience setting waits a few epochs first, in case it is just a bump.
Training ends at the point marked stop here. That is where the test loss turns up.

Common mistake

Stopping on the first bump. Loss jumps around a little, so wait a few epochs before you stop.

Key takeaways

  1. 1
    Watch new data. Track validation loss each epoch.
  2. 2
    Stop at the turn. When it stops falling.
  3. 3
    Keep the best. Save the version from the low point.

Learn it properly