Skip to content
BENIBIX Learn

Overfitting and Underfitting

Level 1 · How a machine learns

Why a model can be too simple, or learn its data too closely. How the gap between two scores shows it, and what bias and variance mean.

Course contentsLesson 3 of 18

After this lesson you can

  • Tell overfitting from underfitting
  • Read the gap between two scores
  • Explain the bias-variance tradeoff

12 mincovers 4 glossary entries

Worth reading first: Training and Test Sets

Too simple or too close: how a model misses

Overfitting is when a model learns its training data too closely, noise and all. Underfitting is when it is too simple to learn the pattern at all. An overfit model might score 99% on training rows and 70% on new ones. An underfit model scores badly on both.

What you want is generalization: doing well on new data the model never saw. That is the whole point of learning. The gap between the training score and the test score shows how well a model generalizes. A small gap is good news. A big gap means it memorised.

Overfitting comes from a model with lots of room and too little data. Give a model 1,000 parameters and only 50 rows, and it can bend to pass through every point. It then learns the odd rows, the typos and the flukes, as if they were the pattern. New data has new flukes, so those parts fail.

Underfitting is the other way round. A straight line drawn through prices that jump sharply in a city centre misses most houses, in training and in testing alike. The fix is more room: a richer model, better features or longer training.

The bias-variance tradeoff names this balance. Bias is error from a model too simple to follow the data, so it misses the same way every time. Variance is error from a model that bends too easily. A small change in the data changes it a lot. Lowering one usually raises the other.

Darts make it easy to see. High bias is a tight group of darts, all off to one side. High variance is darts spread all round the bull. The goal is tight and on target, and it sits in the middle, where the total error on new data is lowest.

People often add more and more model to chase a higher training score. Past a point, that only makes overfitting worse. Watch the score on held-out data instead. More data, a simpler model or stopping training sooner all shrink the gap.

Which fix fits

  1. 1

    If it overfits

    The training score is high and the new data score is much lower. Use more data, a simpler model, or stop sooner.

  2. 2

    If it underfits

    Both scores are low. Use a richer model, better features, or train for longer.

The middle line has learned the pattern. The right one has learned the dots.

Worked example

Lucy compares two spam models

Lucy's first spam model scores 99% on its training mail and 81% on new mail. The gap of 18 points says it memorised.

She trains a simpler one. It scores 96% on training and 95% on new mail, so that is the one she ships.

The words for this lesson4

Check yourself

A model scores 99% on its training data and 70% on new data. What is wrong?

Finished this lesson?

Marking it complete also marks its 4 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.