Skip to content
BENIBIX Learn

Bias-Variance Tradeoff

Also known as Bias-Variance Trade-off, Bias and Variance

Too simple misses the same way each time. Too complex jumps around.

Think of it like

Think of it like throwing darts. You can miss to one side each time, or hit all over the board.

Example

Jack tries trees with depth 2, 6 and 20. Depth 2 underfits, depth 20 overfits, and depth 6 scores best on new data.

Why it matters

It explains why the most complex model is not always the best. Good models sit in the middle.

Definition

The bias-variance tradeoff is the balance between too simple and too complex. Bias is error from a model that is too simple, so it misses the same way every time. Variance is error from a model that changes too much when the data changes a little. Lowering one often raises the other. The goal is the middle, where the total error is lowest.
High bias lands tight but off to one side. High variance scatters. The goal is tight and on target.

Common mistake

Thinking bias here means unfairness. In this phrase, it only means a model that misses the same way every time.

Key takeaways

  1. 1
    Bias. Too simple, misses the same way.
  2. 2
    Variance. Too touchy, changes with the data.
  3. 3
    Find the middle. Where the total error is lowest.

Learn it properly