Skip to content
BENIBIX Learn

Ensemble Learning

AI📈 Machine LearningClassic ML Algorithms

Also known as Ensemble Method, Model Ensemble, Bagging and Boosting

Many models each make a guess. Joined up, they beat most single models.

Think of it like

It is like asking a crowd to guess the sweets in a jar. The average guess beats most single guesses.

Example

Emma trains 200 small trees on random parts of her data. Each one guesses a house price, and the average is her final answer.

Why it matters

Joined models win many contests on table data. Their mistakes tend to cancel out.

Definition

Ensemble learning combines many models into one stronger one. Each model makes its own guess, and the guesses are joined. In bagging, many models train on random slices of data and then vote. A random forest works this way. In boosting, models train one after another, each fixing the last one's mistakes. Gradient boosting works this way.
Two ways to join models: a vote side by side, or a chain where each fixes the last.

Common mistake

Combining models that all make the same mistakes. An ensemble helps only when its models differ.

Key takeaways

  1. 1
    Many models. Each one makes its own guess.
  2. 2
    Bagging votes. Models train side by side.
  3. 3
    Boosting fixes. Each model fixes the last one.

Learn it properly