Skip to content
BENIBIX Learn

Gradient Boosting

AI📈 Machine LearningClassic ML Algorithms

Also known as Gradient Boosted Trees, Boosted Trees, GBM

Many small trees built in a row. Each one fixes what the last got wrong.

Think of it like

Think of it like a team. Each new player fixes what the last one got wrong.

Example

Lucy's team predicts how many meals a restaurant will sell each day. Gradient boosting with 500 small trees beats every other model they try.

Why it matters

It wins many contests on table data. It also runs inside many real products, and good tools make it easy to use.

Definition

Gradient boosting builds many small decision trees, one after another. The first tree makes rough guesses. Each new tree is trained to fix the errors that the trees before it left behind. Their answers are added up to give the final answer. On table data, like sales or loans, it is often the most accurate method there is.
Gradient boosting is the bottom row. The trees work in a chain, each one cleaning up after the last.

Common mistake

Adding trees forever. After a point, new trees start fixing noise in the training data, and the model gets worse on new data.

Key takeaways

  1. 1
    Trees in a row. Built one after another.
  2. 2
    Each fixes the last. Trained on what is still wrong.
  3. 3
    Strong on tables. Often the most accurate.

Compare

Learn it properly