# Gradient Boosting

> Many small trees built in a row. Each one fixes what the last got wrong.

**Field:** Artificial Intelligence  
**Category:** Machine Learning  
**Also known as:** Gradient Boosted Trees, Boosted Trees, GBM

## Think of it like

Think of it like a team. Each new player fixes what the last one got wrong.

## Definition

Gradient boosting builds many small decision trees, one after another. The first tree makes rough guesses. Each new tree is trained to fix the errors that the trees before it left behind. Their answers are added up to give the final answer. On table data, like sales or loans, it is often the most accurate method there is.

## Example

Lucy's team predicts how many meals a restaurant will sell each day. Gradient boosting with 500 small trees beats every other model they try.

## Why it matters

It wins many contests on table data. It also runs inside many real products, and good tools make it easy to use.

## Common mistake

Adding trees forever. After a point, new trees start fixing noise in the training data, and the model gets worse on new data.

## Three principles

- **Trees in a row.** Built one after another.
- **Each fixes the last.** Trained on what is still wrong.
- **Strong on tables.** Often the most accurate.

## Related terms

- [Random Forest](https://learn.benibix.com/term/ai/random-forest)
- [Decision Tree](https://learn.benibix.com/term/ai/decision-tree)
- [Regression (Machine Learning)](https://learn.benibix.com/term/ai/regression)
- [Classification (Machine Learning)](https://learn.benibix.com/term/ai/classification)
- [XGBoost](https://learn.benibix.com/term/ai/xgboost)
- [Ensemble Learning](https://learn.benibix.com/term/ai/ensemble-learning)

## Told apart from

- [Random Forest vs Gradient Boosting](https://learn.benibix.com/compare/random-forest-vs-gradient-boosting): Both are made of many decision trees, so people treat them as the same idea. The trees work in different ways. In a random forest, the trees are built side by side and vote. In gradient boosting, they are built in a row, and each one fixes what the last got wrong.

## Taught in

- [Decision Trees and Forests](https://learn.benibix.com/learn/ai-machine-learning/decision-trees-and-forests): Machine Learning, 10/18

![Top row, random forest: three trees side by side each give a vote, and the most common vote wins. Bottom row, gradient boosting: three trees in a chain, each one fixing the errors left by the one before.](https://learn.benibix.com/diagrams/ml-forest-boosting.svg)

---

Source: https://learn.benibix.com/term/ai/gradient-boosting
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-17
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
