Random Forest vs Gradient Boosting
Both are made of many decision trees, so people treat them as the same idea. The trees work in different ways. In a random forest, the trees are built side by side and vote. In gradient boosting, they are built in a row, and each one fixes what the last got wrong.
Random Forest
Lots of decision trees, each a bit different. They all vote on the answer.
A random forest is a model made of many decision trees that vote. Each tree is trained on a random sample of the data and looks at a random mix of features. So each tree makes different mistakes. When a new example comes in, every tree gives an answer and the most common one wins. The crowd is usually far more accurate than any single tree.
Gradient Boosting
Many small trees built in a row. Each one fixes what the last got wrong.
Gradient boosting builds many small decision trees, one after another. The first tree makes rough guesses. Each new tree is trained to fix the errors that the trees before it left behind. Their answers are added up to give the final answer. On table data, like sales or loans, it is often the most accurate method there is.
In one line
- Random Forest
- Lots of decision trees, each a bit different. They all vote on the answer.
- Gradient Boosting
- Many small trees built in a row. Each one fixes what the last got wrong.