Backpropagation vs Gradient Descent
People often use the two names for the same thing. They are two steps that work as a pair. Backpropagation works out how much each weight added to the error. Gradient descent then uses those numbers to move every weight a small step. One finds the changes, and the other makes them.
Backpropagation
It traces the error back through the layers to see which weights to fix.
Backpropagation finds how much each weight in a network added to its error. After the network makes a guess, the loss is measured at the output. Then the error is passed backwards, layer by layer, toward the input. At each layer, it works out how much each weight should change. Gradient descent then uses those numbers to update the weights.
Gradient Descent
How training improves a model. Each small step nudges the weights downhill.
Gradient descent lowers a model's loss one small step at a time. At each step, it works out which way to nudge every weight so the loss goes down. Then it moves all the weights a little in that direction. It repeats this thousands or millions of times. Most training today uses a version that looks at a small batch of data per step.
In one line
- Backpropagation
- It traces the error back through the layers to see which weights to fix.
- Gradient Descent
- How training improves a model. Each small step nudges the weights downhill.