Skip to content
BENIBIX Learn

Backpropagation

AI📈 Machine LearningNeural Networks

Also known as Backprop, Backward Pass

It traces the error back through the layers to see which weights to fix.

Think of it like

Think of it like a coach after a lost game. The coach goes back through the game to see who caused each miss.

Example

Lucy's network says dog for a cat photo. Backpropagation traces the miss back through all 12 layers. It finds the weights that pushed hardest toward dog.

Why it matters

It makes training deep networks possible. Without a fast way to share out the blame, big networks could not learn.

Definition

Backpropagation finds how much each weight in a network added to its error. After the network makes a guess, the loss is measured at the output. Then the error is passed backwards, layer by layer, toward the input. At each layer, it works out how much each weight should change. Gradient descent then uses those numbers to update the weights.
Backpropagation is the step that runs backwards. It works out the changes before the weights move.

Common mistake

Thinking it is the same thing as gradient descent. This step works out the changes. Gradient descent then makes them.

Key takeaways

  1. 1
    Start at the error. Measured at the output.
  2. 2
    Go backwards. Layer by layer to the input.
  3. 3
    Share the blame. How much each weight should change.

Compare

Learn it properly