📈Gradient Descent
Also known as Stochastic Gradient Descent, SGD
How training improves a model. Each small step nudges the weights downhill.
Think of it like
Think of it like walking down a hill in fog. You feel which way is down and take a step.
Example
Jack's model starts with random weights and a loss of 8.0. After 1,000 steps of gradient descent, the loss is down to 0.3.
Why it matters
It is how almost every neural network learns. Without it, there would be no way to tune billions of weights.
Definition
Common mistake
Key takeaways
- 1Find downhill. Which way lowers the loss.
- 2Take a small step. Nudge all weights a little.
- 3Repeat. Thousands or millions of times.