📈Learning Rate
Also known as Step Size
How big each training step is. Too big jumps around, too small crawls.
Think of it like
Think of it like the size of your steps down a steep hill. Too big and you trip.
Example
Emma trains with a learning rate of 0.5, and the loss bounces wildly. She drops it to 0.001, and the loss falls smoothly.
Why it matters
It is often the single most important setting to tune. The wrong value can waste days of training.
Definition
Common mistake
Key takeaways
- 1Step size. How far each update moves.
- 2Too big. Jumps and never settles.
- 3Too small. Slow, and may stall.