📈Optimizer (Machine Learning)
Also known as Adam Optimizer, Adam, Optimization Algorithm (Neural Networks)
The rule for how a network moves its weights at each step as it learns.
Think of it like
It is like a way to walk down a hill in fog. Some walk slow. Some speed up.
Example
Emma trains an image model with plain gradient descent, and it crawls. She switches to Adam, and the loss drops in half the epochs.
Why it matters
The optimizer decides how fast and how well a network learns. A poor choice can stall training.
Definition
Example code
python
import torch
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
loss.backward()
optimizer.step()Common mistake
Key takeaways
- 1Reads the gradient. Which way lowers the loss.
- 2Picks the step. How big, and for which weights.
- 3Adam is common. It adapts the step for each weight.