📈Hyperparameter Tuning
Also known as Hyperparameter Optimization, Hyperparameter Search, Grid Search
Train a model many times, each with new settings. Keep the best.
Think of it like
It is like finding the best oven heat for a cake. You bake a few and taste each one.
Example
Jack tries three tree depths and three learning rates. That makes nine models. The one with depth 6 and rate 0.1 scores best on his validation set.
Why it matters
The same model can go from poor to good with better settings. Tuning is often the cheapest way to gain that.
Definition
Example code
python
from sklearn.model_selection import GridSearchCV
search = GridSearchCV(model, {"max_depth": [3, 6, 9]}, cv=5)
search.fit(X, y)
print(search.best_params_)
# {'max_depth': 6}Common mistake
Key takeaways
- 1Try many. Each setting mix is one training run.
- 2Score fairly. Use the validation set, not the test set.
- 3Search smart. Random search often beats a full grid.