# Hyperparameter Tuning

> Train a model many times, each with new settings. Keep the best.

**Field:** Artificial Intelligence  
**Category:** Machine Learning  
**Also known as:** Hyperparameter Optimization, Hyperparameter Search, Grid Search

## Think of it like

It is like finding the best oven heat for a cake. You bake a few and taste each one.

## Definition

Hyperparameter tuning is the search for the best settings to train a model with. Settings like the learning rate or the depth of a tree are chosen before training starts. You train the model many times, each with different settings. Then you score each one on the validation set and keep the best. A grid search tries every mix on a list. A random search tries mixes picked by chance, and often finds a good one sooner.

## Example

Jack tries three tree depths and three learning rates. That makes nine models. The one with depth 6 and rate 0.1 scores best on his validation set.

## Example code

```python
from sklearn.model_selection import GridSearchCV
search = GridSearchCV(model, {"max_depth": [3, 6, 9]}, cv=5)
search.fit(X, y)
print(search.best_params_)
# {'max_depth': 6}
```

Three depths tried, each scored five times. The best one is printed at the end.

## Why it matters

The same model can go from poor to good with better settings. Tuning is often the cheapest way to gain that.

## Common mistake

Scoring the settings on the test set. Then the test set no longer gives an honest final score.

## Three principles

- **Try many.** Each setting mix is one training run.
- **Score fairly.** Use the validation set, not the test set.
- **Search smart.** Random search often beats a full grid.

## Related terms

- [Hyperparameter](https://learn.benibix.com/term/ai/hyperparameter)
- [Validation Set](https://learn.benibix.com/term/ai/validation-set)
- [Learning Rate](https://learn.benibix.com/term/ai/learning-rate)
- [Cross-Validation](https://learn.benibix.com/term/ai/cross-validation)
- [Test Set](https://learn.benibix.com/term/ai/test-set)
- [Automated Machine Learning (AutoML)](https://learn.benibix.com/term/ai/automl)

## Told apart from

- [Hyperparameter Tuning vs Fine-Tuning](https://learn.benibix.com/compare/hyperparameter-tuning-vs-fine-tuning): People think both kinds of tuning change the model, and only one does. Hyperparameter tuning picks the settings for training, like the learning rate, by training many times. Fine-tuning takes a model that is already trained and trains it more on new data. The first picks the recipe. The second keeps cooking.

## Taught in

- [Hyperparameter Tuning](https://learn.benibix.com/learn/ai-machine-learning/hyperparameter-tuning): Machine Learning, 12/18

![A three by three grid. Rows are learning rates 0.01, 0.1 and 0.3. Columns are tree depths 3, 6 and 9. Each cell holds a validation score, and the cell for rate 0.1 and depth 6 is marked best at 0.91.](https://learn.benibix.com/diagrams/ai3-tuning-grid.svg)

---

Source: https://learn.benibix.com/term/ai/hyperparameter-tuning
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-17
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
