Machine Learning · Chapter 24 of 40

Hyperparameter Tuning

HYPERPARAMETERS are settings you choose BEFORE training (e.g. tree depth, learning rate). Find good ones with GRID SEARCH or RANDOM SEARCH — combined with cross-validation.

Advanced: Bayesian optimisation (Optuna, Hyperopt).

Example 1 (python)

from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(model, {'max_depth': [3,5,10]}, cv=5).fit(X, y)
print(grid.best_params_)

Output

{'max_depth': 5}

Try all combinations, cross-validated.

Example 2 (python)

from sklearn.model_selection import RandomizedSearchCV
# Sample N random combos instead of exhaustive search

Faster when the grid is huge.

Key points

Hyperparameters are set BEFORE training.
Use GridSearch or RandomSearch.
Always with cross-validation.
Optuna/Hyperopt for smarter search.

💡 Note: Random search often finds nearly-best parameters much faster than exhaustive grid search.

Hyperparameter Tuning

Key points

📝 Quick Quiz