Python✓ Mathematical
◆ The PatternSmart hyperparameter tuning — TPE, pruning, study visualisation, any framework
Optuna is a hyperparameter optimisation framework that uses TPE (Tree-structured Parzen Estimator) to intelligently search the parameter space. It supports pruning (killing bad trials early) and integrates with scikit-learn, XGBoost, PyTorch, and more.
// Optuna search space exploration
# Optuna — hyperparameter optimisation import optuna from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import cross_val_score def objective(trial): params = { 'n_estimators': trial.suggest_int('n_estimators', 50, 500), 'max_depth': trial.suggest_int('max_depth', 3, 20), 'min_samples_split': trial.suggest_int('min_samples_split', 2, 20), 'max_features': trial.suggest_categorical( 'max_features', ['sqrt', 'log2', None] ), } model = RandomForestClassifier(**params) return cross_val_score(model, X, y, cv=5, scoring='f1').mean() study = optuna.create_study(direction='maximize') study.optimize(objective, n_trials=100) print(study.best_params) optuna.visualization.plot_optimization_history(study)
| Feature | What it does |
|---|---|
| TPE sampler | Bayesian sampling — learns from previous trials |
| Pruning | Early stopping for bad trials (MedianPruner) |
| Study dashboard | optuna-dashboard for real-time monitoring |
| Multi-objective | Optimise accuracy AND speed simultaneously |
vs GridSearch: GridSearch tests every combination (exponential). Optuna uses Bayesian optimization — it learns which regions are promising and explores them more.