08 — Understand Your Features

Permutation Importance#

Features✓ Mathematical
◆ The PatternShuffle a feature, measure the damage — a model-agnostic way to rank feature importance

Permutation importance randomly shuffles one feature at a time and measures how much the model's score drops. Big drop = important feature. It works with any model and requires no retraining.

PIᵢ = Scoreoriginal − Scoreshuffled(i)
PI = importance of feature i. Higher = more important. Negative = feature hurts the model.
// Interactive — shuffle features, see score drop
Features5
# Python — permutation importance
from sklearn.inspection import permutation_importance

result = permutation_importance(
    model, X_test, y_test,
    n_repeats=30, random_state=42,
    scoring='accuracy'
)

# Sorted importance
for i in result.importances_mean.argsort()[::-1]:
    print(f"{features[i]}: {result.importances_mean[i]:.3f}")
Correlated features trap: If two features are correlated, shuffling one leaves the other intact — importance is split between them. Consider using SHAP or drop-column importance for correlated features.
← Previous
SHAP Values
Open in the full reader, with the topic sidebar →