The Toolkit

A model that predicts x₁ × x₂, where x₂ is +1 for half the rows and −1 for the other half. The partial dependence on x₁ is nearly flat — 0.08 … −0.08 across the grid — as if x₁ did nothing. The individual (ICE) curves have slopes of +1 and −1: x₁ matters a lot, in opposite directions. Averages hide interactions; ICE shows them.

• PDPs evaluate the model on combinations that may never occur when features are correlated; ALE plots avoid this (Apley & Zhu 2020).
• A flat PDP is not evidence that a feature is unimportant.
• Many ICE lines become unreadable; centre them, or cluster them.

rng = np.random.default_rng(5)
x2 = rng.choice([-1.0, 1.0], 1_000)                 # half the rows have x2 = -1, half +1
predict = lambda x1, x2: x1 * x2                     # a pure interaction
grid = np.linspace(-2, 2, 5)
pdp = [predict(g, x2).mean() for g in grid]          # average over the data
ice_slopes = sorted({float(predict(1, v) - predict(0, v)) for v in x2})

Assumes import numpy as np and import pandas as pd (and SciPy where imported). Each snippet simulates or makes up its own data, as the comments say.

# Python — PDP & ICE
from sklearn.inspection import PartialDependenceDisplay

# PDP with ICE
PartialDependenceDisplay.from_estimator(
    model, X_train,
    features=['age', 'income'],
    kind='both',      # PDP + ICE
    ice_lines_kw={'alpha': 0.1}
)