Features✓ Mathematical
◆ The PatternSpotting redundant features — correlation heatmaps, VIF, and deciding what to drop
Highly correlated features are redundant — they inflate coefficient variance in linear models and confuse importance measures. VIF (Variance Inflation Factor) quantifies how much each feature's coefficient variance is inflated by correlation with others.
VIFᵢ = 1 / (1 − Rᵢ²)
VIF = 1 means no collinearity | >5 is concerning | >10 is severe. Rᵢ² from regressing feature i on all others.
// Interactive correlation heatmap
Features6
# Python — correlation & VIF import pandas as pd from statsmodels.stats.outliers_influence import variance_inflation_factor # Correlation heatmap corr = df.corr() sns.heatmap(corr, annot=True, cmap='RdBu_r', center=0) # VIF for each feature vif = pd.DataFrame() vif['Feature'] = X.columns vif['VIF'] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]
Drop rule: If two features have |r| > 0.9, drop the one less correlated with the target, or the one with higher VIF.