16 — Analyze Your Data

Sampling & Class Imbalance#

Data Prep✓ Mathematical
◆ The PatternSMOTE, undersampling, class weights — strategies when your classes are nowhere near 50/50

Fraud detection: 0.1% positive. Disease screening: 2% positive. With severe class imbalance, a model that predicts "no" always gets 99%+ accuracy while being useless. You need resampling or cost-sensitive learning.

SMOTE: xnew = xᵢ + λ · (xnn − xᵢ)
SMOTE creates synthetic minority samples by interpolating between a sample and its nearest neighbour.
// Interactive — class ratio and resampling effect
Imbalance ratio1:10
Strategy
# Python — handling imbalance
from imblearn.over_sampling import SMOTE
from imblearn.under_sampling import RandomUnderSampler
from imblearn.pipeline import Pipeline

# SMOTE + undersampling combo
pipeline = Pipeline([
    ('smote', SMOTE(sampling_strategy=0.5)),
    ('under', RandomUnderSampler(sampling_strategy=0.8)),
])
X_res, y_res = pipeline.fit_resample(X_train, y_train)

# Or just use class weights
model = RandomForestClassifier(class_weight='balanced')
Never SMOTE the test set. Apply resampling only to training data, inside the CV loop.
Performance in practice
  • class_weight='balanced' is often enough — it's free, no extra data, and works with any sklearn model. Try this first
  • SMOTE improves recall by 5-15% on average but can hurt precision. Best combined with undersampling the majority
  • At extreme ratios (1:10000+, e.g. click fraud), even SMOTE struggles. Consider anomaly detection (Isolation Forest) instead of classification
  • In Kaggle competitions, the top fraud/anomaly solutions almost always use ensemble + threshold tuning rather than heavy resampling
When to use this
Use when: Your minority class is < 10% of data. Your model's recall on the minority class is poor. You're working in fraud, disease detection, churn prediction, or any domain with naturally rare events.
Skip when: Classes are roughly balanced (30-70% split). You have enough minority samples (>5K). You're using models that handle imbalance natively (like focal loss in neural nets).
← Previous
Data Drift & Distribution Shift
Open in the full reader, with the topic sidebar →