Fraud detection: 0.1% positive. Disease screening: 2% positive. With severe class imbalance, a model that predicts "no" always gets 99%+ accuracy while being useless. You need resampling or cost-sensitive learning.
class_weight='balanced'is a cheap first try: no extra data, and supported by most scikit-learn models- Oversampling (SMOTE and kin) usually trades precision for recall and distorts predicted probabilities; for risk scores it can do more harm than good (van den Goorbergh et al. 2022)
- At extreme ratios (1:10,000 and beyond) consider treating the problem as anomaly detection rather than classification
- Often the simplest fix is to choose the decision threshold from the cost of each error (see the worked example)
If a missed case costs ten false alarms, the right decision threshold for a calibrated probability is 1/(1 + 10) = 0.091. A case with P = 0.15 then costs 0.85 in expectation if you act and 1.5 if you do not, so you act. Choosing the threshold from the costs handles imbalance without resampling the data.
- The rule needs calibrated probabilities; scores from a resampled or class-weighted model are not.
- For risk prediction, imbalance corrections such as SMOTE often harm calibration without improving discrimination (van den Goorbergh et al. 2022).
- Costs are rarely known exactly; check how sensitive the decision is to them.
cost_fp, cost_fn = 1.0, 10.0 # a missed case costs ten false alarms threshold = cost_fp / (cost_fp + cost_fn) # act when P(positive) exceeds this expected_cost = lambda p, act: cost_fp * (1 - p) if act else cost_fn * p at_15 = (expected_cost(0.15, True), expected_cost(0.15, False)) # a case with P = 0.15
Assumes import numpy as np and import pandas as pd (and SciPy where imported). Each snippet simulates or makes up its own data, as the comments say.
The same idea with a library
# Python — handling imbalance from imblearn.over_sampling import SMOTE from imblearn.under_sampling import RandomUnderSampler from imblearn.pipeline import Pipeline # SMOTE + undersampling combo pipeline = Pipeline([ ('smote', SMOTE(sampling_strategy=0.5)), ('under', RandomUnderSampler(sampling_strategy=0.8)), ]) X_res, y_res = pipeline.fit_resample(X_train, y_train) # Or just use class weights model = RandomForestClassifier(class_weight='balanced')
- C. Elkan, “The Foundations of Cost-Sensitive Learning”, IJCAI, 2001
- H. He & E. A. Garcia, “Learning from Imbalanced Data”, IEEE Transactions on Knowledge and Data Engineering 21(9), 2009
- R. van den Goorbergh, M. van Smeden, D. Timmerman & B. Van Calster, “The Harm of Class Imbalance Corrections for Risk Prediction Models”, Journal of the American Medical Informatics Association 29(9), 2022
Last reviewed October 2026 · How this site checks its content · Report a mistake
1. 1% of transactions are fraud. What accuracy does a model get by predicting “not fraud” every time?
Show answer
B. It is right on all 99% of normal transactions and catches no fraud at all. With rare classes, look at precision and recall for the class you care about.
2. You oversample the fraud cases to 50/50 for training. The model’s predicted probabilities are now…
Show answer
A. The model learned that fraud is common because that is what it saw. Rankings may be fine, but if you use the probabilities — for thresholds, costs or reporting — recalibrate them to the true base rate.
3. Which is usually most informative when the positive class is rare?
Show answer
C. ROC-AUC counts true negatives, which are plentiful, so it can look good while most alarms are false. Precision–recall focuses on the rare class (Saito & Rehmsmeier 2015).