E11 — Pattern Essays

Simpson’s Paradox#

Essay2 min read
1 min read
◆ The PatternA trend can point one way in every group — and the opposite way once they are combined

In 1973, Berkeley appeared to admit men at a higher rate than women, hinting at bias. But when admissions were broken down department by department, most departments actually favoured women slightly. The reversal was real, not a mistake. Women had applied in larger numbers to the most competitive departments, where everyone’s odds were low. The aggregate hid the structure.

This is Simpson’s paradox: a relationship that holds within every subgroup can vanish or flip when the subgroups are pooled. A treatment can help both mild and severe patients yet look worse overall, simply because it was given more often to the sicker ones. The lurking variable — department, severity, the way cases were sorted — quietly steers the total.

The pattern is a warning about aggregation: a single number summarising a mixed population can point in a direction that is true of no one inside it. The fix is not better arithmetic — it is asking what was combined, and why.

Group separation 70%
Two rising groups, one falling total — drag to separate them
What to rememberBefore trusting an overall trend, split the data by the obvious subgroups — the aggregate can point where no group does.
References
[1] Simpson, E. H. (1951). The Interpretation of Interaction in Contingency Tables. Journal of the Royal Statistical Society B, 13(2), 238–241. doi:10.1111/j.2517-6161.1951.tb00088.x
[2] Bickel, P. J., Hammel, E. A. & O’Connell, J. W. (1975). Sex Bias in Graduate Admissions: Data from Berkeley. Science, 187(4175), 398–404. doi:10.1126/science.187.4175.398
[3] Pearl, J. (2014). Comment: Understanding Simpson’s Paradox. The American Statistician, 68(1), 8–13. doi:10.1080/00031305.2014.876829
Pattern bridge: The feature correlation topic shares this lurking-variable trap, and Bayesian A/B testing must guard against pooling groups that should stay apart.
← Previous
The Fractal
Open in the full reader, with the topic sidebar →