E9 — Pattern Essays

Survivorship Bias#

Essay2 min read
1 min read
◆ The PatternWe study what survived — and quietly forget everything that did not

During the Second World War, the U.S. military studied bombers returning from missions and mapped where they were riddled with bullet holes — the wings, the fuselage, the tail. The instinct was to reinforce those areas. The statistician Abraham Wald saw it differently: the holes showed where a plane could be hit and still come home. The places with no holes — the engines, the cockpit — were where the lost planes had been struck. Reinforce the gaps, he argued, not the marks.

This is survivorship bias: we draw conclusions from the things that made it through the filter and never see the ones that did not. We study successful founders and copy their habits, ignoring the identical habits of thousands who failed. We admire old buildings and call past craftsmanship superior, forgetting the flimsy ones that already collapsed.

The pattern is a hole in the data, not in the analysis. The missing observations are invisible by definition — which is exactly why they are so easy to forget, and so dangerous to ignore.

Survival cutoff 40
Only survivors are seen — drag to watch the visible average inflate
What to rememberBefore trusting any lesson drawn from winners, ask what happened to everyone who is no longer in the sample.
References
[1] Wald, A. (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Statistical Research Group, Columbia University. Reprinted CRC 432 (1980).
[2] Mangel, M. & Samaniego, F. J. (1984). Abraham Wald’s Work on Aircraft Survivability. Journal of the American Statistical Association, 79(386), 259–267. doi:10.1080/01621459.1984.10478038
[3] Brown, S. J., Goetzmann, W., Ibbotson, R. G. & Ross, S. A. (1992). Survivorship Bias in Performance Studies. Review of Financial Studies, 5(4), 553–580. doi:10.1093/rfs/5.4.553
Pattern bridge: The outlier detection topic deals with the data you can see, while survivorship & look-ahead bias is the backtesting trap built on funds and strategies that quietly disappeared.
← Previous
The Threshold
Open in the full reader, with the topic sidebar →