The Toolkit

1,000 funds with no skill at all, each year closing the worst 10% of those still alive. After five years 590 survive. Across all funds the average annual return is 0.26% — zero, as it should be. Across the survivors it is 2.16%. A database of today’s funds reports skill that never existed.

• Survivorship hides in index constituents, delisted stocks and closed funds; the data vendor’s history must include the dead.
• The bias is largest exactly where returns are most volatile.
• Selection on success happens in research too: only working strategies get written up.

rng = np.random.default_rng(13)
funds, years = 1_000, 5
returns = rng.normal(0.0, 0.10, (funds, years))      # no fund has any skill
alive = np.ones(funds, bool)
for t in range(years):                                # each year the worst 10% of survivors close
    cut = np.quantile(returns[alive, t], 0.10)
    alive &= ~(returns[:, t] < cut)
all_avg, survivors_avg = returns.mean(), returns[alive].mean()

Assumes import numpy as np and import pandas as pd (and SciPy where imported). Each snippet simulates or makes up its own data, as the comments say.

# Checklist — is your backtest honest?
# 1. Does your universe include delisted stocks?
# 2. Are you using point-in-time data?
# 3. Are fundamentals lagged by reporting delay?
# 4. Is there any future information leaking?
# 5. Did you optimise parameters on the test period?
# 6. How many strategies did you test? (data snooping)

# Point-in-time fundamental data
# Use 'as_of' date columns, not 'period_end'
df = df[df['report_date'] <= df['trade_date']]