Guides

Home

Guides · Checklist · 5 min

Is my backtest honest?

Fourteen checks that separate a strategy from a story. Tick them off; your progress stays in your browser.

A backtest is a search through the past. Most of these checks guard against finding something that was never there.

Background The Garden of Forking Paths · Walk-Forward Validation

0 of 14 checked

Data Every input uses only information available at the moment of the decision — no look-ahead.Feature Stores · Feature Engineering The universe includes companies that were later delisted, acquired or went bankrupt.Survivorship & Look-Ahead Bias · Survivorship Bias Prices are adjusted for splits and dividends, consistently.pandas-ta & yfinance · Resampling & Frequency

Costs and capacity Commissions, spreads and slippage are included — and the result survives doubling them.Stop-Loss Strategies · Walk-Forward Validation Positions are small relative to traded volume; the strategy could actually be executed.Maximum Position Limits

Validation The rule was written down before testing, and the test period was never used for tuning.Hypothesis Testing & p-values · Walk-Forward Validation Results come from walk-forward or out-of-sample periods, not from the period used to choose the rule.Walk-Forward Validation · Cross-Validation for TS Every variant tried is counted, and the result is judged against that number.The Garden of Forking Paths · scipy.stats & statsmodels Nearby parameter values work too — the result is not a single lucky setting.Optuna · Monte Carlo Simulation The test covers different regimes: calm and crisis, rising and falling markets.Correlation Risk · Changepoint Detection

Results Performance is compared with buy-and-hold or the relevant benchmark, risk-adjusted.Risk-Adjusted Performance · Benchmark Tracking The Sharpe ratio comes with a confidence interval, from enough history to mean something.Sharpe Ratio & Risk-Adjusted Returns · Confidence Intervals Drawdowns are studied as a distribution (resampled orders), not one historical path.Monte Carlo Simulation · Maximum Drawdown & Recovery The strategy’s edge has a reason to exist — and a reason it has not been competed away.Alpha Generation · Factor Models

Last reviewed: 2 October 2026 · How this site checks its content · Report a mistake