19 — Backtest & Validate

Walk-Forward Validation#

BacktestingStatistical 1 min read
◆ The PatternRolling window backtesting — the right way to validate strategies on time-ordered data

Walk-forward slides a training window through time, always testing on the next unseen period. It simulates real deployment: train on the past, predict the future. Anchored mode grows the training window. Rolling keeps it fixed.

Train: [t−W, t] → Test: [t+1, t+S]
W = training window  |  S = test step  |  then slide forward by S and repeat.
// Interactive — rolling vs anchored walk-forward
Window size8
Mode
Pattern bridge: Walk-forward validation is cross-validation adapted for time. The same "never test on training data" principle, but respecting temporal order — critical in both ML deployment and market trend analysis. Guide: Is my backtest honest?
Real-world pipeline: backtesting a strategy
  1. Define windows: Train on 2 years, test on next 3 months, slide forward by 3 months
  2. Feature engineering inside each fold — never compute indicators on future data
  3. Track per-window metrics: Sharpe, drawdown, hit rate — look for consistency, not just the average
  4. Compare to benchmark: Does your model beat buy-and-hold in each window, or just on average?
  5. Stress test: Include windows with crashes (2008, 2020, 2022) — how does the model perform under stress?
Common pitfall — look-ahead bias: Using indicators like 52-week high/low that peek into the future of your test window. Also: survivorship bias — backtesting on today's S&P 500 ignores all the companies that went bankrupt. Use point-in-time datasets.
Worked example

200 strategies with no edge at all, tested on two years of daily data. The best one on the first year has a Sharpe ratio of 1.74; on the second year the same strategy scores −1.07, and the average out of sample is −0.07. Selecting on in-sample results is selecting on luck; only the out-of-sample period says anything.

Where it misleads
  • Re-optimising at every walk-forward step and then reporting the combined result is still a search; count every variant you tried.
  • Short out-of-sample windows are noisy; one good year proves little.
  • Walk-forward cannot fix data problems such as survivorship or look-ahead in the inputs.
In code
rng = np.random.default_rng(11)
r = rng.normal(0, 0.01, (200, 1_000))               # 200 strategies with no edge, 1,000 days
ins, outs = r[:, :500], r[:, 500:]
sharpe = lambda x: x.mean(axis=-1) / x.std(axis=-1) * np.sqrt(252)
best = np.argmax(sharpe(ins))
result = (round(sharpe(ins)[best], 2), round(sharpe(outs)[best], 2), round(sharpe(outs).mean(), 2))
Python, in your browser — edit the code and run it again

Assumes import numpy as np and import pandas as pd (and SciPy where imported). Each snippet simulates or makes up its own data, as the comments say.

The same idea with a library
# Python — walk-forward with TimeSeriesSplit
from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5)
results = []
for train_idx, test_idx in tscv.split(X):
    X_train, X_test = X[train_idx], X[test_idx]
    model.fit(X_train, y[train_idx])
    score = model.score(X_test, y[test_idx])
    results.append(score)

print(f"Walk-forward: {np.mean(results):.3f}")
Sources
  • D. H. Bailey, J. M. Borwein, M. López de Prado & Q. J. Zhu, “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance”, Notices of the AMS 61(5), 2014
  • R. Pardo, The Evaluation and Optimization of Trading Strategies (2nd ed.), Wiley, 2008
  • H. White, “A Reality Check for Data Snooping”, Econometrica 68(5), 2000

Last reviewed October 2026 · How this site checks its content · Report a mistake

Check yourself

1. Why not use ordinary shuffled k-fold cross-validation on a trading strategy’s daily data?

Show answer

A. Walk-forward testing keeps time order: fit on the past, test on the next period, roll forward.

2. You test 200 parameter sets walk-forward and report the out-of-sample Sharpe ratio of the best one. What is wrong?

Show answer

A. Selection is fitting. Either choose parameters inside each training window, or correct for the number of trials (e.g. the deflated Sharpe ratio).

3. What is the trade-off between an expanding (anchored) training window and a rolling one?

Show answer

B. Neither leaks if every training window ends before its test period. The choice is about stability versus adaptivity.

Linked from 8
← Previous
Maximum Drawdown & Recovery
Open in the full reader, with the topic sidebar