Standard k-fold cross-validation breaks temporal ordering and causes data leakage. Time series requires expanding window (growing training set), sliding window (fixed-size window), or walk-forward validation. A purge gap between train and test prevents contamination from lagged features.
- Sort data by timestamp and freeze a final holdout period before feature engineering.
- Create lag and rolling features using only past values; shift rolling features by one period.
- Use expanding-window validation for growing history or sliding-window validation for changing regimes.
- Add a purge gap when features use delayed labels, rolling windows, or overlapping horizons.
- Report metrics by horizon, not just one average score.
- MAE is easiest to explain in original units.
- RMSE is better when large forecast misses are expensive.
- MAPE is readable for stakeholders but unstable near zero.
- Always compare against naive and seasonal-naive baselines.
100 time steps, four folds of 10 test points each, with a gap of 2. The training windows grow — 0–58, 0–68, 0–78, 0–88 — and each is followed by its test window 60–70, 70–80, 80–90, 90–100. The model is always trained on the past and tested on the future, and the gap keeps lagged features from bridging the two.
- Hyperparameters tuned on these folds still need a final, untouched test period.
- Early folds have little training data, so their errors are pessimistic; weight or drop them knowingly.
- For purely autoregressive models with uncorrelated errors, ordinary k-fold can be valid (Bergmeir et al. 2018) — but only then.
n, folds, test_size, gap = 100, 4, 10, 2
splits = []
for i in range(folds): # expanding window
test_start = n - (folds - i) * test_size
splits.append(((0, test_start - gap), (test_start, test_start + test_size)))Assumes import numpy as np and import pandas as pd. Each snippet simulates or makes up its own series, as the comments say.
The same idea with a library
# Python — time series cross validation from sklearn.model_selection import TimeSeriesSplit tscv = TimeSeriesSplit(n_splits=5, gap=7) for train_idx, test_idx in tscv.split(X): X_train, X_test = X[train_idx], X[test_idx] y_train, y_test = y[train_idx], y[test_idx] # fit and evaluate
- C. Bergmeir & J. M. Benítez, “On the Use of Cross-Validation for Time Series Predictor Evaluation”, Information Sciences 191, 2012
- C. Bergmeir, R. J. Hyndman & B. Koo, “A Note on the Validity of Cross-Validation for Evaluating Autoregressive Time Series Prediction”, Computational Statistics & Data Analysis 120, 2018
- L. J. Tashman, “Out-of-Sample Tests of Forecasting Accuracy: An Analysis and Review”, International Journal of Forecasting 16(4), 2000
Last reviewed October 2026 · How this site checks its content · Report a mistake
1. Why leave a gap between the training and test periods in time-series cross-validation?
Show answer
C. Without a gap, the last training rows and the first test rows can share overlapping windows of the same data.
2. Random k-fold on lagged features scores much better than a time-ordered split. The most likely reason:
Show answer
C. The random score measures interpolation inside the past, not forecasting the future.
3. What does scikit-learn’s TimeSeriesSplit guarantee?
Show answer
A. Training windows expand forward; the test fold is always the next block in time.