Practice✓ Mathematical
1 min read
◆ The PatternNever shuffle time series
Standard k-fold cross-validation breaks temporal ordering and causes data leakage. Time series requires expanding window (growing training set), sliding window (fixed-size window), or walk-forward validation. A purge gap between train and test prevents contamination from lagged features.
// Interactive — temporal cross-validation splits
Folds5
StrategyExpanding
# Python — time series cross validation from sklearn.model_selection import TimeSeriesSplit tscv = TimeSeriesSplit(n_splits=5, gap=7) for train_idx, test_idx in tscv.split(X): X_train, X_test = X[train_idx], X[test_idx] y_train, y_test = y[train_idx], y[test_idx] # fit and evaluate
Pattern bridge: Walk-forward validation is the forecasting equivalent of cross-validation from The Toolkit. In markets, this is exactly how trading strategy backtesting disciplines prevent look-ahead bias.
Real-world pipeline: time-series validation
- Sort data by timestamp and freeze a final holdout period before feature engineering.
- Create lag and rolling features using only past values; shift rolling features by one period.
- Use expanding-window validation for growing history or sliding-window validation for changing regimes.
- Add a purge gap when features use delayed labels, rolling windows, or overlapping horizons.
- Report metrics by horizon, not just one average score.
Common pitfall — shuffled folds: Random k-fold makes future observations available to the model indirectly. It usually overstates forecast quality.
Metrics in practice
- MAE is easiest to explain in original units.
- RMSE is better when large forecast misses are expensive.
- MAPE is readable for stakeholders but unstable near zero.
- Always compare against naive and seasonal-naive baselines.
Use this pattern on real data
Pattern Portal Case: Energy Demand Forecast
Use the case workflow to test lag features, rolling windows, naive baselines, and walk-forward validation.