Practice✓ Mathematical
◆ The PatternHow good are your predictions, really?
Rolling-origin backtesting simulates how your model would have performed if deployed at each past point. Evaluate with MAE (robust to outliers), RMSE (penalises large errors), MAPE (scale-free), and SMAPE (symmetric). Always evaluate across multiple horizons.
MAE = (1/n)∑|yt − 𝑓̂t| | MAPE = (100/n)∑|yt − 𝑓̂t| / |yt|
Choose your metric based on your decision context. MAE in original units, MAPE for stakeholder communication, RMSE when large errors are costly.
// Interactive — forecast accuracy metrics
# Python — backtesting with Darts from darts.metrics import mae, rmse, mape backtest = model.historical_forecasts( series, start=0.7, forecast_horizon=12, stride=1, retrain=False ) print(f"MAE: {mae(series, backtest):.3f}") print(f"RMSE: {rmse(series, backtest):.3f}")
Pattern bridge: Forecast backtesting is the time-series version of loss metrics from statistics. In markets, this directly maps to strategy backtesting — both test historical performance without look-ahead bias.