08 — Classical Models

ARIMA#

ClassicalStatistical 1 min read
◆ The PatternThe workhorse of classical forecasting

ARIMA(p,d,q) combines autoregression (AR), differencing (I), and moving average (MA). The Box-Jenkins methodology uses ACF/PACF to identify orders, fits the model, then checks residuals for white noise. Auto-ARIMA automates this.

Δdyt = c + φ1Δdyt−1 + … + θ1εt−1 + … + εt
ARIMA unifies the three operations: differentiate to stationarize, AR for lag dependence, MA for shock dependence.
// Interactive — ARIMA forecast with confidence intervals
p (AR)1
d (diff)1
q (MA)1
Pattern bridge: ARIMA’s model selection via AIC/BIC is the same bias-variance tradeoff as cross-validation in ML. In markets, ARIMA forecasts on price returns connect directly to moving average signals.
Performance in practice
  • In the M4 competition (100,000 series), most pure machine-learning methods were less accurate than simple statistical benchmarks; the winner combined exponential smoothing with a neural network (Makridakis et al. 2020)
  • Automatic order selection (auto.arima, pmdarima) fits a typical univariate series in seconds; manual Box-Jenkins is mainly useful for understanding
  • For financial returns, ARIMA captures linear dependence but misses volatility clustering — pair it with GARCH for the variance
  • ARIMA struggles with structural breaks — test for stationarity (ADF, KPSS) and watch for regime changes
When to use this
✓ Use when: Univariate forecasting where interpretability matters. Short-horizon forecasts (1-30 steps). As a strong baseline before trying complex models. Demand forecasting, inventory planning, economic indicators.
✗ Skip when: You have many exogenous features (use machine learning instead). Very long horizons where uncertainty dominates. Non-stationary data with structural breaks. Multivariate dependencies are important (use VAR or neural methods).
Try it on real data
Kaggle: Airline Passengers (classic seasonal time series, 144 monthly observations) FRED: US Unemployment Rate (macroeconomic monthly series, 900+ observations)
The airline dataset is perfect for SARIMA (clear trend + seasonality). The FRED series tests ARIMA on regime-change data — notice how 2008 and 2020 break the model's assumptions.
Quick start — copy to notebook
pip install pmdarima statsmodels pandas matplotlib
# ────────────────────────────────────────
import pandas as pd, matplotlib.pyplot as plt
from pmdarima import auto_arima
from statsmodels.tsa.stattools import adfuller

df = pd.read_csv('AirPassengers.csv', parse_dates=['Month'], index_col='Month')
print(f"ADF p-value: {adfuller(df['#Passengers'])[1]:.4f}")  # test stationarity

model = auto_arima(df, seasonal=True, m=12, stepwise=True, trace=True)
fc, ci = model.predict(24, return_conf_int=True)
plt.plot(df.index, df.values, label='Observed')
plt.fill_between(pd.date_range(df.index[-1], periods=24, freq='M'), ci[:,0], ci[:,1], alpha=.2)
plt.show()
Worked example

For a random walk — ARIMA(0,1,0) — with one-step error 1, the 95% interval is ±1.96 one step ahead, ±3.92 at 4, ±6.79 at 12 and ±14.13 at 52. Uncertainty grows with the square root of the horizon, without limit. For integrated series the forecast itself is easy; honest intervals are the point.

Where it misleads
  • Automatic order selection (auto.arima and its kin) is a search; the chosen model is optimistic about its own fit.
  • Intervals assume the model is right and the errors normal; real coverage is usually lower than stated.
  • Simple benchmarks such as naive or seasonal naive often come close; always report them alongside.
In code
sigma = 1.0                                       # one-step error of a random walk, ARIMA(0,1,0)
width = {h: round(1.96 * sigma * np.sqrt(h), 2) for h in (1, 4, 12, 52)}   # 95% interval half-width
Python, in your browser — edit the code and run it again

Assumes import numpy as np and import pandas as pd. Each snippet simulates or makes up its own series, as the comments say.

The same idea with a library
# Python — auto ARIMA
from pmdarima import auto_arima

model = auto_arima(series, seasonal=False,
                   stepwise=True, trace=True)
print(model.summary())
forecast = model.predict(n_periods=30)
Sources
  • G. E. P. Box & G. M. Jenkins, Time Series Analysis: Forecasting and Control, Holden-Day, 1970
  • R. J. Hyndman & Y. Khandakar, “Automatic Time Series Forecasting: The forecast Package for R”, Journal of Statistical Software 27(3), 2008
  • Forecasting: Principles and Practice (3rd ed.), R. J. Hyndman & G. Athanasopoulos, OTexts, 2021

Last reviewed October 2026 · How this site checks its content · Report a mistake

Linked from 1
← Previous
MA Models
Open in the full reader, with the topic sidebar