06 — Classical Models

AR Models#

ClassicalStatistical
◆ The PatternRegressing a series on its own past

An AR(p) model predicts each value as a weighted sum of the previous p values plus noise. The weights (φ coefficients) capture how strongly the series depends on each lag. PACF cutoff at lag p identifies the order.

yt = c + φ1yt−1 + φ2yt−2 + … + φpyt−p + εt
Each φ coefficient tells you how much influence a past value has. Stationarity requires all roots of the characteristic polynomial to lie outside the unit circle.
// Interactive — AR(p) process simulation
φ10.70
Order (p)1
Pattern bridge: AR models are time-series linear regression where the features are your own lagged values. In markets, the idea that past prices predict future prices is the foundation of momentum.
Worked example

200 points from an AR(1) with φ = 0.8: least squares estimates 0.763. From a value 3 above the mean, the forecasts are 2.40, 1.92, 0.98, 0.32 at 1, 2, 5 and 10 steps — decaying towards the mean, with a half-life of 3.1 steps. An AR forecast is a memory that fades geometrically.

Where it misleads
  • Least squares underestimates φ in small samples, more so as φ approaches 1.
  • Near φ = 1 the series behaves like a random walk and AR forecasts become fragile.
  • Choosing p by in-sample fit overfits; compare information criteria or out-of-sample error.
In code
rng = np.random.default_rng(4)
phi, n = 0.8, 200
y = np.zeros(n)
for t in range(1, n): y[t] = phi * y[t - 1] + rng.normal()
phi_hat = np.linalg.lstsq(y[:-1, None], y[1:], rcond=None)[0][0]
last = 3.0                                        # forecast from a value 3 above the mean of 0
forecast = [round(last * phi ** h, 2) for h in (1, 2, 5, 10)]
half_life = np.log(0.5) / np.log(phi)
Python, in your browser — edit the code and run it again

Assumes import numpy as np and import pandas as pd. Each snippet simulates or makes up its own series, as the comments say.

The same idea with a library
# Python — fit AR model
from statsmodels.tsa.ar_model import AutoReg

model = AutoReg(series, lags=3).fit()
print(model.summary())
forecast = model.predict(start=len(series), end=len(series)+10)
Sources
  • Time Series Analysis: Forecasting and Control (5th ed.), G. E. P. Box, G. M. Jenkins, G. C. Reinsel & G. M. Ljung, Wiley, 2015
  • Time Series Analysis, J. D. Hamilton, Princeton University Press, 1994

Last reviewed October 2026 · How this site checks its content · Report a mistake

Linked from 2
← Previous
Resampling & Frequency
Open in the full reader, with the topic sidebar