24 — Practice & Tooling

Anomaly Detection#

PracticeStatistical 1 min read
◆ The PatternFinding the signal in the noise

Temporal anomalies are observations that deviate from expected patterns. Point anomalies are individual outliers. Contextual anomalies are normal values at the wrong time (e.g., summer demand in winter). Methods range from simple z-score thresholds to autoencoders that learn the normal pattern.

// Interactive — anomaly detection methods
Threshold (σ)2.0
Pattern bridge: Anomaly detection in time series uses the same z-score logic from statistics. In markets, anomalies are price gaps and volume spikes. In MLOps, data quality gates perform anomaly detection on incoming features.
Worked example

Two years of daily temperatures with warm summers, and one 15° day in January. Against the whole series its z-score is 0.58 — completely normal. Against the surrounding fortnight it is 7.87. Contextual anomalies need a local baseline. And a 3σ rule checked on 10,000 metrics a day raises about 27 false alarms a day by chance.

Where it misleads
  • Thresholds set on normal-looking history trigger constantly once the data drifts; baselines need updating.
  • Anomalies in the training window distort the baseline itself; use robust statistics (median, MAD).
  • Most alerts at scale are false; tune for what the people receiving them can act on.
In code
rng = np.random.default_rng(14)
day = np.arange(730)
temp = 10 - 12 * np.cos(2 * np.pi * day / 365) + rng.normal(0, 2, 730)   # two years, warm summers
temp[20] = 15                                      # a warm day in January: normal in July
z_global = (temp - temp.mean()) / temp.std()
resid = temp - pd.Series(temp).rolling(15, center=True, min_periods=5).median()
z_local = resid / resid.std()
flags = (round(z_global[20], 2), round(z_local[20], 2))
false_alarms_per_day = 10_000 * 0.0027             # 10,000 metrics checked daily at 3 sigma
Python, in your browser — edit the code and run it again

Assumes import numpy as np and import pandas as pd. Each snippet simulates or makes up its own series, as the comments say.

The same idea with a library
# Python — anomaly detection
from sklearn.ensemble import IsolationForest

# Z-score method
z = (series - series.rolling(30).mean()) / series.rolling(30).std()
anomalies = series[z.abs() > 2.5]

# Isolation Forest on features
clf = IsolationForest(contamination=0.05)
labels = clf.fit_predict(features)
Sources
  • V. Chandola, A. Banerjee & V. Kumar, “Anomaly Detection: A Survey”, ACM Computing Surveys 41(3), 2009
  • J. Hochenbaum, O. S. Vallis & A. Kejariwal, “Automatic Anomaly Detection in the Cloud Via Statistical Learning”, arXiv:1704.07706, 2017

Last reviewed October 2026 · How this site checks its content · Report a mistake

Linked from 1
← Previous
Backtesting Forecasts
Open in the full reader, with the topic sidebar