ML incidents are different from software bugs: the code runs fine, but predictions are wrong. Rollback reverts to the previous model version. Fallbacks (rule-based defaults, cached predictions) serve something when the model is down. Circuit breakers automatically switch to fallback when error rates spike.
| Severity | Response | Timeline |
|---|---|---|
| P0 — Model serving errors | Rollback immediately | Minutes |
| P1 — Accuracy degradation | Switch to fallback, investigate | Hours |
| P2 — Slight drift detected | Schedule retraining | Days |
| P3 — Feature quality warning | Monitor and log | Next sprint |
A bad model goes live at 50 requests a second. If someone notices after 45 minutes and rolls back in 5 more, 150,000 requests are served by it. A circuit breaker that switches to a fallback when the error rate passes 5% over two minutes limits it to 6,000 — 25× fewer. Detection time, not repair time, dominates.
- ML failures are often silent — predictions are wrong but no error is raised — so breakers on error rates miss them; watch prediction distributions too.
- A fallback is only safe if it has been tested recently.
- Postmortems that end in “be more careful” change nothing; fix the system that let it through.
rps = 50 # requests per second bad_from = 0 # a bad model goes live at t = 0 detect_manual = 45 * 60 # someone notices after 45 minutes detect_breaker = 2 * 60 # error rate > 5% over a 2-minute window trips the breaker rollback = 5 * 60 # manual rollback takes 5 minutes hit_manual = rps * (detect_manual + rollback) hit_breaker = rps * detect_breaker # the breaker switches to the fallback at once
Assumes import numpy as np and import pandas as pd. Each snippet carries its own example numbers; the comments say which are made up or simulated.
- Site Reliability Engineering: How Google Runs Production Systems, B. Beyer, C. Jones, J. Petoff & N. R. Murphy (eds.), O’Reilly, 2016 — “Postmortem Culture”
- Release It! Design and Deploy Production-Ready Software, M. T. Nygard, Pragmatic Bookshelf, 2007 — the circuit breaker
- S. Shankar, R. Garcia, J. M. Hellerstein & A. G. Parameswaran, “Operationalizing Machine Learning: An Interview Study”, arXiv:2209.09125, 2022
Last reviewed October 2026 · How this site checks its content · Report a mistake