19 — Scale & Optimize

Auto-Scaling Endpoints#

Infrastructure✓ Mathematical
◆ The PatternScaling with demand — and scaling back to save money

Horizontal Pod Autoscaler (HPA) adds replicas when CPU/memory/custom metrics exceed thresholds. Scale-to-zero (Knative, serverless) eliminates idle costs but adds cold start latency. The right strategy depends on traffic patterns — bursty vs steady, latency-tolerant vs strict SLO.

// Interactive — replica count vs traffic
Traffic Load50%
Replicas
Pattern bridge: Auto-scaling is dynamic position sizing for infrastructure — scale up exposure when opportunity (traffic) increases, scale down when it drops. The Monte Carlo approach helps simulate traffic scenarios for capacity planning.
← Previous
Caching & Prediction Stores
Open in the full reader, with the topic sidebar →