Auto Scaling

What is autoscaling?

Autoscaling adjusts compute capacity automatically in response to demand — adding instances/pods/containers under load, removing them when idle. It’s the mechanism behind cloud’s central pitch: pay for what you use, absorb spikes without over-provisioning for a peak that’s rare. But “automatic” hides real decisions — what signal to scale on, how fast to react, and what bounds to enforce — and getting those wrong turns autoscaling from a cost-saver into an outage source or a runaway bill.

The two axes and the metric problem

Horizontal scaling adds more instances (the cloud-native default — stateless services scale out cleanly); vertical scaling makes instances bigger (limited by max instance size, usually needs a restart). The harder question is the signal. CPU is the classic metric and often the wrong one: a service bottlenecked on database connections or queue depth won’t show CPU pressure while it drowns. Scaling on the metric that actually reflects load — request latency, queue depth, concurrent connections, or a custom business metric — is what separates responsive autoscaling from autoscaling that reacts to the wrong thing. Kubernetes HPA, cluster autoscaler, and KEDA (event-driven scaling on queue length, etc.) exist to broaden beyond CPU.

The failure modes

Reaction lag: scaling takes time (new instances boot, pods pull images, cold starts warm), so a fast spike can overwhelm you before capacity arrives — pre-scaling on schedules or predictive signals bridges the gap. Flapping: aggressive scale-in/scale-out oscillates, churning capacity and cost — cooldowns and stabilization windows damp it. Runaway scaling: a bug or attack drives infinite scale-out and an infinite bill — maximum limits are mandatory, the autoscaling equivalent of a circuit breaker. And scaling only works if the app is genuinely stateless and the downstream dependencies (database, connection pools) can absorb the added fleet.

What people get wrong

• Scaling on CPU when the real bottleneck is elsewhere — the service melts while CPU looks calm.
• No maximum bound — turning a traffic spike or bug into an unbounded cost event.
• Forgetting the database: scaling stateless tiers into a fixed-capacity datastore just moves the bottleneck and can take the DB down.

Primary source: AWS Auto Scaling documentation