What is autoscaling, and how does it decide when to add or remove instances?
What they are really testing: Whether you understand metric-driven scaling and the difference between scaling out and scaling up. They want the closed loop of metric, threshold, and action, plus awareness that scaling is not instant.
A real interview question
What is autoscaling, and how does it decide when to add or remove instances?
What most people say
drag me
“It just adds more servers when the site gets busy so it does not crash.”
It gestures at the idea but never names the metric, the threshold, or the cooldown, and ignores that scaling is not instantaneous and that the app must be stateless. That gap is exactly where real autoscaling configs go wrong.
The follow-ups they ask next
Traffic spikes for 30 seconds and your app falls over before autoscaling reacts. Why, and what would you change?
Boot and warm-up time means scaling lags short spikes. Keep a warm baseline of capacity, use a faster-reacting metric like request count, or add a buffer (queue) to absorb the burst.
How would scaling on CPU mislead you for an app that is mostly waiting on a slow database?
CPU stays low while requests pile up waiting on I/O, so it never triggers a scale-out even though the app is overloaded. Scale on request count or latency or queue depth instead.
What the interviewer is listening for
- Names a concrete metric and threshold
- Distinguishes scaling out from scaling up
- Knows scaling lags and needs a stateless app
What sinks the answer
- No mention of any metric driving the decision
- Confuses adding instances with resizing one
- Assumes scaling is instantaneous
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Autoscaling changes [the number of running instances] so capacity tracks demand. A policy watches [a metric like CPU or request count] against [a threshold] and scales out or in. This is [scaling out, not up]. It lags because [instances take time to boot], and it needs [a stateless app] plus [min and max bounds].”
Keep going with architecture & design
Junior
How does putting a load balancer in front of your app improve availability, and what does it NOT solve?
Senior
Design a highly available web application on AWS. Walk me through the architecture and your trade-offs.
Senior
Design a disaster recovery strategy for a revenue-critical workload on AWS. How do you pick between backup-and-restore, pilot light, warm standby, and active-active?
Senior
Design a scalable async processing pipeline for spiky, long-running work (think video transcoding or report generation). Walk me through the queue-based architecture and its failure modes.
Senior
You own a system of many services and an incident took too long to diagnose. Design an observability strategy across the whole estate.
Senior
Design a CI/CD pipeline that lets a team deploy many times a day safely. How do you make production changes low-risk with progressive delivery?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free