Design autoscaling for a service with a sharp traffic spike every day at 9am.
What they are really testing: Whether you understand that reactive autoscaling is always late, and can reason about the time it takes to add capacity versus the shape of the demand curve.
A real interview question
Design autoscaling for a service with a sharp traffic spike every day at 9am.
What most people say
drag me
“Set up horizontal pod autoscaling on CPU with a target of 70% and let it handle the spike.”
Reactive CPU-based scaling is guaranteed to be late for a sharp spike: by the time CPU rises, the metric is scraped, a decision is made and pods are scheduled, the spike has already caused errors. It also assumes CPU tracks demand, which for IO-bound services it does not.
The follow-ups they ask next
What dominates that 2 to 5 minutes in practice?
Usually node provisioning when the cluster has no spare capacity, and image pull for large images. Keeping warm node headroom and shrinking images attack the two biggest components directly.
Why is scaling on CPU bad for an IO-bound service?
Such a service is mostly waiting, so CPU stays low while latency climbs and connection pools saturate. You would never scale up despite being overloaded. Concurrency or queue depth reflects the real pressure.
How do you stop autoscaling thrashing up and down?
Stabilisation windows and asymmetric behaviour: scale up quickly, scale down slowly. Removing capacity aggressively right before the next peak is a common self-inflicted incident.
Is over-provisioning ever the right answer?
Often, for spiky critical paths. Compare the cost of idle capacity against the cost of the outage. For a 9am spike, running extra capacity for 30 minutes is usually far cheaper than a failed morning.
What the interviewer is listening for
- Quantifies end-to-end scale-up latency
- Uses scheduled scaling for a known pattern
- Questions CPU as the scaling signal
- Adds shedding as a fallback
What sinks the answer
- Reactive CPU autoscaling only
- No awareness of provisioning delay
- Runs at high utilisation with no headroom
- No graceful degradation
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“First measure [end-to-end scale-up time: metric delay, decision interval, scheduling, node provisioning, image pull, warm-up], which is often [2 to 5 minutes]. If demand triples in 60 seconds, [reactive scaling cannot win]. So [pre-warm on a schedule for the known 9am spike], keep [reactive scaling for the unpredictable part], scale on [concurrency or queue depth rather than CPU], and add [shedding plus headroom].”
Keep going with reliability
Senior
You are incident commander for a total outage. It is 2am, 6 engineers are online, and nobody knows the cause. What do you do?
Senior
Design a CI/CD system for 15 microservices owned by 4 teams deploying several times a day.
Senior
Design an observability stack for a system where nobody can currently answer why a request was slow.
Senior
Leadership asks you to prove the platform investment is working. What do you measure?
Senior
You inherit 200 Jenkins jobs with no documentation and are asked to migrate to a modern CI system. How do you approach it?
Senior
Four teams want to share one Kubernetes cluster. How do you isolate them, and when would you give them separate clusters instead?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free