How does a load balancer know a backend is unhealthy, and how do you avoid dropping requests during a deploy or scale-in?
What they are really testing: Practical operations: health-check tuning and connection draining. Getting this wrong means users see errors during routine deploys, which is a common self-inflicted outage.
A real interview question
How does a load balancer know a backend is unhealthy, and how do you avoid dropping requests during a deploy or scale-in?
What most people say
drag me
“The load balancer pings the servers and removes the ones that do not respond, then sends traffic to the rest.”
It gets the basic idea but ignores the operational half: threshold tuning, a real readiness check, and draining. Without draining, every deploy drops in-flight requests, which is the actual question.
The follow-ups they ask next
Users get a burst of 5xx every time you deploy. What is likely missing?
Connection draining and/or graceful shutdown: targets are removed (or the process exits) while requests are still in flight. Add a deregistration delay and have the app finish in-flight work before exiting.
Why not make the health check as fast and strict as possible?
Flapping: a strict, frequent check ejects healthy nodes on transient blips (GC, brief latency), causing churn and capacity loss. Tune for stable detection, not maximum speed.
What the interviewer is listening for
- Explains thresholds/interval tuning
- Wants a real readiness endpoint
- Knows connection draining + graceful shutdown for deploys
What sinks the answer
- Only "it pings and removes dead ones"
- No notion of draining
- Would set the strictest possible check
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“The LB [probes each target on an interval and flips it unhealthy after N failures]. I tune [interval/timeout/thresholds to avoid flapping] and point it at [a real readiness endpoint, not a trivial 200]. For deploys I rely on [connection draining/deregistration delay so in-flight requests finish] plus [graceful shutdown: stop new work, drain, exit], so a rollout [never cuts active requests].”
Keep going with networking
Foundation
What is a VPC, and what is a subnet within it? Why split a network into subnets at all?
Foundation
What is the difference between TCP and UDP, and when would you choose each?
Foundation
What happens, step by step, when you type a URL into a browser and press enter?
Foundation
How does DNS resolution actually work, and what is the difference between an A record and a CNAME?
Foundation
What is the difference between IPv4 and IPv6, and why does IPv6 exist?
Junior
An EC2 instance in a private subnet needs to download OS updates from the internet. Walk me through how you make that work, and why.
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free