How do rate limiting, load shedding, and backpressure protect a service under overload?
What they are really testing: Whether you know that under overload it is better to reject some requests cleanly than to let everything collapse, and the distinct roles of these three mechanisms.
A real interview question
How do rate limiting, load shedding, and backpressure protect a service under overload?
What most people say
drag me
“You add rate limiting so users cannot send too many requests.”
It covers one mechanism for one purpose. The deeper point is overload protection: load shedding and backpressure keep the service alive by rejecting/slowing excess, because serving some requests well beats collapsing for all.
The follow-ups they ask next
Why is shedding load (rejecting some requests) better than trying to serve everything?
Serving everything at overload makes latency explode and queues grow until the whole service fails, so nobody gets served. Shedding the excess (429/503) keeps capacity for the rest, so most requests still succeed.
What is backpressure and when do you need it?
A signal to upstream/producers to slow down (bounded queues, blocking, flow control), so work does not pile up unbounded. Needed when a fast producer can outpace a slower consumer, for example a queue consumer or stream.
What the interviewer is listening for
- Distinguishes the three mechanisms
- Knows shed/throttle excess beats collapse
- Mentions token bucket / bounded queues / 429-503
What sinks the answer
- Only rate limiting
- Would try to serve everything under overload
- No notion of backpressure
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Three overload protections: [rate limiting caps per-client requests (token bucket) for fairness], [load shedding rejects excess (429/503) to protect the rest under overload], [backpressure signals upstream to slow down (bounded queues/flow control)]. The principle: [serve some well, not none, gracefully reject/slow the overflow instead of collapsing for everyone].”
Keep going with reliability
Foundation
What is a Region versus an Availability Zone, and why does the difference matter?
Foundation
What is the difference between vertical scaling and horizontal scaling, and when would you reach for each?
Foundation
What does "highly available" actually mean, and how do you achieve it?
Foundation
Why are stateless services easier to scale and make reliable than stateful ones?
Junior
How should a service retry a failed call to a dependency, and what is a retry storm?
Junior
What is idempotency, and why does it matter for reliable distributed systems?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free