Design a global rate limiter that protects your APIs across many regions, enforcing per-customer quotas. Walk me through the design and, more importantly, the trade-offs you would make and how you would sequence building it.
What they are really testing: Whether you can reason about the CAP-style tension between strict global accuracy and latency and availability, choose a pragmatic point on that spectrum, and sequence delivery rather than over-building on day one.
A real interview question
Design a global rate limiter that protects your APIs across many regions, enforcing per-customer quotas. Walk me through the design and, more importantly, the trade-offs you would make and how you would sequence building it.
What most people say
drag me
“I would put a counter in a central distributed cache keyed by customer, increment it on every request, and reject when it exceeds the quota. That gives a single accurate global count that every region shares.”
Technically workable but it makes one strict choice and ignores the trade-offs. A synchronous central counter adds cross-region latency to every call and becomes a single dependency whose outage can break all APIs. No mention of fail-open, approximation, or sequencing.
The follow-ups they ask next
A whale customer games the approximate limiter by spreading traffic across regions to exceed their quota. How do you respond?
Listen for tightening reconciliation or stricter coordination for high-value keys only, a targeted fix rather than making everything strict.
Your reconciliation layer goes down. What happens to the system, and was that by design?
Listen for graceful degradation to local-only limiting, a deliberate fail-open choice rather than a global outage.
What the interviewer is listening for
- Surfaces the accuracy-versus-latency trade-off explicitly and picks a point on purpose
- Prefers approximate local limiting with async reconciliation for resilience
- Reasons about fail-open versus fail-closed per use case
- Sequences delivery to ship protection early before global accuracy
What sinks the answer
- Reaches for a strict synchronous global counter with no trade-off discussion
- Ignores cross-region latency and the single shared dependency it creates
- No failure mode plan, so the limiter can take down the API it protects
- Tries to build the most precise system at once instead of sequencing
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“I would first clarify whether the limit is for [billing or abuse]. The core trade-off is [strict global accuracy versus latency and availability]. For most APIs I would choose [local buckets with async reconciliation], decide [fail-open or fail-closed] per use case, and sequence delivery as [single-region, then per-region, then cross-region reconciliation] so protection ships [early].”
Keep going with architecture & design
Junior
How does putting a load balancer in front of your app improve availability, and what does it NOT solve?
Junior
What is autoscaling, and how does it decide when to add or remove instances?
Senior
Design a highly available web application on AWS. Walk me through the architecture and your trade-offs.
Senior
Design a disaster recovery strategy for a revenue-critical workload on AWS. How do you pick between backup-and-restore, pilot light, warm standby, and active-active?
Senior
Design a scalable async processing pipeline for spiky, long-running work (think video transcoding or report generation). Walk me through the queue-based architecture and its failure modes.
Senior
You own a system of many services and an incident took too long to diagnose. Design an observability strategy across the whole estate.
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free