One slow dependency takes down your entire system. Why does that happen and how do you prevent it?

Question

Accepted Answer

The mechanism is resource exhaustion propagating backwards. My service calls a dependency that slows from 50ms to 8 seconds. Each in-flight request now holds a thread and a connection for 8 seconds instead of 50ms, so at any real request rate my pool is exhausted within seconds, and now my service cannot serve any request, including the 90% that never needed that dependency. Then my callers do the same thing, and it walks up the graph. Counter-intuitively a dependency being completely down is safer than being slow, because a refused connection frees the resource instantly. The controls, in order of value: aggressive timeouts on every network call, because unbounded waiting is what enables the whole cascade, then bulkheads so each dependency gets its own bounded pool and cannot starve the others, then circuit breakers to stop sending traffic to something already failing, then graceful degradation so the 90% of the response that does not need it still works. Retries need care: naive retries amplify load on the failing thing, so they need exponential backoff, jitter, and a budget capping retries as a fraction of traffic.

Answer

How do you choose a timeout value?

From the dependency latency distribution, typically a bit above p99, so normal requests are unaffected but pathological ones are cut. Then ensure the caller timeout exceeds the callee, or you waste work the callee still completes.

Answer

What is a retry budget and why does it matter?

A cap on retries as a percentage of total requests, say 10%. It preserves retries for transient blips while making it impossible for retry traffic to multiply load during a real outage.

One slow dependency takes down your entire system. Why does that happen and how do you prevent it?

One slow dependency takes down your entire system. Why does that happen and how do you prevent it?

The follow-ups they ask next

What the interviewer is listening for

What sinks the answer

If you genuinely do not know

Keep going with troubleshooting

Knowing the answer is not the same as recalling it under pressure