It is 2pm, peak traffic. Half the targets behind your ALB just flipped to unhealthy, and the surviving half are now taking double the load and starting to shed requests. Walk me through the next fifteen minutes.

Question

Accepted Answer

In the first two minutes I do not care why. Half the fleet is gone and the other half is absorbing double the traffic, so my first action is to scale out the target group, say from 8 tasks to 16, and if there was a deploy in the last hour I roll it back straight away. Stabilise, then investigate. Then I look at the target group health check reason in the console, because it tells me which layer broke. Timeout means the target is alive but too slow or a security group is dropping the health check port. Connection refused means the process is not listening, usually a crashed container or a port mismatch after a deploy. A 503 from the app means the target is up but its own dependency is down. I also check whether the failures cluster in one availability zone, because if all the dead targets are in one AZ this is an AZ or subnet problem, not an app problem, and I would shift traffic away from that AZ. I then look at target response time on the ALB. If p99 was already climbing from 200 milliseconds to 3 seconds before the health checks started failing, the real cause is downstream saturation and the health check is a symptom, not the disease. Once traffic is stable I write the timeline: first bad minute, action taken, recovery, and the one hypothesis I proved.

Answer

The health check reason is Timeout, but you can curl the target from your own box. Explain that.

The ALB uses its own security group and a specific health check port. Curl from your box does not prove the ALB can reach it.

Answer

Every unhealthy target is in one availability zone. What do you do? Treat it as an AZ level fault. Shift traffic out of that AZ and confirm the subnet, route table and NAT are healthy.

Answer

How would you make the health check less trigger happy without hiding real failures?

Separate liveness from readiness, tune the unhealthy threshold and interval, and never make the check depend on a downstream service.

It is 2pm, peak traffic. Half the targets behind your ALB just flipped to unhealthy, and the surviving half are now taking double the load and starting to shed requests. Walk me through the next fifteen minutes.

It is 2pm, peak traffic. Half the targets behind your ALB just flipped to unhealthy, and the surviving half are now taking double the load and starting to shed requests. Walk me through the next fifteen minutes.

The follow-ups they ask next

What the interviewer is listening for

What sinks the answer

If you genuinely do not know

Keep going with troubleshooting

Knowing the answer is not the same as recalling it under pressure