You are paged: the overnight batch agent has been running 6 hours instead of 20 minutes and has spent 400 dollars. Walk me through your response.
What they are really testing: Incident discipline applied to an agentic system. The order matters, stop the spend, preserve evidence, diagnose, then prevent, and the prevention list reveals whether they know runaway loops are a designed-against failure class, not bad luck.
A real interview question
You are paged: the overnight batch agent has been running 6 hours instead of 20 minutes and has spent 400 dollars. Walk me through your response.
What most people say
drag me
“I would stop the agent and look at the logs to see what went wrong, then fix the bug and add better error handling.”
Generic incident words with nothing agent-shaped. No check for executed side effects, no reading of the tool-call trace, and "better error handling" does not name the missing budget caps that made 6 hours possible.
The follow-ups they ask next
The agent sent 200 duplicate notification emails while looping. Now what?
Side-effect triage: dedup and apology where needed, then the structural fix, idempotency keys on side-effecting tools and confirmation gates on externally-visible actions, so a loop can waste tokens but not repeat sends.
Why must the budget live outside the model rather than in the prompt?
A model deep in a failure loop is precisely the model least likely to follow a "stop after 10 tries" instruction, its context is polluted. Enforcement has to sit where the loop runs, in code.
What the interviewer is listening for
- Kills spend and checks executed side effects before diagnosing
- Reads the tool-call trace and can describe the self-reinforcing loop mechanism
- Prevention names hard external budgets, loop detection and anomaly alerts
What sinks the answer
- Straight to debugging with the meter still running
- Never considers side effects the loop already executed
- Prevention is a prompt instruction asking the agent to behave
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Order: [kill the run, confirm spend stopped, check executed side effects], [read the full tool-call trace to find where the loop began], [identify the mechanical cause, usually an unactionable tool error or unreachable goal], then prevent the class with [external budgets at 2x normal envelope, loop detection, actionable errors, and spend anomaly alerts].”
Keep going with troubleshooting
Junior
An alert fires: 30 percent of your model API calls are failing with 429 rate-limit errors. Users see errors. What do you do?
Mid
Users report your RAG assistant is giving wrong answers. Walk me through how you debug it.
Mid
Overnight, p95 latency on your AI assistant tripled from 4 to 12 seconds. No deploy happened. Diagnose it.
Senior
Support reports the AI assistant got noticeably worse this week. Nothing was deployed. Walk me through the investigation.
Senior
Your evals pass at 92 percent, but production complaints keep coming. Users say the assistant is failing at things your suite says it does well. Debug the gap.
Mid
What are hybrid search and reranking, and when does a RAG system actually need them?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free