You need to restore production after a bad migration dropped a column, and the nightly backup fails to restore. You have a business asking when the data will be back. What do you do?
What they are really testing: This tests your behaviour when the safety net itself has failed: whether you communicate honestly, look for a second source of truth, and time box each recovery attempt rather than tunnelling.
A real interview question
You need to restore production after a bad migration dropped a column, and the nightly backup fails to restore. You have a business asking when the data will be back. What do you do?
What most people say
drag me
“I would keep retrying the restore until it works, and tell the business it will be back soon.”
It tunnels on the one path that already failed, it gives a commitment with no evidence behind it, and it ignores every other recovery source that might hold the data.
The follow-ups they ask next
Why restore into a new instance rather than over production?
It keeps the current state as evidence and lets you compare, and it means a failed restore does not destroy your last option.
How do you make sure this backup never silently rots again?
Automated restore verification on a schedule, with an alert if the restore or the row count check fails.
What the interviewer is listening for
- Freezes writes before attempting any recovery
- Lists multiple independent recovery sources, not just the backup
- Time boxes attempts and communicates a range honestly
- Verifies the restored data against a business invariant
What sinks the answer
- Restores over production and destroys the current state
- Promises a recovery time with no evidence
- Retries the same failing path indefinitely
- Never proposes testing restores after the incident
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“I have not lived through a failed restore, but here is how I would reason about it: freeze the damage, list every place the data might still exist, time box each attempt, and be honest with the business about what I actually know.”
Keep going with troubleshooting
Junior
You get paged at 2am. Checkout latency has gone from 200ms to 4 seconds and it started right after the 11pm deploy. Walk me through what you do.
Junior
It is 3am and you get paged: the primary application server is at 100% disk usage and the app is throwing write errors. You are the only one online. Walk me through exactly what you do.
Junior
You push a new version of a service and within two minutes the pods are all showing CrashLoopBackOff. The team is watching you share your screen. Walk me through exactly what you do.
Junior
A developer pings you: the checkout service cannot reach the payments service, the calls just hang and time out. You have never worked on either service. Walk me through what you do.
Junior
A developer messages you: 'My app suddenly gets Access Denied writing to the S3 bucket. Nothing changed.' It worked yesterday. Walk me through what you do.
Mid
Your load balancer shows 5xx errors jumping from 0.1 percent to 12 percent of requests over ten minutes. There was no deploy. What do you do?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free