A rollout of your payments service has been stuck half done for eleven minutes. Four new pods are Ready, six old pods are still serving, and kubectl rollout status just hangs. Error rate is up slightly but not enough to page. The release manager wants to know if he should tell the business we are down. What do you say and what do you do?
What they are really testing: They are testing your judgment under ambiguity, not your kubectl. A half done rollout is a system running two versions at once, and they want to hear you decide quickly whether to go forward or back, and say it in a sentence a non engineer can act on.
A real interview question
A rollout of your payments service has been stuck half done for eleven minutes. Four new pods are Ready, six old pods are still serving, and kubectl rollout status just hangs. Error rate is up slightly but not enough to page. The release manager wants to know if he should tell the business we are down. What do you say and what do you do?
What most people say
drag me
“I would keep watching the rollout for a bit longer and hope the remaining pods come up. If they do not, I would look at the logs and try to work out what is wrong with the new version.”
It reads junior because it makes no decision, and a half done rollout of a payments service is a decision, not a status. Hoping is not a strategy, and every extra minute the system runs two versions is a minute of risk that nobody has agreed to take.
The follow-ups they ask next
The new version contains a database migration that has already run. Does that change your rollback?
Yes, and say so immediately. A rollback is only safe if the old code can read the new schema. If the migration was not backward compatible, rolling back is the dangerous move, so you go forward, and you say that out loud rather than reflexively undoing.
The four new pods are Ready but the six old ones will not terminate. Why?
Look at the PodDisruptionBudget and at terminationGracePeriodSeconds, and at whether a preStop hook or a stuck connection drain is holding them. A PDB with minAvailable equal to the replica count will freeze a rollout forever.
What would have made this fail loudly instead of hanging?
A sane progressDeadlineSeconds, an alert on rollout status rather than a human watching a terminal, and a deploy pipeline that automatically rolls back when the deadline is exceeded.
What the interviewer is listening for
- Answers the non engineer in plain language before touching kubectl
- Treats running two versions of a payments service as a risk on its own
- Puts a time box on the decision instead of watching and hoping
- Names PDB, quota, readiness and progressDeadlineSeconds as the four usual causes
- Checks migration compatibility before committing to a rollback
What sinks the answer
- Waits it out with no deadline and no decision
- Rolls back reflexively without asking whether a schema migration already ran
- Cannot explain how a PodDisruptionBudget can freeze a rollout
- Gives the release manager a technical answer he cannot act on
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“I have not had a payments rollout hang on me, but here is how I would reason about it. A half done rollout means we are running two versions at once, so the first thing I owe the release manager is a plain sentence and a deadline, and my default would be to get back to one version before I go digging, unless a migration has already run and made the rollback the riskier move.”
Keep going with troubleshooting
Junior
You get paged at 2am. Checkout latency has gone from 200ms to 4 seconds and it started right after the 11pm deploy. Walk me through what you do.
Junior
It is 3am and you get paged: the primary application server is at 100% disk usage and the app is throwing write errors. You are the only one online. Walk me through exactly what you do.
Junior
You push a new version of a service and within two minutes the pods are all showing CrashLoopBackOff. The team is watching you share your screen. Walk me through exactly what you do.
Junior
A developer pings you: the checkout service cannot reach the payments service, the calls just hang and time out. You have never worked on either service. Walk me through what you do.
Junior
A developer messages you: 'My app suddenly gets Access Denied writing to the S3 bucket. Nothing changed.' It worked yesterday. Walk me through what you do.
Mid
Your load balancer shows 5xx errors jumping from 0.1 percent to 12 percent of requests over ten minutes. There was no deploy. What do you do?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free