Your LLM feature must return JSON that downstream code parses. How do you make that reliable?
What they are really testing: The bread-and-butter integration problem. It separates people who ask nicely in the prompt from people who use schema enforcement and validate anyway, and it surfaces whether they know parse failures are retryable events, not crashes.
A real interview question
Your LLM feature must return JSON that downstream code parses. How do you make that reliable?
What most people say
drag me
“I put "respond only with valid JSON, no other text" in the prompt and give an example of the format.”
Prompt-only JSON works most of the time, and most of the time is a production incident schedule. It leaves markdown fences, trailing commentary and schema drift as regular events with no plan for any of them.
The follow-ups they ask next
Structured output mode is on, so why validate at all?
Constrained decoding guarantees syntax and shape, not semantics: wrong-but-valid values, hallucinated content in valid fields, and provider edge cases. The boundary check is cheap insurance.
How do you handle a field the model cannot determine from the input?
Make absence expressible: nullable field or explicit "unknown" enum value. A required field with no honest answer is a fabrication generator.
What the interviewer is listening for
- Reaches for schema-constrained output modes, not prompt begging
- Validates in code and treats the model as untrusted input
- Has a retry-with-error and dead-letter path designed
What sinks the answer
- Relies on prompt instructions alone for format
- No plan for invalid output beyond hoping
- Required fields that force fabrication
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Reliable JSON is [schema-enforced generation], plus [validation in code at the boundary], plus [a failure path: retry with the error, then dead-letter]. And design the schema so [enums constrain categories] and ["unknown" is expressible], so the model is never forced to fabricate.”
Keep going with structured output
Junior
How do you decide how to chunk documents for a RAG system, and what goes wrong with naive chunking?
Junior
What is prompt injection, and why is it a bigger deal once your model can use tools?
Junior
Your LLM feature works in the demo. What will it cost in production, and where do the surprises come from?
Junior
Users say your AI feature feels slow. What are the levers for making an LLM-backed feature feel fast?
Junior
How does tool calling actually work under the hood, and what makes a tool definition good versus bad?
Junior
You changed a prompt and the feature "seems better". How do you know it actually is?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free