When is fine-tuning actually the right call, and what does doing it properly involve?
What they are really testing: The counterpart to the foundation RAG-versus-fine-tune question, one level up: not just knowing the distinction but the decision discipline, exhaust cheaper options first, and the operational reality of data, evals and regression risk.
A real interview question
When is fine-tuning actually the right call, and what does doing it properly involve?
What most people say
drag me
“Fine-tuning is right when you need the model to perform better on your specific domain, you collect examples and train on them.”
"Perform better on your domain" is the phrase that launches a hundred doomed projects, most of which needed retrieval or three good examples in the prompt. And "collect examples and train" skips the data quality and regression testing where the actual work lives.
The follow-ups they ask next
What is LoRA and why did it change the economics here?
Parameter-efficient tuning: train small adapter matrices instead of all weights, cutting compute and memory dramatically and making per-task adapters cheap to store and swap.
Your fine-tune improved the target task but support noticed weirder answers elsewhere. What happened?
Catastrophic-forgetting-lite: narrow tuning shifted general behaviour. Caught by off-task regression evals; mitigated with more diverse training data, fewer epochs, or scoping the tuned model to the target task only.
What the interviewer is listening for
- Runs the prompt, few-shot, RAG ladder before reaching for tuning
- Names distillation economics with rough numbers
- Evals exist before training, including off-task regression checks
What sinks the answer
- Fine-tunes for factual knowledge or as the first resort
- No held-out set, no baseline, success judged by vibes
- Ships a tuned model with no versioning or re-tuning plan
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Fine-tune when [the task is stable, prompting, few-shot and RAG are exhausted, and economics justify it]: [style and format, distillation to a cheaper model, prompt-token savings, narrow domain language]. Properly means [clean representative examples with a held-out split], [baseline and post evals including off-task regressions], and [a versioned artifact with a re-tuning plan].”
Keep going with model adaptation
Foundation
Your team wants the chatbot to "know our product docs". Someone proposes fine-tuning on the docs. What do you say?
Mid
Users report your RAG assistant is giving wrong answers. Walk me through how you debug it.
Mid
What are hybrid search and reranking, and when does a RAG system actually need them?
Mid
What actually is an agent, mechanically, and what stops one from looping forever or going off the rails?
Mid
What problem does MCP, the Model Context Protocol, solve, and what should you check before plugging a third-party MCP server into your assistant?
Mid
Design the guardrails for a customer-facing AI assistant. What layers do you put around the model?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free