Your team wants the chatbot to "know our product docs". Someone proposes fine-tuning on the docs. What do you say?
What they are really testing: The single most common architecture misconception in the field. Fine-tuning teaches behaviour and style, not reliable factual recall, and this scenario is the cheapest way to detect who knows that.
A real interview question
Your team wants the chatbot to "know our product docs". Someone proposes fine-tuning on the docs. What do you say?
What most people say
drag me
“Fine-tuning on the docs sounds reasonable, the model will learn the product information.”
Agreeing is the failing answer here. A fine-tune on a doc corpus produces a model that sounds like the docs while still fabricating details from them, and each docs release now requires a training run.
The follow-ups they ask next
When would fine-tuning plus RAG together be right?
RAG supplies the facts, a fine-tune teaches the answering style, format and domain jargon, common in support bots once RAG alone plateaus on tone and structure.
The team says fine-tuning worked in their test. What do you check?
Whether the eval questions leak from the training data, whether recall holds on updated facts, and the hallucination rate on questions just outside the fine-tuned set.
What the interviewer is listening for
- Draws the knowledge-versus-behaviour line immediately
- Names the update problem, retraining per docs release
- Keeps fine-tuning for style, format and cost distillation
What sinks the answer
- Endorses fine-tuning for factual knowledge
- Never mentions how updates would work
- Chooses an architecture with no eval plan
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Fine-tuning teaches [behaviour and style], not [reliable factual recall]. "Know our docs" is a knowledge problem, so [RAG: index, retrieve, cite], where updates are [seconds, not a training run]. Fine-tuning stays for [tone, format, and distilling tasks onto cheaper models].”
Keep going with model adaptation
Mid
When is fine-tuning actually the right call, and what does doing it properly involve?
Foundation
What is a token, and why does it matter that models bill and limit by tokens rather than words?
Foundation
Why do language models hallucinate, and why can you not simply prompt them to stop?
Foundation
A model advertises a 200k context window. What can you actually rely on it for, and what not?
Foundation
What is an embedding, and what does "similar" actually mean when you search with one?
Foundation
Explain RAG to me, and tell me what problem it solves that a bigger model does not.
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free