What does temperature do, and how would you set it for a customer-facing extraction API versus a marketing copy generator?
What they are really testing: A small knob that reveals whether you have shipped anything. The scenario halves check you can map a sampling parameter to product requirements, determinism versus variety.
A real interview question
What does temperature do, and how would you set it for a customer-facing extraction API versus a marketing copy generator?
What most people say
drag me
“Temperature controls randomness, low is more deterministic and high is more creative.”
True and universally known. Without the mechanism or the mapping to the two products in the question, it is the answer of someone who read one blog post, and it dodges the actual scenario asked.
The follow-ups they ask next
Why might you still get different outputs at temperature 0?
Floating point non-determinism across hardware and batching, plus silent model version updates. Providers explicitly do not guarantee determinism.
What other levers change output besides temperature?
top_p nucleus sampling, frequency and presence penalties, stop sequences, max tokens, and most powerfully the prompt and any provided schema or tool definitions.
What the interviewer is listening for
- Explains it as reshaping a probability distribution, not magic randomness
- Gives concrete settings for both products with the product reason
- Knows temperature 0 is neither correctness nor guaranteed determinism
What sinks the answer
- Says low temperature makes the model more accurate
- One setting recommended for everything
- Cannot connect the parameter to a product requirement
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Temperature [rescales the next-token distribution before sampling]. Extraction API: [0, because consistency is the product and variance breaks parsers]. Marketing copy: [around 0.8, because regenerate must produce variety]. And [0 means consistent, not correct, and not even perfectly deterministic].”
Keep going with llm fundamentals
Foundation
What is a token, and why does it matter that models bill and limit by tokens rather than words?
Foundation
Why do language models hallucinate, and why can you not simply prompt them to stop?
Foundation
A model advertises a 200k context window. What can you actually rely on it for, and what not?
Junior
The model has no memory between API calls. How do chat products remember the conversation, and what goes wrong as it grows?
Foundation
What is an embedding, and what does "similar" actually mean when you search with one?
Foundation
Explain RAG to me, and tell me what problem it solves that a bigger model does not.
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free