What actually belongs in a system prompt, and what is it unable to guarantee?
What they are really testing: Whether you treat the system prompt as configuration with known limits or as a magic contract. The second half separates people who have been burned in production from people who have not.
A real interview question
What actually belongs in a system prompt, and what is it unable to guarantee?
What most people say
drag me
“The system prompt tells the model how to behave, you put your rules there and it follows them.”
The word "follows" is doing illegal work. Models follow system prompts most of the time, and the gap between most and all is where every prompt-leak and jailbreak incident lives.
The follow-ups they ask next
Why does putting an API key rule only in the system prompt fail?
Prompt injection or persistent jailbreaking can override instructions. Secrets and permissions must be enforced by the application layer where the model cannot negotiate.
How do you test that a system prompt change did not break behaviour?
An eval suite: a fixed set of inputs with expected behaviours, run before and after the change, like regression tests. Never ship prompt edits on vibes.
What the interviewer is listening for
- Separates stable configuration from per-request content
- States plainly that it is influence, not enforcement
- Moves security-critical rules into application code
What sinks the answer
- Believes instructions in the system prompt are guaranteed
- Puts secrets or access rules in the prompt
- Edits prompts in production without regression evals
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“The system prompt carries [stable instructions: role, constraints, format], and stability also [makes it cacheable, cutting cost]. It cannot guarantee anything because [it is influence, not enforcement], so [security and correctness rules are enforced in code, outside the model].”
Keep going with prompting
Junior
People say prompt engineering became context engineering. What do you actually put in the context for a request, and in what order?
Mid
Your team has 30 prompts across 6 features, edited by 5 people. How do you manage prompts like production code?
Foundation
What is a token, and why does it matter that models bill and limit by tokens rather than words?
Foundation
Why do language models hallucinate, and why can you not simply prompt them to stop?
Foundation
A model advertises a 200k context window. What can you actually rely on it for, and what not?
Foundation
What is an embedding, and what does "similar" actually mean when you search with one?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free