Design a document Q&A system over all internal knowledge, wikis, drives, tickets, for a 10,000-person company.
What they are really testing: The canonical AI system design prompt. Differentiators at senior: treating permissions as the hard requirement, ingestion as a living pipeline rather than a one-off load, and defining quality operations before scale theatre.
A real interview question
Design a document Q&A system over all internal knowledge, wikis, drives, tickets, for a 10,000-person company.
What most people say
drag me
“Embed all the documents into a vector database, retrieve the top chunks for each question, and have the model answer with citations, scaling the vector store as needed.”
The tutorial architecture, and at company scale its omissions are the design: no permissions, so the assistant leaks the salaries folder to anyone who asks; no deletion sync, so it cites removed documents; no eval story, so quality is a rumour.
The follow-ups they ask next
How do permissions stay correct when a user loses access to a folder today?
ACL sync propagates the change to chunk metadata within the sync SLA, and query-time filtering uses current membership, so the window equals the sync lag, which is why permission-change events get priority processing and an alert on lag.
A user asks something whose answer spans a wiki page and a ticket thread. What in the design serves that?
Retrieval is source-agnostic over one index, so both surface together; reranking selects across sources; and the model synthesises with per-claim citations. Sliced evals should include cross-source questions explicitly.
What the interviewer is listening for
- Leads with per-user permission enforcement at retrieval time
- Ingestion designed as continuous sync with deletion propagation and lag alerts
- Quality operations: labelled evals per source, sampled grading, feedback triage
What sinks the answer
- Permissions unmentioned or delegated to the model behaviour
- One-time indexing with no deletion or update story
- Scale discussion without a single quality measurement
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Three planes: [ingestion: incremental sync, structure-aware chunks with ACL metadata, deletion propagation], [query: permission-filtered hybrid retrieval, rerank, grounded generation with citations and honest refusal], [operations: per-source evals, sampled grading, feedback triage, tracing]. The hard requirement is [no answer derived from a document the caller cannot open].”
Keep going with system design
Senior
Design an AI customer support system for a SaaS product: 50,000 tickets a month, target 60 percent automated resolution without wrecking satisfaction.
Senior
Design an AI code-review assistant for a 200-engineer organisation. What does it check, and how do you stop it becoming noise everyone ignores?
Senior
Design the evaluation platform for a company running 12 LLM features. What does it provide, and how do teams use it without a central bottleneck?
Senior
Design an agent that automates invoice processing end to end: receive, extract, match to purchase orders, and schedule payment. Finance signs off on it. What does the design look like?
Senior
Design the model-serving layer for a company with 15 LLM features. One gateway or per-team integrations, and what lives in it?
Senior
Support reports the AI assistant got noticeably worse this week. Nothing was deployed. Walk me through the investigation.
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free