Legal asks: what happens to our customer data when we use third-party LLM APIs, and what controls do we need? Answer them.
What they are really testing: Senior engineers get this exact meeting. It tests knowing the actual data flows, prompts, logs, traces, indexes, the difference between consumer and enterprise API terms, and designing controls that work rather than banning everything.
A real interview question
Legal asks: what happens to our customer data when we use third-party LLM APIs, and what controls do we need? Answer them.
What most people say
drag me
“Enterprise API terms say they do not train on your data, so we are covered as long as we use the API rather than the consumer apps.”
One true fact standing in for an analysis. It says nothing about retention windows, our own logs and indexes, which data even needs to be sent, regional processing, or what we tell customers, which is most of what legal actually asked.
The follow-ups they ask next
A customer contract forbids their data leaving the EU. What breaks and what do you do?
Provider regional endpoints and data-residency options where offered, regional routing at the gateway keyed on tenant, and excluding that tenant data from any US-processed features until covered.
How do RAG permissions actually get enforced at query time?
Retrieval filters by the caller effective permissions, ACL metadata on chunks checked per query, synced from the source system. Post-hoc filtering of generated answers is too late, the leak already entered the context.
What the interviewer is listening for
- Maps all three flows: provider transit, own trace stores, RAG indexes
- Knows enterprise API terms concretely: no-training, retention windows, DPA
- Treats retrieval as a permission surface enforced per user at query time
What sinks the answer
- Provider terms cited as the entire answer
- Trace and log stores unconsidered as data copies
- RAG index flattens document permissions and nobody noticed
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Data goes [three places: through the provider, into our own traces and logs, and into RAG indexes]. Controls per hop: [enterprise terms, DPA, bounded retention at the provider], [classification, access control and redaction on our trace stores], [per-user permission enforcement at retrieval], plus [minimisation at the boundary and documented flows for audits].”
Keep going with security
Junior
What is prompt injection, and why is it a bigger deal once your model can use tools?
Mid
Design the guardrails for a customer-facing AI assistant. What layers do you put around the model?
Senior
Design the model-serving layer for a company with 15 LLM features. One gateway or per-team integrations, and what lives in it?
Senior
Support reports the AI assistant got noticeably worse this week. Nothing was deployed. Walk me through the investigation.
Senior
When do multi-agent architectures actually earn their complexity over one well-tooled agent, and what fails in them?
Senior
The CTO asks whether you should build your AI capability on frontier APIs or self-host open-weight models. How do you frame the decision?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free