Do you actually need a dedicated vector database to ship a RAG feature? Walk me through how you would decide.
What they are really testing: Infrastructure judgment. The hype answer is a dedicated vector store on day 1, the engineering answer starts from corpus size and the database already in the stack, and knows pgvector holds surprisingly far.
A real interview question
Do you actually need a dedicated vector database to ship a RAG feature? Walk me through how you would decide.
What most people say
drag me
“Yes, RAG needs a vector database like Pinecone or Weaviate to do similarity search at scale.”
Tool-first reasoning with no numbers. Most RAG features launch on corpora that a Postgres extension serves in single-digit milliseconds, and the answer never asks how big the corpus even is.
The follow-ups they ask next
What goes wrong when vectors live in a separate store from the source data?
Drift: updates and deletes must propagate through a pipeline, and when it fails silently, retrieval serves stale or deleted content. Transactionality is the thing you gave up.
What is HNSW, roughly, and what is the trade-off it makes?
An approximate nearest neighbour graph index: much faster than exact search at the cost of exactness, tunable recall versus speed and memory.
What the interviewer is listening for
- Asks for corpus size and query rate before naming any product
- Knows pgvector and roughly where it stops scaling
- Prices the operational cost, sync drift, backups, security
What sinks the answer
- Reaches for a dedicated store with no numbers
- Unaware Postgres or existing stores can do vector search
- Ignores the consistency problem a second datastore creates
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“Decide on [vector count and query rate], not hype. Default: [pgvector in the Postgres we already run], which holds to [millions of vectors] and keeps [data and embeddings transactional]. A dedicated store earns it at [tens of millions of vectors or heavy filtered QPS], and the hidden cost is [the sync pipeline and a second system to operate].”
Keep going with rag & retrieval
Foundation
What is an embedding, and what does "similar" actually mean when you search with one?
Foundation
Explain RAG to me, and tell me what problem it solves that a bigger model does not.
Junior
How do you decide how to chunk documents for a RAG system, and what goes wrong with naive chunking?
Mid
What are hybrid search and reranking, and when does a RAG system actually need them?
Foundation
What is a token, and why does it matter that models bill and limit by tokens rather than words?
Foundation
Why do language models hallucinate, and why can you not simply prompt them to stop?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free