Architecting RAG that actually ships

Everyone can wire an embedding model to a vector store and get a demo working in an afternoon. Getting RAG to answer reliably from real data, with citations, in production — that's the actual work.

Start with retrieval, not the model

Most RAG quality problems are retrieval problems. If the right chunk never makes it into the context window, no model can save you. I spend most of my time on chunking strategy, embeddings and reranking before touching the prompt.

Chunk on meaning, not character count. Split on headings and semantic boundaries so a chunk is a self-contained idea. Overlap a little so context isn't cut mid-thought.

Rerank before you trust

Vector search gives you recall; a reranker gives you precision. Pull a wider set of candidates with embeddings, then rerank the top-k so the model reads the best few, not the merely-similar many.

Cite everything

Ground every answer in retrieved sources and surface the citations in the UI. It's the difference between a toy and a tool people trust — and it turns hallucinations into visible, debuggable retrieval misses.

The pattern that ships: chunk well, embed, retrieve wide, rerank, answer with citations, and log every step so you can see exactly where a bad answer came from.