RAG that survives contact with real documents

A vector store is not a product. Customers ask for the clause, the date, and the source PDF. Demos hide retrieval bugs behind a fluent paragraph. Production users screenshot the wrong citation and send it to legal.

We design retrieval around citations first. If the model cannot point at a passage, it does not get to speak. That single rule prevents most of the expensive hallucinations.

Chunk like a librarian

Naive 512-token windows smash tables and headings. We split on structure — titles, sections, captions — then embed. Recall goes up. Hallucinations go down. Tables stay tables.

We also keep the original file. A citation that cannot open the PDF is theatre. The UI shows page and snippet, not a vibe.

Evaluate on real documents

We freeze a set of questions from actual users: policy exceptions, fee schedules, last year's addendum. CI fails when retrieval misses the right chunk or the answer drops the citation.

RAG that survives production is boring infrastructure with a strict mouth. That is the product.