Versioned sources in RAG: retrieval quality comes before answer generation
A practical framework for document identity, chunking, source versions, permissions and grounded response evaluation.
RAG combines two fallible components
In retrieval-augmented generation, a search component returns relevant passages and a language model uses that context to compose an answer. A well-written prompt cannot repair retrieval of an outdated or unrelated document. Evaluate search relevance and document freshness separately from the fluency of the generated answer.
Preserve document identity and versions
When a document changes, leaving chunks from old and new versions in the same index can produce contradictory answers. Store a document ID, version or timestamp, source URL and access classification on every chunk. Make index changes through a controlled transition; check that the replacement is queryable and correctly authorized before retiring the old version. Users should be able to trace cited text to its source.
Chunk sizes depend on the task
Very small chunks can lose surrounding meaning while oversized chunks bring unrelated content into the model's context. Respect headings and section boundaries where possible and retain offsets that point back to the original text. Evaluate chunk size, overlap and retrieval method against the same query set. Documents with similar words must not be treated as the same identity automatically.
Test whether answers are actually grounded
Include questions with no supported answer, documents with obsolete versions and tasks involving restricted material. Measure whether citations lead to real passages, whether the system avoids unsupported claims and whether it can decline when evidence is missing. Enforce access control in retrieval itself; telling a model not to use already retrieved private information is not a security boundary.
Comments
0 real commentsNo real reader comments yet.
✎ Write a comment