BLUF: treat every retrieved chunk as a data-access decision, not as harmless context. A production RAG system should decide what a caller may retrieve before retrieval, avoid passing forbidden text to the model, and record enough evidence to reconstruct the decision. A vector database alone does none of this.
The failure mode is architectural
Enterprise knowledge bases combine HR policies, customer material, engineering runbooks and contract documents. A chatbot can return a plausible answer while leaking one sentence from a document the user was never allowed to open. Removing citations after generation is too late: the model has already received the content, and the answer may reveal it indirectly.
OWASP calls out sensitive-information disclosure and vector/embedding weaknesses as LLM application risks. NIST AI 600-1 similarly puts data privacy, governance and documented controls inside the operating model for generative AI. These are implementation requirements, not a reason to avoid RAG.
A production control plane: classify, authorize, retrieve, prove
1. Classify at ingestion. Extract source system, owner, business domain, sensitivity, retention class, legal hold and allowed groups. Keep the source document identifier and version beside every chunk. Do not infer clearance from an embedding.
2. Authorize before vector search. Resolve the user or service identity into stable group and attribute claims. Apply those claims as metadata filters in the retrieval query. If the vector store cannot enforce filters reliably, put an authorization-aware retrieval service in front of it or choose a different store.
3. Redact deliberately. Keep a source-of-truth copy under the original policy, and create a retrieval representation that removes credentials, personal identifiers or restricted fields where the use case does not need them. Redaction quality needs sampled review; pattern matching alone misses context.
4. Prove each answer. Log request identity or a privacy-preserving subject reference, policy version, retrieval filter, returned chunk IDs, source versions, model deployment and final decision. Restrict access to those logs and set a retention period; audit logs can themselves be sensitive.
Decision matrix: where each control belongs
| Ingestion | classify, label, redact, attach owner and retention metadata | prevents ungoverned content from entering the index. |
|---|---|---|
| Retrieval | enforce document- and chunk-level filters from identity claims | prevents unauthorized context reaching the model. |
| Generation | constrain tools, cite sources, detect obvious sensitive patterns | reduces residual disclosure but cannot replace retrieval controls. |
| Operations | record policy version, chunk IDs and exceptions; test revocation | makes incidents investigable and deletion verifiable. |
Deletion and revocation are the real test
A user leaving a project, a document moving under legal hold, or a supplier portal being retired must change access quickly. Design for an explicit revocation event: remove or invalidate source records, chunks and embeddings; invalidate caches; and record completion. “The nightly re-index will eventually catch up” is not an adequate control for sensitive data.
Measure the path with four tests: an unauthorized-user retrieval test, a changed-group-membership test, a deleted-document test and an indirect prompt-injection test against retrieved content. Run them in CI for policy code and on a representative staging index before connector changes.
A practical 30-day implementation sequence
Week 1: inventory connectors and select one bounded corpus with a named owner. Week 2: define the metadata contract and map identity groups to retrieval filters. Week 3: add trace records and the four negative tests. Week 4: run an access-revocation drill and review sampled redactions with the document owner. Expand only after the drill has a measurable result.
What this does not solve
These controls do not make an AI system compliant by themselves, and they do not eliminate prompt injection, overbroad source permissions or a compromised identity provider. Legal classification, retention duties and data-protection obligations require review by qualified legal and privacy professionals. Engineering should make the evidence available for that review.
Sources
NIST, AI RMF: Generative AI Profile (NIST AI 600-1), 2024: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
OWASP, RAG Security Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/RAG_Security_Cheat_Sheet.html
OWASP Top 10 for LLM Applications 2025: https://owasp.org/www-project-top-10-for-large-language-model-applications/
Build the control plane before scaling the index
If you are moving a knowledge assistant from pilot to production, I help teams turn existing identity, document and logging systems into an authorization-aware RAG architecture—with a bounded pilot, negative tests and operational handover.
