BLUF: RAG does not become enterprise-ready when the answer quality looks good in a demo. It becomes enterprise-ready when retrieval is permission-aware, sensitive fields are redacted before they reach the model, and every answer can be traced back through documents, principals, prompts, and model responses.
The failure mode is simple: a language model can only answer from the context it receives, but a retrieval layer can quietly assemble context that the user should never have seen. In enterprise knowledge bases this turns identity and access management into a retrieval-time control, not a portal-login feature.
Why RAG security is different from chatbot security
A normal chatbot security review focuses on prompt injection, output validation and model behaviour. RAG adds a second attack surface: the search index. If the index contains HR files, legal memos, customer contracts and engineering runbooks, then ranking and filtering become security-relevant operations.
OWASP lists prompt injection and sensitive information disclosure among the core LLM application risks. In RAG systems these two risks meet: an attacker can try to influence retrieval or force the model to reveal retrieved fragments that should have been filtered, summarized, or withheld.
Reference architecture: enforce access before generation
A practical enterprise RAG pipeline should separate six controls: identity resolution, document ACL ingestion, retrieval security trimming, redaction, generation, and audit logging. Do not rely on the LLM to “remember” policy. Policy must shape the context before the prompt is built.
Implementation pattern: 1) resolve the user and group principals from your identity provider; 2) store allowed principals with each indexed chunk; 3) apply a filter at query time so only authorized chunks are candidates; 4) redact regulated fields before prompt construction where the user has no need-to-know; 5) generate the answer with citations; 6) write immutable audit events for the query, retrieved chunk IDs, policy decisions, prompt hash, model ID and response ID.
Microsoft describes this pattern for Azure AI Search as security trimming: store user or group identifiers in a filterable field and use a query-time filter such as search.in so results are included only when the caller’s principals match the document principals. The same architectural principle applies outside Azure: filter before ranking or at least before prompt assembly, and test that unauthorized chunks never enter the model context.
Checklist: controls that decide whether enterprise RAG is safe enough
| Control | Good production default | Failure mode if missing |
|---|---|---|
| Document permissions | Ingest ACLs from the source system and preserve them per chunk | Users receive snippets from files they cannot open in SharePoint, Confluence, Google Drive or DMS. |
| Security trimming | Apply principal-based filters at retrieval time | Login works, but retrieval bypasses the source system’s authorization model. |
| Redaction | Remove or mask secrets, personal data and contractual values before prompt construction when not required | The model can quote sensitive values even if the UI later tries to hide them. |
| Audit logs | Record user, principals, query, retrieved chunk IDs, policy outcome, model and response metadata | Incidents cannot be reconstructed and compliance teams cannot verify what happened. |
| Evaluation | Test with unauthorized users, stale groups, poisoned documents and prompt-injection strings | The system passes happy-path demos but fails under realistic misuse. |
Permissions: treat chunks as records with ownership
The most common RAG mistake is flattening documents into chunks and losing the security context. Each chunk needs at least source document ID, source version, allowed principals, sensitivity label, retention class and ingestion timestamp. If a document permission changes, either update all affected chunks or exclude stale chunks until re-indexing completes.
For high-risk repositories, use deny-by-default indexing. A chunk without a valid ACL should not be searchable. If the source system cannot export reliable permissions, do not connect it to a broad employee-facing assistant until you have a compensating approval workflow.
Redaction: reduce what the model can see, not only what the user can see
Redaction after generation is weaker than redaction before generation. Once sensitive text is in the context window, it can influence summaries, reasoning and citations. Use deterministic redaction for known patterns such as API keys, IBANs, personnel numbers and contract amounts, then add human review for domains where false positives or false negatives carry material risk.
This does not mean redacting everything. Over-redaction destroys usefulness and creates shadow processes. The engineering target is purpose limitation: retrieve the minimum context that answers the user’s authorized business question.
Audit logs: what to capture without logging new secrets
NIST SP 800-53 includes audit and accountability controls, and the NIST AI RMF emphasizes governance, measurement and risk management across the AI lifecycle. For RAG, auditability needs to cover both application security and AI behaviour.
Log the user ID, resolved groups, query timestamp, source systems queried, retrieved document IDs, chunk IDs, policy decisions, redaction events, prompt template version, model name, response ID, latency, token usage and escalation decision. Avoid storing full prompts when they contain sensitive text unless your retention and access controls are ready for that new data store.
Operational trade-offs
Security trimming adds latency when users belong to many groups. Redaction can remove useful context. Detailed logs can become a sensitive dataset. Re-indexing after permission changes increases infrastructure cost. These are not reasons to skip controls; they are reasons to design budgets, cache invalidation and retention policies explicitly.
A workable starting point for many German B2B environments is: source connectors for two or three high-value repositories, deny-by-default chunk indexing, query-time ACL filters, deterministic secret redaction, answer citations, and a 30-day audit log with restricted security-team access. Expand only after evaluation shows that unauthorized retrieval, stale permissions and prompt injection tests are failing closed.
Where Ade can help
If your team is planning an internal RAG assistant, the technical question is not “which vector database should we use?” The first question is whether your retrieval layer can enforce the same trust boundaries as your enterprise systems. Ade Christanto helps B2B teams design and implement AI automation with permission-aware retrieval, auditability and operational controls that survive production use.
Sources
OWASP Top 10 for LLM Applications — prompt injection and sensitive information disclosure risks: https://owasp.org/www-project-top-10-for-large-language-model-applications/
Microsoft Azure AI Search security filters — query-time filtering with principal identifiers: https://learn.microsoft.com/en-us/azure/search/search-security-trimming-for-azure-search
NIST AI Risk Management Framework — governance and measurement framing for AI risk: https://www.nist.gov/itl/ai-risk-management-framework
NIST SP 800-53 Rev. 5 — security and privacy control catalogue including audit and accountability: https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final
