BLUF: a RAG application is not secure because users sign in. It is secure only when the identity presented to retrieval is translated into an allow-list, enforced before retrieval, and preserved in evidence. If a forbidden chunk enters the prompt, a later refusal cannot undo the disclosure.
The control point is retrieval, not the answer
Enterprise knowledge bases usually inherit permissions from SharePoint, file stores, ticketing systems, or line-of-business applications. The architecture must keep that source authority intact through ingestion, indexing, query execution, and observability. Authentication answers who is calling; authorization answers which document and chunk that caller may retrieve. OWASP recommends least privilege, deny by default, and permission validation on every request.
A practical sequence is: (1) resolve the caller and groups from the identity provider; (2) evaluate source ACLs or an approved entitlement projection; (3) apply the resulting filter inside the retrieval query; (4) retrieve only permitted chunks; (5) send only those chunks to the model; (6) write an auditable event. The model receives evidence, not authority.
Reference architecture: four controls that must agree
1. Stable identities and group claims
Use immutable subject and group identifiers rather than display names. Cache group expansion cautiously and define a revocation target: an access change is not effective until the index permission metadata and any claim cache have converged. Service-to-service retrieval needs its own workload identity; do not reuse a broad ingestion credential at query time.
2. Permission metadata on documents and chunks
Attach policy-relevant metadata during ingestion: source document ID, ACL version or hash, classification, tenant, and deletion state. Every derived chunk must inherit the parent document’s restriction unless a more restrictive chunk rule is explicitly designed and tested. A vector embedding is an index feature, not a security boundary.
3. Query-time enforcement before ranking
Put the authorization constraint into the retrieval engine’s filter, then rank within that permitted set. Post-retrieval trimming creates avoidable exposure: prohibited content may already be returned to the application, cache, trace, or model prompt. Microsoft documents this same pattern as document-level filtering at query time; its implementation options depend on source and identity model.
4. Evidence that supports incident response
Log request ID, pseudonymous subject ID, effective group-set version, policy decision, index/version, retrieved document IDs, chunk IDs, model invocation ID, and outcome. Avoid copying sensitive prompt or chunk text into routine logs. NIST’s Generative AI Profile calls out documentation, logging, change-management records, version history, and metadata as useful for incident response and accountability.
Choose the enforcement pattern deliberately
Security filters fit custom identity systems and push-index pipelines: the application supplies an approved user or group attribute and the search query filters on stored ACL metadata. Native source-permission synchronization can reduce duplicated identity logic when the platform supports it. Microsoft lists security filters, ACL/RBAC scopes, sensitivity labels, and SharePoint ACLs as distinct approaches; some integrations are preview features, so validate their synchronization lag, failure handling, and contractual suitability before making them a production dependency.
Production tests that catch the expensive failures
Build a fixed authorization evaluation set, not just relevance tests. Include: a user with no access; a user with access to one of two near-duplicate documents; a recently revoked user; a document moved between groups; an adversarial prompt asking for restricted material; and a request made through an agent tool rather than the chat UI. For every case assert that forbidden document IDs and chunks never appear in retrieved results, prompts, citations, caches, or logs.
Measure both security and operations: unauthorized-retrieval rate (target zero in the test suite), ACL synchronization delay, filter latency, empty-result rate after filtering, and the percentage of retrieval events with complete decision evidence. A rise in empty results may be a legitimate permissions issue, a stale group claim, or an indexing defect; do not “fix” it by widening access.
Limits and delivery plan
Document-level filtering does not solve prompt injection in allowed documents, weak model instructions, over-privileged tools, or data residency. It can also lower recall and increase query complexity. Treat it as one layer alongside source hygiene, content scanning, tool allow-lists, model-output controls, and a tested incident process. This article is implementation guidance, not legal advice.
Start with one bounded corpus and one identity source. Inventory its ACL semantics, prove deny-by-default tests, then add an audit event schema before extending to more repositories. That creates a defensible foundation for RAG quality work without turning the model into the permission system.
Primary sources
Microsoft Learn, Document-level access control in Azure AI Search: https://learn.microsoft.com/en-us/azure/search/search-document-level-access-overview
OWASP, Authorization Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html
NIST AI 600-1, Generative AI Profile: https://doi.org/10.6028/NIST.AI.600-1
