RAG Document Permissions: Where ACL Filters Leak

A retrieval system does not know who is asking. The permission check runs at query time against a copy of the document's access list that was written into the index when the content was ingested, and nothing in that path re-asks the source system. So three things can be wrong while your filter expression is perfectly correct: the chunks never inherited the parent document's access list, the permission you fixed was inherited from a folder and never re-synced, or the identity string your application passes into the filter is not a validated token. The first symptom of a permission-aware rollout is usually not a leak. It is your lowest-access users quietly getting worse answers than you get, and you will not notice, because you tested as an admin.

The access list in your index is a copy, not a check

Your source of truth is Google Drive, SharePoint, or a table in your own database. At ingest, whatever permission lives there gets flattened into a field on the indexed record. At query time the engine compares the caller's identity to that field and drops the rest. That is the whole mechanism.

Microsoft is unusually blunt about what that field actually is. In the Azure AI Search security-filter pattern, the one most hand-built pipelines copy, the documentation states there is "no authentication or authorization through the security principal" and that "the principal is just a string, used in a filter expression, to include or exclude a document from the search results."

Read that as a warning about your own code. Whatever your application puts into the string is the authorization decision. If a workflow step takes a user ID out of a request body and drops it into the filter, you have built an object-reference vulnerability with vector search behind it. The identity has to be derived server-side from a validated session, never from a value the caller can edit. This is the same discipline as deciding how much access to give an AI agent: the boundary has to be enforced by something the model and the caller cannot reach.

Here is where the same permission has to be restated in a typical build.

Layer	Carries the access list by default	What goes wrong when it does not
Source system (Drive, SharePoint, your database)	Yes, this is the source of truth	Nothing, but it is often the only place anyone checks
Indexed document record	No, you write it at ingest	The filter matches nothing, or everything, depending on the engine
Each chunk vector	No, it needs an explicit projection from the parent	Retrieval returns text the caller cannot open in the source app
Cached answers, semantic or exact	No, the cache key is usually just the question	One user is served an answer computed from another user's files
Citation and source list in the reply	Only if assembled from the trimmed result set	Filenames and titles leak even when the answer body was trimmed
Pre-computed summaries and digests	No, they were built across the whole corpus	A digest written from restricted files is served to everyone

The cache row is the one people rediscover the hard way. If you run a shared cache in front of the model, the key has to include the identity boundary the answer depends on, which is covered in more detail in semantic caching and the unscoped cache key.

Retrieval returns chunks, and chunks do not inherit the access list

The unit you set permissions on is a file. The unit retrieval returns is a chunk, usually a few hundred to a couple thousand tokens of that file. A splitting step sits between the two, and permission metadata does not cross it on its own.

Microsoft documents the requirement exactly: when a skillset chunks documents, the ACL fields move out of indexer field mappings and into index projections, and "without this projection, chunk-level references aren't filtered." The same trap is easier to fall into on a hand-built pipeline. You write group_ids onto the parent record in your ingest script, then upsert 40 chunk vectors carrying doc_id, text, and page, and nothing else. Every filtered query after that matches on a field the chunks do not have.

There is a five-minute test. Pull one chunk out of your index by ID and look at its payload. If the access list is not on that record, you do not have document-level permissions. You have a document-level intention.

The permission you fixed is often the one that never syncs

When someone discovers oversharing, they fix it at the folder or the site. That is the right fix in the source system and the one least likely to reach your index.

Azure's SharePoint connector spells out the split. As of the 2026-05-01 preview, changes on items with unique permissions are detected and refreshed on each successful indexer run, while changes inherited from a parent scope, meaning the site, library, list, or folder, require an explicit refresh such as a resync with the permissions option or a document reset. The overview page states the general rule: permission changes in the source system "are only reflected in search results after that metadata is synchronized to the index," and the preview notes warn plainly that a timing lag occurs before permission changes are recognized.

Put those together and the practical consequence is backwards from what you would guess. Revoke one person on one file and the index catches up on the next run. Lock down the entire folder of 4,000 files that had been shared with everyone, and the assistant keeps answering out of them until a human runs a permission resync. The single most common remediation action is the one that does not propagate.

So the revocation window is a configuration decision you make before the first request, not a runbook you execute after one. It is the same shape as the erasure problem covered in deleting customer data from an AI stack: for anything you can only expire rather than delete, the number that matters is the length of the window. If your answer to "how long after I remove someone does the assistant stop quoting their files" is "I would have to check," that unknown number is your exposure, and it is usually measured in days.

Filtering a vector index is not a WHERE clause

Operators assume the access-list filter behaves like SQL. It does not, and the reason is the index structure. HNSW is a navigable graph, and search walks from node to neighboring node. Remove most of the points and you can cut the paths between the rest.

Qdrant's writeup on filtering says it directly. With low-cardinality pre-filtering, "the filter becomes restrictive and it can disrupt the connections within the graph," and once points are disconnected, "vector search can't cross the grayed out area and it won't reach the nearest neighbor." Post-filtering fails from the opposite end: you retrieve the global top 20 and then drop what the user cannot see, so "if your desired items aren't in this initial set, you won't find them, even if they exist in the database." A dispatcher with access to two percent of the corpus gets a top 20 made of 19 documents they cannot see and one they can.

Both failures look identical from the outside. The assistant is vaguer, hedges more, or says it does not have that information. Nobody files that as a security bug. It gets filed as "the AI is not very good," or it does not get filed at all.

This is also why the permission boundary belongs in the retrieval design instead of bolted on at the end. Where most users see a thin, stable slice of the corpus, that slice should be a partition or a namespace rather than a needle-in-haystack filter over one giant index. Options differ by engine: a collection per boundary, tenant-aware index parameters, or Postgres row-level security with pgvector so the query planner sees the constraint as part of the query. Which one fits depends on how many boundaries you have and how much they overlap, which is part of the same do you actually need a vector database decision. One thing to rule out first: if the right chunk is being retrieved for the narrow account but ranked below a wrong one, that is a precision problem, not a permission problem, and it belongs in reranking.

Test it as your narrowest user, not as yourself

You almost certainly have a set of test questions for the assistant, and you almost certainly built it while logged in as an admin. That set cannot detect any of the failures above.

Get the account with the least access in the company: a new hire, a part-time dispatcher, one location's supervisor. Run the same 20 questions through both accounts and put the answers side by side. There are only four things you can see.

What the narrow account gets	What it means	Where to look
The same answer as the admin, quoting a file that account cannot open in Drive	The access list never reached the chunk records	Chunk payloads and the ingest projection
A correctly trimmed answer, but a citation naming a file the account cannot open	Retrieval was filtered, the source list was not	Wherever the citation list is assembled
A much vaguer answer, or "I do not have that information," when the account does own a relevant document	Filtered recall collapse, not a permission bug	Pre-filter versus post-filter, partitioning, top-k
A correct trimmed answer	Working as designed for that boundary	Nothing, keep the pair as a regression test

Then keep both accounts and re-run the pair after any change to chunk size, embedding model, top-k, or the source connector. Those four changes silently break permission propagation, and not one of them looks like a security change in a pull request.

How to start

Pick the narrowest real account you have and ask the assistant three questions that are answered only in documents that account cannot open. It takes ten minutes and tells you more than a design review will. If it answers any of the three, stop the rollout and go check whether the access list is on the chunk records at all.

Then write down two numbers before you scale it to everyone: how long after a permission change the index reflects it, and whether each user's boundary is enforced by a filter or by a partition. Both are cheap to decide now and expensive to retrofit. If you are wiring an internal assistant onto a shared drive and want the permission model checked before the whole company gets the link, that is the kind of build we work on in custom software platforms, and you can tell us what it is pointed at.