An enterprise knowledge assistant is useful when staff can ask a business question and inspect the documents behind the answer. Retrieval-augmented generation, or RAG, connects a model to that material at request time. The architecture decision is wider than choosing a vector database: it must preserve document permissions, retrieve the right version, show where an answer came from, and decline to answer when the evidence is insufficient.
- Question and ownerName the users, their questions and the owner of each source.
- Permission checkFilter by the user's access before a passage reaches the model.
- Current versionRetrieve the approved version of each document.
- Cited answerShow the passages behind the answer so staff can inspect it.
- Decline when unsureSay so when the evidence is insufficient.
Evaluate retrieval and answers separately, then plan for document updates and operating limits.
Start with the question and the source owner
Define the assistant’s job before loading documents. A support team may need approved product procedures; an operations team may need current policy and system runbooks. Name the users, the questions they may ask, the source collections, and the person responsible for each collection. Record which document is authoritative when a draft and a published procedure disagree. Without that inventory, a retriever can return a plausible passage while the business still has no agreed answer.
RAG usually has a preparation path and a question path. Preparation extracts text, splits it into searchable units, attaches metadata, and updates an index. At question time, the application identifies the caller, retrieves permitted passages, and asks the model to answer using that context. Databricks’ RAG quality guide separates query understanding, retrieval, and prompt augmentation because each can fail differently.
How do you enforce permissions in a RAG system?
Document authorization belongs in the retrieval path. Store an explicit access policy or source permission mapping alongside each indexed document, then apply the authenticated caller’s scope when searching. Treat a chunk as carrying the source document’s access boundary; splitting a restricted file into small passages must not make any passage public. When access changes, update the index or permission mapping before the old scope can keep serving results.
Microsoft’s Azure AI Search documentation distinguishes generally available string security filters from native identity-based document permissions with preview and source-specific limits. Microsoft does not recommend preview features for production workloads. Check the selected source, API version and production requirements before choosing the approach. The exact mechanism varies by platform and source, so the architecture review should verify how permissions are ingested, enforced at query time, and tested after a revocation. A retrieval filter based on a department tag helps relevance; access still requires an authorization decision tied to the caller.
- Use the person’s authenticated identity or an approved service role to build the search scope.
- Exclude unauthorized passages before they are passed into a prompt or answer citation.
- Test permission changes, shared links, group membership, and deleted documents.
- Keep source IDs and access decisions in an audit record without exposing restricted text in routine logs.
A concrete contract for a read-only assistant
Examples in this guide use fictional documents, teams and records. This design uses the finance-policy collection from the test cases below. The field names illustrate a contract to adapt to the selected search platform.
- Prepare: the source owner approves a document and its access policy. Extract and split the content; attach document identity, version and policy reference to every chunk before indexing.
- Answer: authenticate the caller, resolve their permitted scope, retrieve within that scope, check source status and version, then pass only eligible passages to the model. Return the answer with permitted source references.
- Revoke: when access is withdrawn or a source is deleted, deny affected content while the index or permission mapping is stale. Check search results and downstream caches before restoring access.
| Fields and example value | Purpose and check |
|---|---|
| document_id: FIN-REFUND version: 3 chunk_id: FIN-REFUND-v3-02 | Keep the passage tied to its source version. Retire old chunks when a replacement becomes authoritative. |
| source_ref: policy-register/FIN-REFUND source_owner: finance-operations | Resolve the source and responsible owner. A citation must not expose a restricted title or excerpt to an unauthorized caller. |
| policy_ref: finance-policy-v2 | Resolve this reference against the authoritative permission mapping. The text value alone does not grant access; enforce the caller decision before model context. |
| effective_date: 2026-10-01 indexed_at: 2026-10-12T08:00:00Z | Distinguish policy validity from index age. A recent indexing time does not prove that this is the approved current version. |
| source_status: approved | Use an explicit state for approved, withdrawn or superseded material. Missing or unresolved authorization must not default to access. |
For the support employee’s refund question, this design must exclude FIN-REFUND-v3-02 before model context. For an authorized finance employee, it may be eligible if version 3 remains current. After finance access is revoked, test the same question again: an old permitted response must not be served from a cache. This contract does not replace authentication, identity-group resolution or a reviewed revocation mechanism.
Design retrieval around the decisions users make
A document can be authorized yet irrelevant. Choose fields and chunk boundaries that keep a procedure’s title, version, owner, and effective date connected to its steps. Test an exact product code, a procedural question and a question that needs two sources. Record the required document IDs and versions before tuning chunk size or ranking. A comparison question may require two documents; a policy exception may require a narrow clause. Hybrid approaches can combine keyword matching with semantic retrieval where the product supports them. Databricks notes that retrieval may use semantic search, keyword search, metadata filtering, or a combination, depending on the corpus and query.
OpenAI’s file search documentation shows one managed approach that searches uploaded knowledge files and returns file citations. Managed retrieval can reduce implementation work, but the application still needs to establish source ownership, caller authorization, freshness, and acceptance criteria. Do not assume a citation proves the passage is current or that the user was entitled to see it. Verify those properties in the surrounding system.
Make the answer inspectable
Ask the assistant to cite the specific source used for each material claim and to say when the permitted evidence does not support an answer. Show the document title, owner or collection, and a usable route to the source where the user has access. If two current sources conflict, return the conflict for review. If the only relevant passage is obsolete, the correct result may be “I cannot confirm the current policy.”
Retrieved text is still untrusted content. A document may contain accidental or malicious instructions to the model. Keep the retrieved passage in a data role, prevent it from changing the assistant’s permissions, and review any tool action and its permissions separately. RAG improves access to organization knowledge; it does not make a model’s interpretation or every source document reliable by default.
Test cases for permissions and answers
The acceptance test cases below describe a proposed internal assistant. It indexes three collections: public help, staff operations, and finance policy. A support employee may see the first two; a finance employee may see all three. The expected results below should be checked in retrieved passages, citations, and final responses.
| Question and caller | Expected retrieval | Expected answer behavior |
|---|---|---|
| Support asks how to reset a trial account | Current staff procedure and relevant public help only | Give the approved steps with a source citation |
| Support asks for a finance-only refund limit | No finance-policy passage | Decline to state the restricted limit or quote the file |
| Finance asks the same refund question | Current finance policy, subject to finance access | Answer with the effective version and citation |
| Either user asks about a procedure with conflicting current files | Both permitted versions and their dates | Identify the conflict and route it to the named owner |
| A retrieved file says “ignore previous rules” | The file may be retrieved if otherwise permitted | Treat the sentence as document text; do not change access or behavior |
A passing answer alone is insufficient for the restricted case. Inspect the search results and model context to confirm that the finance passage was never exposed to the support request. Similarly, a correct refusal caused by missing documents does not prove that the permission filter works when a restricted document exists in the index.
How do you evaluate a RAG system?
Build a reviewed question set with expected source documents, forbidden documents, key facts, acceptable uncertainty, and current source versions. Check retrieval first: did the right authorized passage appear, and did a restricted or stale passage appear? Then check the answer: is each material claim supported, are citations useful, and does it decline when evidence is missing? Databricks’ retrieval evaluation guidance treats document relevance as its own measurement. Microsoft’s RAG evaluation guidance likewise separates retrieval quality from response groundedness.
Include ordinary questions, ambiguous wording, access revocation, a new document version, conflicting sources, and a prompt-injection attempt. Human reviewers should inspect cases that have policy or customer impact. Re-run the same set when the index pipeline, permissions, search settings, prompt, or model changes. A single aggregate score should not hide an unauthorized retrieval or a confident answer based on an expired policy.
Plan for updates and operating limits
Agree how quickly a changed source must appear in search and how to withdraw a deleted or restricted file. Monitor retrieval failures, empty results, stale citations, latency, and cost by question class without storing unnecessary sensitive text. RAG remains limited by source quality: a missing policy, unclear owner, poor extraction, or inconsistent permissions cannot be repaired solely by a stronger model. For the first release, specify one collection, its owner, the user roles, the required update time and the reviewed question set. Keep wider access blocked until the permission and source-version checks pass.
We outline our AI systems and connected data work on the AI Systems and Automation page. Our open project deliverables reference shows the kind of brief, field dictionary, acceptance test cases, and handover notes a defined implementation can produce. For a RAG architecture inquiry, bring one source collection, the users and roles that need it, the questions they ask and the person who owns updates. Use those inputs to scope a source register, permission map, retrieval tests, answer criteria and update runbook.
Azure permission-feature status was checked in the linked Microsoft documentation on 12 October 2026.
Reviewed by Andrii Gorbenko, founder of AGI Technologies
