How to Secure RAG Workflows

Retrieval-Augmented Generation (RAG) workflows connect AI systems to enterprise knowledge bases, making retrieval authorization the most consequential security decision in the pipeline. An agent with valid credentials can surface documents that the requesting user is not authorized to access without triggering traditional access-control flags. Data and model defenses are necessary, but neither alone governs retrieval authorization. By authenticating agents, enforcing authorization at query time, and propagating user context across every retrieval hop, an identity security fabric anchored by an identity control plane can help secure RAG workflows.

The three layers of RAG security

RAG workflows introduce risk at three distinct layers. Security guidance often focuses on two of them.

While the data and model layers are essential, neither answers the core RAG question: is this identity, human or agent, authorized to retrieve and act on this information? In many deployments, an agent’s technical access and the requesting user’s actual entitlements are two different things. That gap is the core identity problem in RAG security.

The RAG security threat model, mapped to OWASP LLM Top 10

The OWASP Top 10 for LLM Applications is a widely referenced framework for identifying security risks in AI systems. Several of the top risks have a direct identity dimension in RAG architectures.

Identity governance is a crucial factor in each of these risks. An agent acting beyond its authority, or a user’s entitlements left unenforced downstream, can turn any of these vulnerabilities into a data exposure event.

How to secure RAG workflows step by step

The data and model layers have mature playbooks. The steps below focus on the identity layer, where most RAG architectures still leave gaps, and briefly cover data and model controls.

1. Secure the data layer: ingestion, vector stores, and retrieval

Source validation filters out documents from untrusted origins before they reach the vector store. Data classification tags content by sensitivity level at ingestion, enabling downstream authorization decisions. PII filtering prevents personally identifiable information from being embedded and indexed, reducing exposure before retrieval authorization is ever evaluated.

Two distinct threats operate at this layer and require separate defenses. Data poisoning corrupts the knowledge base during ingestion through malicious or tampered-with source documents. Indirect prompt injection embeds instructions in otherwise legitimate content to hijack agent behavior at inference time, and is covered at the model layer in step 5. Identity controls are an effective upstream check against indirect prompt injection. When an access policy restricts a document, the agent never retrieves it, regardless of its contents.

In the vector store, encryption at rest and in transit is baseline. Tenant segmentation keeps embeddings isolated across tenants in multi-tenant architectures.

Encryption and filtering protect data from outsiders. They do not stop a legitimate agent from retrieving data that the requesting user should not access. That is an authorization issue.

2. Enforce fine-grained authorization on retrieval

Coarse-grained RBAC was designed for human users with predictable job functions. In a RAG workflow, a user’s entitlement to a folder does not imply entitlement to every document it contains. Role-level gates are too coarse. Document-level access controls, and in some high-assurance architectures, chunk-level controls, require a more granular policy model. Two established FGA approaches apply to RAG retrieval:

• ReBAC (relationship-based access control) grants access based on the user’s relationship to the resource (e.g., project membership, ownership, or organizational hierarchy).
• ABAC (attribute-based access control) evaluates user, resource, and context attributes, which are useful when decisions depend on data classification tags.

An important distinction: metadata filtering in the vector store is a policy enforcement point (PEP), but the policy decision point (PDP) should sit in the identity control plane. The vector store enforces decisions issued by the policy engine; it does not make them. Authorization must be evaluated by the retriever acting with delegated authority on behalf of the user, not only using the agent's own system credentials. Collapsing the PDP into the PEP, or into the retriever's own credentials, is a common architectural failure.

3. Establish identity for AI agents

AI agents should be treated as non-human identities within the enterprise identity security fabric. Without centralized identity governance, RAG pipelines and the agents that serve them can proliferate across teams without inventory, ownership, or access controls, creating shadow AI. Discovery and posture management for non-human identities are prerequisites for governing them. You cannot enforce least privilege on agents you have not inventoried. Every agent requires a unique identity, no shared API keys, or over-scoped service accounts.

How an agent authenticates itself determines the extent of damage a compromised credential can cause. OAuth 2.0 client credentials and mTLS both require a stored secret or certificate, though both support rotation. Workload identity federation eliminates the need for long-lived stored secrets. The platform issues a signed identity assertion that the agent exchanges for a scoped, short-lived access token. When properly implemented and rotated, each approach offers stronger security properties than shared keys or standing service accounts.

4. Propagate user context with OAuth 2.0 Token Exchange

In agentic RAG, the confused deputy problem occurs when an agent with valid credentials retrieves data that the requesting user is not authorized to access. OAuth 2.0 Token Exchange (RFC 8693) is one of the standardized responses. Under RFC 8693, the user's identity token is the subject and the agent's identity is the actor, establishing a verifiable delegation chain. 

That delegation direction helps mitigate the confused deputy problem. The agent cannot act beyond what the user is permitted to see. Effective access is bounded by the requesting user's entitlements and further constrained by the agent's authorized scopes, never the union of the two. That constraint is what closes the authorization gap at every hop.

5. Defend the prompt and output layer

Input sanitization, output filtering, guardrails, and tool-call validation apply at the system prompt and at every point where external content enters the context window. Indirect prompt injection is the RAG-specific risk to prioritize. The retrieval step is an authorized path through which untrusted content can reach the model. The OWASP Top 10 for LLM Applications provides widely adopted implementation guidance for this layer.

6. Monitor, log, and audit every agent action

Every retrieval, prompt, tool call, and response should be logged with agent identity context (which agent, acting under whose delegation, accessed what, and when). Compliance audits can then confirm that authorization controls were enforced, not just configured. SIEM/SOAR integration brings RAG telemetry into the broader security monitoring stack. Anomaly detection on retrieval patterns can surface early signs of compromised agents, including access outside the normal scope, volumes inconsistent with the initiating user’s entitlements, or deviations from established behavioral baselines.

7. Apply Zero Trust across the RAG workflow

In RAG, Zero Trust is primarily expressed through identity-aware authorization at the retrieval layer, not just through network boundaries. Never trust, always verify, including agent-to-agent calls within the same workflow. Short-lived tokens, continuous authorization, and policy enforcement at every hop are the operational expression of that principle. NIST SP 800-207 provides a foundational reference for Zero Trust.

When RAG goes agentic: what changes for security

The seven controls above apply to any RAG workflow. When RAG goes agentic, introducing multi-step reasoning, tool calling, and chained agent delegation, each step is a new opportunity for authorization gaps to compound. The identity control plane should propagate user context at every hop, including agent-to-agent and agent-to-application calls, not just during the initial retrieval. Standards-based delegation patterns extend OAuth 2.0 Token Exchange across downstream calls, helping ensure that no agent in the chain exceeds the requesting user’s entitlements.

Common RAG security mistakes

Many high-impact RAG security failures are predictable identity failures that compound quietly.

• Treating the agent’s permissions as the user's permissions: The agent may have broader access than the user it serves. When the retriever acts on the agent’s credentials rather than the user’s entitlements, it may return documents the user was never authorized to see.
• Relying on the LLM to enforce access control: The model cannot determine what the user is authorized to access. Moving that decision into the prompt removes it from the identity control plane entirely.
• Over-scoped service accounts for retrievers: A retriever with standing read access to the entire knowledge base returns whatever is semantically relevant, regardless of who asked.
• No audit trail from prompt to retrieval to response: Without attribution at every step, data exposure events cannot be investigated or demonstrated to auditors.
• Static, long-lived credentials for agents: A compromised agent with long-lived credentials can operate undetected for an extended period, expanding the blast radius of any breach.

Frequently asked questions

What is the biggest security risk in RAG?

In enterprise RAG deployments, the biggest risk is over-permissioned retrieval, where an agent returns data that the requesting user is not authorized to see. Many RAG architectures secure the data and the model, but leave the retrieval layer unguarded against legitimate agents acting beyond the requesting user's entitlements.

How is RAG security different from traditional application security?

RAG introduces non-human identities that act autonomously, a retrieval layer that requires real-time authorization decisions, and the need to propagate user context across agent chains. Traditional access control models were not designed for any of those conditions.

What role does identity play in RAG security?

Identity is a foundational control plane that connects all three layers of RAG defense: data, model, and retrieval. It determines who, human or agent, can retrieve what, under what delegated authority, and provides the audit trail to verify it.

What is the authorization gap in RAG?

The authorization gap is the difference between what an AI agent can technically access and what the requesting user is entitled to see. It is most common when an agent's credentials exceed the user's permissions. The standards-based approach combines fine-grained authorization at the retrieval layer with OAuth 2.0 Token Exchange, so every retrieval is evaluated against the requesting user's entitlements rather than the agent's credentials.

How do you apply Zero Trust to a RAG workflow?

Treat every retrieval, tool call, and agent-to-agent interaction as untrusted until verified. Issue short-lived, scoped tokens for each action. Enforce policy at every hop. The practical expression of Zero Trust in RAG is authorization at the retrieval layer, not perimeter defense.

Secure AI agents and RAG at scale with Okta

Closing the authorization gap starts with an identity control plane that connects human identities, AI agents, and retrieval authorization across RAG workflows. The Okta Platform helps organizations govern AI agents as non-human identities, enforce fine-grained authorization at the retrieval layer, propagate user context across agentic workflows, and support a traceable audit trail from the requesting user to the final AI response.

Learn more

These materials are for general informational purposes only and do not constitute legal, privacy, security, compliance, or business advice.

The content may not reflect the most current security, legal and/or privacy developments. You are solely responsible for obtaining advice from your own legal and/or professional advisor and should not rely on these materials.

Okta makes no representations or warranties regarding this content and is not liable for any loss or damages resulting from your implementation of these recommendations. Information on Okta’s contractual assurances to its customers may be found at okta.com/agreements.