The Future of AI Security

In September 2025, Anthropic detected and disrupted the first documented large-scale cyberattack executed predominantly by an AI agent. A Chinese state-sponsored group (GTG-1002) manipulated Claude Code to target approximately 30 organizations across financial services, technology, manufacturing, and government. (Anthropic report) The AI autonomously executed 80-90% of tactical operations: reconnaissance, vulnerability discovery, exploitation, credential harvesting, lateral movement, and data exfiltration. Humans intervened only at critical strategic junctures, spending roughly 20 minutes of hands-on direction per phase. A handful of targets were successfully breached.

This was not a proof of concept. It was an operational campaign that exploited the same architectural gaps most enterprises have today: no model-layer screening caught the manipulated prompts, no agent identity framework detected unauthorized autonomous operation, and no agent relay enforced tool-level controls at runtime. The OWASP Top 10 for Agentic Applications, released December 2025 with input from over 100 industry experts, codifies these exact failure modes: Agent Goal Hijack (ASI01), Tool Misuse (ASI02), and Identity and Privilege Abuse (ASI03) as the top three risks.

This is not an isolated event. 88% of organizations reported confirmed or suspected AI agent security incidents in the past year (Gravitee survey), and among organizations that experienced AI-related breaches, 97% lacked proper access controls for their AI systems (IBM/Ponemon). Only 22% treat AI agents as independent, identity-bearing entities (Gravitee).

Put differently: if you have agents in production and haven't had an incident, you likely just haven't found it yet.

The Right Architecture

Enterprises are deploying agents across three distinct trust boundaries. Internal agents (IT helpdesk bots, HR agents, finance bots pulling ERP reports) operate within the enterprise perimeter. Customer-facing agents (support assistants, self-service AI querying internal knowledge bases on behalf of end users) bridge internal resources and external consumers. Partner agents (supply chain integrations, cross-organization service delivery) operate across organizational boundaries entirely. The security requirements escalate with each pattern. Internal agents need identity and lifecycle governance. Add a customer-facing agent and you need delegated authorization and consent. Extend to partners and you are managing cross-domain trust and data flow controls. All three require the same foundational architecture.

AWS, Google, Anthropic, and Microsoft have each shipped agent security controls within their own platforms that map to these same three layers. That validates the architecture. But enterprises don't run agents on a single platform. They need identity and authorization that works across all of them. Gartner predicts 25% of enterprise breaches will be traced to AI agent abuse by 2028. McKinsey calls AI agents "digital insiders" requiring the same governance as employees. The architecture is converging. The question is whether your organization will adopt it proactively or reactively.

Layer 1: Model Security, Secure the Intelligence

In the September 2025 campaign, attackers bypassed safety guardrails through role-playing deception and task decomposition, breaking multi-stage attacks into discrete requests that appeared legitimate in isolation (Anthropic). No human is reviewing thousands of prompts per minute. Model-layer security is what stands between a manipulated prompt and a compromised network.

Model evaluation and adversarial red-teaming surface vulnerabilities before production. Prompt injection defense catches malicious instructions hiding in documents, emails, or tool responses, while output guardrails and DLP keep sensitive data from leaking out. Continuous I/O monitoring ties it together, flagging anomalous behavior in real time. These are not theoretical risks. Researchers have found real MCP vulnerabilities in the wild: tool poisoning through malicious MCP servers, prompt injection via tool responses, and supply chain attacks on MCP packages (OWASP). AWS Bedrock Guardrails and Google Cloud Model Armor both provide configurable, model-agnostic screening at runtime. If the model is compromised, nothing else you build on top of it matters.

Layer 2: Agent Identity, Secure the Actor

Once the model is trustworthy, the agent needs an identity. Not a shared service account. Not a static API key. A real identity, with an owner who answers when something goes wrong, scoped permissions that expire, and a lifecycle that ends when it should.

Before AI, it was all code. You knew deterministically what would happen. But now with AI agents, you give it a prompt... there is no guarantee which systems it is going to access in response. You do not know if it is going to use a higher level of access to go to a system that you did not even anticipate. 

Harish Peri, SVP & GM, Okta

Agent Identity starts with discovery: how many agents operate, who built them, what credentials do they hold? Okta's AI at Work survey found 91% of organizations already deploying agents, yet only 10% have an effective governance strategy. In every customer workshop we run, the first question is the same: how many agents do we actually have? The answer is almost always "we don't know." Every agent gets a durable identity, an assigned owner, and a classification. Authorization follows delegation: the agent acts on behalf of a specific user, with a specific set of permissions, for a specific task. Okta's Cross App Access (XAA), an open protocol extending OAuth now adopted into MCP as Enterprise-Managed Authorization, makes these delegation chains auditable. When the chain breaks, access stops. Lifecycle management through SCIM provides the same governance as human identities: access reviews, automated deprovisioning, owner accountability. When the delegating user is offboarded or a policy changes, revocation propagates across every downstream system.

But identity alone does not close the gap. Knowing an agent is authorized is not the same as knowing whether it should transfer $50,000 from this account, right now, at 2 AM, to a recipient it has never interacted with before.

Layer 3: Data Authorization, Secure the Action

OAuth policies check the token. Valid scopes, valid audience, not expired. That is necessary but nowhere near sufficient. The token says the agent can make transfers. It says nothing about whether this transfer, to this recipient, at this amount, makes any sense.

Identity and perimeters only provide coarse controls around agent actions. Tool-use guardrails give more power to finely control which actions to allow. 

Google Cloud, Agent Development Kit Safety Documentation

Agent behavior is not deterministic. The LLM picks tools at runtime, fills in parameters based on whatever reasoning it has done, and chains operations in sequences that nobody anticipated when the policy was written. This is where an agent relay sits: the enforcement point between the agent and the tools it invokes. Okta's Agent Relay enforces fine-grained runtime controls: tool-level access (which tools can this agent invoke?), parameter constraints (is the amount within limits?), human-in-the-loop approval via CIBA (Client-Initiated Backchannel Authentication) for high-risk operations, cross-app data flow governance (is Tool A's output authorized as Tool B's input?), and a complete invocation audit trail.

The Agent Relay does not replace identity. It picks up where identity leaves off. Identity decides what goes into the token. The Agent Relay decides what happens at the moment of execution, using what is in that token. The claims carry context from one layer to the next. No gap between who the agent is and what it is allowed to do.

What Breaks Without Each Layer

A Kiteworks study of 225 security leaders found that 100% have agentic AI on their roadmap, yet 63% cannot enforce purpose limitations and 60% cannot terminate a misbehaving agent. The controls that are missing are the controls that matter most.

Where the Architecture Is Heading

The three-layer architecture closes the structural gaps that most enterprises have today. Three areas are rapidly maturing and will define the next phase:

• Recursive delegation: Agent A delegates to Agent B, which delegates to Agent C. How far does authority propagate? Who revokes at depth? Okta’s Cross App Access (XAA) with ID-JAG maintains the chain of custody by embedding user identity and agent identity into every token exchange, with scope narrowing at each delegation hop. SCIM lifecycle management helps ensure that when a delegating user is offboarded, revocation propagates. The remaining frontier is standardizing delegation depth limits and multi-hop revocation signaling, work that the OAuth Identity and Authorization Chaining specification is actively addressing.
• Authorization drift: An agent is authorized at invocation time but the user’s permissions change mid-chain. How does revocation reach an operation already in flight? Short-lived tokens with aggressive expiry windows limit the blast radius. CIBA-based step-up authentication can re-verify authorization at high-risk decision points. Okta’s Access Policies enforce scope grants per group membership, so a permission change at the directory level takes effect at the next token exchange. The frontier is real-time revocation signaling across active execution chains without breaking legitimate operations.
• Cross-domain trust boundaries: An agent chains tools across two SaaS applications with different trust models and data classification schemes. What governs the boundary? Within a single trust domain, Okta’s XAA and per-resource Custom Authorization Servers already enforce scoped access at each boundary crossing. Auth0 Token Vault extends this to third-party OAuth resources (Google, GitHub, Salesforce) through brokered consent and token translation. The cross-organizational boundary challenge, where two independent IdPs must establish mutual trust for agent delegation, is the next frontier. The OAuth Identity Chaining specification provides the architectural foundation, and Okta is actively contributing to its development.

These are hard problems with clear architectural paths forward. The OWASP Agentic Security Initiative and the OpenID Foundation are both developing frameworks. Organizations that establish the three-layer foundation now will be positioned to adopt these controls as they mature.

Where Are You Today?

Score your organization against each layer. Most enterprises are Reactive in at least two.

Okta and Auth0 provide the identity and authorization infrastructure to move from Reactive to Adaptive across Layers 2 and 3. As the only independent identity platform purpose-built for both workforce and customer identity, Okta treats agent identity as a first-class citizen with the same governance as human identity, built on open standards like XAA and SCIM rather than proprietary lock-in.

If you scored yourself as Managed across all three layers, you are probably being generous.

The 90-Day Path

Conclusion

The September 2025 attack proved that agentic AI threats are operational. The industry data confirms most enterprises are not ready. The three-layer architecture is not new thinking. It is what the largest platform companies in the world have independently built, what the standards bodies are codifying, and what the incident data demands. The only question left is speed.

Context is the new credential. Intent is the new perimeter. The future of AI security is built by connecting model security, agent identity, and data authorization into a single chain of trust.

The technology to solve this exists today. Learn how Okta and Auth0 secure AI agents at okta.com/ai and auth0.com/ai. We can help you build this. Collaborate with us.

Explore more AI Security insights:

• AI Agent Security: Building Autonomous Trust at Machine Speed. A Seven-Part Thought Leadership Series by Okta. 
• AI agents force a fundamental choice. That’s a false choice. Identity is the building block to refuse it. Agents Can Either Be Useful, or Secure.