The rapid rise of Generative AI has presented a pivotal challenge for every enterprise: how to empower teams to innovate at the speed of this new technology without compromising on security, privacy, and data sovereignty. Many see this as a choice between moving fast and staying safe.
At Okta, we see this as a false choice. We see responsible AI adoption not as a barrier to innovation, but as its essential enabler. To operationalize this conviction, we launched an internal initiative, “Take Okta^AI”, with the ambitious goal of embedding AI into our organizational DNA to save over 250,000 hours of work. But to get there, first we had to solve the same problem every leader is facing today: taming “Shadow AI”, the proliferation of decentralized API keys, inconsistent data handling policies, and the inherent risk of sensitive corporate data inadvertently leaking into public large language models (LLMs).
Our strategic response was the creation of a unified AI gateway. This is the story of how we successfully deployed a secure, model-agnostic AI environment for our employees and the foundational security architecture that underpins it.
A Control Plane for AI Identities
Instead of permitting a “Wild West” of unmanaged integrations, we established a single unified control plane to funnel all external AI usage. This approach treats AI tools not just as applications, but as non-human identities that require governance throughout their entire lifecycle.
Deployment Detail
Our gateway is a resilient, scalable service powered by LiteLLM, deployed within Okta's secure private cloud. It functions as a secure proxy and a single point of entry for every AI interaction, from a developer calling a code-generation API to an employee using our internal webUI. This architecture gives us a central location to enforce security, manage costs, and enable compliance, all while decoupling our internal users from the underlying LLM providers.
Security First: The Four Pillars of Our AI Defense
Our unified AI gateway is built on four non-negotiable security pillars, showcasing how we use our own products to solve some of the industry's toughest challenges.
1. Identity as the Perimeter (Powered by Okta)
We treat AI access as a critical corporate resource. This is Identity as the Perimeter in Action, powered by our own technology. Access is strictly gated by Okta Single Sign-On (SSO), and we enforce phishing-resistant Multi-Factor authentication (MFA) for every user. We also verify that only trusted, managed devices can connect to the gateway. Critically, when an employee’s role changes or they leave the company, their access is instantly and automatically deprovisioned. This identity-centric process reduces the risk of orphaned accounts and stray credentials.
2. Data Privacy & Automated PII Redaction
To mitigate the risk of accidental data leakage, our gateway includes a centralized Personally Identifiable Information (PII) redaction layer. Before any prompt leaves our trusted environment, the system is designed to automatically scan for and scrub PII like names, emails, or phone numbers. This provides a critical safety net that protects our employees and customers, no matter which AI model is being used.
One of the most significant risks in enterprise AI is the leakage of PII. We mitigate this risk proactively and centrally, before any data ever leaves our trusted environment.
3. The "No Training" Guarantee (Zero Data Retention)
Our intellectual property is non-negotiable. Wherever we can, we include zero data retention in our agreements with our model providers and restrict them from using Okta’s data to train models. This allows our teams to use these powerful tools with confidence.
4. Full Observability and Auditability
By centralizing all LLM traffic, we are able to gain 100% visibility into AI usage. Every single interaction meticulously logged and routed to our Defensive Cyber Operations (DCO) team for continuous auditing. This allows us to track usage, monitor for abuse, and reliably attribute every request back to a specific, authenticated user identity, which is essential for maintaining critical certifications like SOC 2 Type 2 and ISO 27001.
Unlocking Innovation with "Model Agility"
By addressing the security challenge centrally, we unlocked a major operational advantage: Model Agility. Because security and compliance are handled at the gateway, we are not locked into a single AI vendor. We can rapidly integrate the latest models and make them available to employees with minimal friction. Engineers write code against a single, standardized API endpoint, and if we decide to switch the backend model, it’s a simple configuration update in the gateway; no code changes, procurement cycles, or repetitive security reviews required.
- For All Employees - They gain access to a Universal Workbench where they can compare and leverage different models side-by-side. For instance, they can use one model for creative writing tasks and a separate model specialized for complex logical reasoning, all within the same secured environment.
- For Engineers - They write code against a single, consistent, and standardized API endpoint. If the business decides to switch the underlying backend model for a specific feature, we simply update a configuration within the gateway. This avoids any need for code changes, new procurement cycles, or repetitive security reviews on the application side.
Conclusion: The Freedom to Innovate, Securely
The Unified AI Gateway has fundamentally transformed how Okta operates. By placing identity at the very center of our strategy, we have successfully moved away from a rigid licensing model to a flexible, usage-based architecture that expands AI access to our entire workforce while strengthening our security posture. The most profound result is the confidence this provides: giving our employees the freedom to experiment, learn, and build with the world's most powerful AI tools, safely and responsibly.
Disclaimer: These materials are for general informational purposes only and do not constitute legal, privacy, security, compliance, or business advice.
The content may not reflect the most current security, legal and/or privacy developments. You are solely responsible for obtaining advice from your own legal and/or professional advisor and should not rely on these materials.
Okta makes no representations or warranties regarding this content and is not liable for any loss or damages resulting from your implementation of these recommendations. Information on Okta’s contractual assurances to its customers may be found at okta.com/agreements.
