Transparency and customer success are core values at Okta. Regrettably, this morning many Okta customers hosted in the US infrastructure experienced intermittent authentication and API errors starting at 4:30AM PT due to a distributed denial-of-service (DDoS) against the Domain Name Service (DNS) provider Dyn. We took steps to mitigate the issues impacting the core AWS services immediately; while investigating, it became clear that we needed to switch to an alternative DNS provider. By 6:18AM PT, we switched to our secondary DNS provider. By 7:10AM PT, the overall rate of failures dropped off significantly. By 8:30AM PT service was fully restored, however because of DNS propagation issues, we continue to work with customers to troubleshoot any residual disruptions.
While only a percentage of our traffic was impacted, we understand any impact is a big impact. Ultimately, our architectural and vendor choices are our responsibility, and so is the effect on customers. We’ve had all hands on deck resolving the issues and restoring service to impacted customers, and we’re committed to ensuring that similar issues do not arise again. We’re designing infrastructure changes that will allow us to mitigate these issues faster in the future.
For detailed updates, please visit trust.okta.com. Okta customers having DNS issues, can refer to remediation steps here. If you are having issues, please open a case with us and our support team will be in touch.
UPDATE 10/25: Review the full Root Cause Analysis.