TL;DR
Authentication protocols are the standardized methods computer systems use to verify that a user is who they claim to be. Ten common protocols are covered here — from Kerberos and Lightweight Directory Access Protocol (LDAP) to Open Authorization (OAuth) 2.0, Security Assertion Markup Language (SAML), and Remote Authentication Dial-In User Service (RADIUS) — each with distinct strengths and trade-offs. Because no single protocol is completely foolproof, choosing the right one requires weighing your application's sensitivity, your existing infrastructure, implementation effort, and long-term scalability.
How authentication protocols protect your systems
Authentication is the process of confirming that a user is who that person claims to be. An authentication protocol is the method you use to accomplish that task.
Several authentication protocols exist. None are 100 percent foolproof. Choose your method carefully, however, and you will reduce the risk of hacking and data theft.
What are authentication protocols?
An authentication protocol allows the receiving party (such as a server) to verify the identity of another party (such as a person using a mobile device to log in). Almost every single computer system uses some kind of network authentication to verify users.
As more critical information is stored electronically, and as hackers become more and more adept at theft, authentication becomes more important. Without it, losses can be significant. For example, Deloitte experienced a data breach in 2017 that exposed client email (including some tied to government agencies). Authentication may never keep your information perfectly secure. But it can make theft harder to accomplish. Hackers may move to a different target if your servers are too difficult to penetrate.
What are the most common types of authentication protocols?
IT administrators have plenty of options available to them.
| Protocol | Full Name | How It Works / Key Use Case |
|---|---|---|
| Kerberos | Kerberos | Uses symmetric keys from a centralized key distribution center; common in Windows environments. |
| LDAP | Lightweight Directory Access Protocol | Open, vendor-neutral protocol for accessing and maintaining directory data (usernames, passwords, email addresses, etc.). |
| OAuth 2.0 | Open Authorization 2.0 | Allows an application to pull resources on a user's behalf without sharing credentials; enables cross-site logins. |
| RADIUS | Remote Authentication Dial-In User Service | Verifies a username and password by comparing it against a database. |
| SAML | Security Assertion Markup Language | XML-based protocol that exchanges authentication data between identity providers and service providers. |
| CHAP | Challenge-Handshake Authentication Protocol | Periodically reauthenticates users mid-session; each challenge differs from the last. |
| DIAMETER | DIAMETER | Framework for authentication and accounting messages; an improvement upon RADIUS. |
| EAP | Extensible Authentication Protocol | Used in wireless networks and point-to-point connections. |
| PAP | Password Authentication Protocol | User submits a username and password, which the system compares to a database. |
| TACACS | Terminal Access Controller Access-Control System | Enables IP-based authentication; later versions include encryption. |
We'll list a few here, but know there are many more.
The five most common authentication protocols
The five most common authentication methods companies use include the following:
- Kerberos: If you work within the Windows environment, you've used this protocol. The system leans on symmetric keys pulled from a centralized key distribution center. While the protections are significant, Kerberos isn't perfect. In 2020, Kerberos stopped working after a system update.
- Lightweight Directory Access Protocol (LDAP): As we explained in a recent blog post, companies store usernames, passwords, email addresses, printer connections, and other static data within directories. LDAP is an open, vendor-neutral application protocol for accessing and maintaining that data.
- OAuth 2.0: If you've ever used a login from another site (like Facebook) to get into a new site (like The New York Times), you've used OAuth 2.0. An application pulls resources on your behalf, and you don't have to share credentials. This system can also be hacked, as GitHub discovered in 2020.
- Remote authentication dial-in user service (RADIUS): You provide a username and password, and the RADIUS system verifies the information by comparing it to data in a database.
- Security Assertion Markup Language (SAML): This Extensible Markup Language (XML)-based protocol exchanges authentication data between identity providers (IdPs) and service providers.
Five additional authentication protocols to know
These are five other types of authentication protocols to know:
- Challenge-Handshake Authentication Protocol (CHAP): This system reauthenticates users periodically, even within the same session. Each challenge is different from the last version.
- DIAMETER: This protocol provides a framework for authentication and accounting messages. It's derived from RADIUS, and it's considered an improvement upon that protocol.
- Extensible authentication protocol (EAP): Wireless networks and point-to-point connections often lean on EAP.
- Password authentication protocol (PAP): A user submits a username and password, which the system compares to a database.
- Terminal Access Controller Access-Control System (TACACS): Accomplish Internet Protocol (IP)-based authentication via this system. Later versions of this protocol include encryption.
How to choose among authentication protocols
With so many options, how can you choose the version that's right for you?
What key factors should you evaluate before choosing a protocol?
Consider your:
- Application needs. What systems and resources require access? How significant or private are they?
- Infrastructure. What protocols can you launch without overhauling your existing system?
- Effort. How much training or programming will you need before you can get started?
- Future. Can the system grow and change with your company?
Frequently asked questions
Is any authentication protocol completely secure?
No authentication protocol offers perfect security. While strong protocols significantly reduce the risk of unauthorized access, determined attackers can still find vulnerabilities — as demonstrated by real-world breaches affecting Kerberos, OAuth 2.0, and others. The goal is to make unauthorized access difficult enough that attackers move on to easier targets.
What is the difference between RADIUS and DIAMETER?
Both protocols handle authentication and accounting, but DIAMETER was developed as a direct improvement upon RADIUS. It provides a more robust framework for managing authentication messages and is generally considered the more modern of the two options.
When should an organization use SAML instead of OAuth 2.0?
SAML is an XML-based protocol designed to exchange authentication data between an identity provider and a service provider, making it well-suited for enterprise single sign-on scenarios. OAuth 2.0, by contrast, is commonly used when an application needs to access resources on a user's behalf across different platforms — such as using a social media login to access a third-party website.
What factors should guide the choice of an authentication protocol?
Four key considerations are outlined in the article: the sensitivity of the systems and resources that need access, the compatibility of the protocol with your existing infrastructure, the level of training or programming effort required to implement it, and whether the protocol can scale as your organization grows.
What makes Kerberos a common choice in enterprise environments?
Kerberos is widely used in Windows environments because it relies on symmetric keys distributed through a centralized key distribution center, providing strong authentication protections. However, it is not without risk — a 2020 security update caused Kerberos to stop functioning in some Windows systems, highlighting the importance of testing updates before deployment.
What is CHAP, and why does it offer stronger session security than simpler protocols?
The Challenge-Handshake Authentication Protocol (CHAP) goes beyond a one-time login check by periodically reauthenticating users throughout an active session. Because each challenge is unique and differs from the previous one, it is harder for an attacker to replay captured credentials to gain access.
References
When Two-Factor Authentication Fails: Rethinking the Approach to Identity Security. (February 2018). Forbes.
Windows Kerberos Authentication Breaks Due to Security Updates. (November 2020). Bleeping Computer.
Hackers Stole GitHub and GitLab OAuth Tokens From Git Analytics Firm Waydev. (July 2020). ZDNet.
