10.2. Protocols

As we saw in Chapter 9, we can think of IPsec as consisting of three protocols: AH, ESP, and IKE. IKE is the key-management protocol that negotiates VPN parameters between endpoints. We defer discussion of it until Chapter 13.

The other two protocols are the ones that provide security services to the VPN by encapsulating the IP datagrams that flow through it. We will see the exact encapsulations when we study AH in Chapter 11 and ESP in Chapter 12.

The AH protocol provides datagram-origin authentication, data integrity, and protection from replay attacks. The authentication and integrity services are provided by calculating a cryptographic MAC over a datagram's payload and parts of the datagram's IP header. Because the source and destination fields of the IP header are included in the MAC, we can be sure that those fields were not tampered with while the datagram was in transit. Because the payload is also included in the MAC, we can be sure that it wasn't tampered with either. Figure 10.1 shows an overview of AH encapsulation.

Figure 10.1. AH Encapsulation Overview

[View full size image]

Notice that the authentication covers only part of the IP header.

ESP also has an authentication function, which is almost identical to that provided by AH. The difference is that ESP authentication does not protect the IP header fields.

More precisely, ESP does not protect the outer IP header fields. As we'll see shortly, AH and ESP operate in two modes, one of which adds an additional IP header. In this mode, ESP protects the inner IP header but not the outer. AH, on the other hand, protects all the inner IP header and parts of the outer IP header.

It's a matter of some contention as to whether there's a compelling reason to have AH at all. Those who take the negative position (see [Ferguson and Schneier 1999], for example), point out that the outermost IP header is used only for routing and that possession of the authentication key proves that it's from the appropriate sender. Furthermore, an attacker who has compromised the authentication key can change the outer IP header in any way desired.

Those who take the affirmative position argue that some specialized applications require the additional securityRFC 2461 [Narten, Nordmark, and Simpson 1998], for example, specifies the use of AH for neighbor discovery messages when security is requiredand that in any event, some sites are using it. Those taking the affirmative position appear to be winning the argument, because the latest draft revision to RFC 2401 still includes AH.

Both AH and ESP also protect against replay attacks by including a sequence number in their protocol headers. The use of sequence numbers with an unreliable datagram delivery service, such as IP, is difficult because datagrams can be lost or delivered out of order. In Chapter 11, we see how IPsec solves this problem.

ESP also provides data confidentiality by encrypting parts of the datagram, including the payload. Figure 10.2 shows an overview of the ESP encapsulation.

Figure 10.2. ESP Encapsulation Overview

[View full size image]

Exactly what is encrypted besides the payload depends on the mode that ESP is using. We study this in Chapter 12.

Note that ESP can do everything that AH can, except authenticate the IP header fields, and provide confidentiality as well. For this reason, most real-world VPNs run ESP only and use it to provide both authentication and privacy. Of course, it's easy to imagine situations in which a combination of AH and ESP might make sense, and we examine some of those possibilities later in the chapter.