6.3. The SSL Protocol

Many of the SSL messages and mechanisms deal with crypto export restrictions that no longer exist. Although now not required, these mechanisms continue to be supported in the name of backward compatibility. Because they all involve weakening the encryption in some way, they are of little interest to us, and we will not cover them. [Rescorla 2001] has a nice discussion of these mechanisms for those interested.

We begin by examining the exchange of packets in a typical SSL session. As a transport-layer protocol, SSL depends on a transport-layer protocol to carry its packets. In principle, there is no reason that SSL couldn't run over UDP, but reliability would have to be built into the SSL protocol itself. To avoid these complications, SSL requires a reliable transport protocol to run over. In practice, this means TCP.

In response to the need for a general protocol that protects UDP traffic, Rescorla and Modadugu have proposed the Datagram Transport Layer Security (DTLS) protocol. DTLS provides a retransmission timer for the SSL handshake but otherwise does not attempt to add reliability to the underlying transport. In particular, DTLS is insensitive to lost or reordered data messages. DTLS offers optional replay protection using the same method that IPsec implements in AH (Chapter 11) and ESP (Chapter 12).

In early 2005, the DTLS proposal was still at the Internet Draft stage. The latest draft of the proposal is available from the Transport Layer Security Working Group's Web site at <http://www.ietf.org/html.charters/tls-charter.html>.

As with TCP, we can think of an SSL session as having three stages: connection setup, data transfer, and connection teardown. In the first stage, the encryption, authentication, and compression algorithms are negotiated; the identity of the server and, optionally, the client is verified, and a key exchange takes place.

In the second stage, the client and server exchange application data. These exchanges are encrypted and authenticated to ensure that the data cannot be read by third parties (encryption) and that third parties cannot alter the data without detection (authentication).

When the applications have finished exchanging data, one or both of them send a closure notification as an EOF. Because the closure notification is authenticated, it can't be forged by a third party. This prevents malevolent parties from forging a TCP FIN and truncating the data prematurely.

The SSL 3 and TLS specifications require that both sides send closure notifications, but in practice, this is often ignored, and only one side sends it.

Basic SSL Packet Flow

Figure 6.2 shows a typical SSL session. The first nine messages comprise connection establishment. In this phase, the client sends the server a ClientHello message that specifies which version of SSL it supports and lists the cipher suites and compression algorithms that it is willing to use.

The server responds with three messages.

As we'll see later, the peers generally coalesce the messages in each step of the handshake, so these three messages will be sent as a single TCP segment.

The first, the ServerHello message, informs the client which cipher suite and compression algorithm the server has chosen.

The second message is the server's certificate. The certificate serves two purposes. First, it allows the client to verify the identity of the server. Second, it contains the server's public key, which the client uses to encrypt a secret that will be used by both sides to generate the cryptographic keys needed for the session.

Finally, the server sends a ServerHelloDone message. Because some SSL modes require additional server handshake messages, the ServerHelloDone packet serves to mark the end of the server's hello sequence. At this point, the server has positively identified itself to the client, the client and server have agreed on cipher suites and compression algorithms, and the client has the necessary key to securely send the server some key-generating material.

In the next three messages, the client sends the server some key-generating material (ClientKeyExchange), tells the server that it will henceforth use the newly generated keys to encrypt and authenticate its messages (ChangeCipherSpec), and informs the server that it has completed its part of the handshake (Finished). The server responds with its own ChangeCipherSpec and Finished messages.

Now the applications are ready for the data exchange phase. The data transfer looks like any other TCP data transfer except that the data in the TCP segments is encrypted and authenticated. When we watch this transfer with tcpdump, we'll see that it is indistinguishable from any other TCP data transfer except that the application data appears to be random bits.

The last messages are the closure notifications. At this point, data can no longer be transmitted, and the connection is shut down.

The SSL Record Layer

Before taking a detailed look at SSL in action, we need to understand the SSL record layer. All SSL messages are carried in SSL records that identify what type of messages are in the records, the length of the messages, and the version of SSL being used. Figure 6.3 shows the general format of these records.

Figure 6.3. The SSL Record Format

[View full size image]

The 8-bit type field identifies the type of message contained in this record. Figure 6.4 lists the four message types.

Figure 6.4. SSL Record-Layer Messages Types

Record Type	Value	Description
CHANGE_CIPHER_SPEC	20	switch to the last negotiated cipher suite
ALERT	21	error or CloseNotify messages
HANDSHAKE	22	hello and other connection-initiation messages
APPLICATION_DATA	23	data messages

The major and minor version numbers indicate what version of SSL is being used. SSL 3 has a major version of 3 and a minor version of 0. TLS has a major version of 3 and a minor version of 1.

The length field gives the length in bytes of the data field. The length field does not include the HMAC, padding, or pad length fields.

The data field carries the message data. This field can be empty, as in the case of a ChangeCipherSpec message, or it can carry handshake data, alert messages, or application data.

HMAC is a cryptographically secure message authentication code. This field is used to provide authentication. If any of the previous fields are altered, the receiver will calculate a different HMAC and detect the alteration. Because one of the keys generated during the key exchange is used as an input to the HMAC calculation, an attacker cannot adjust the HMAC in the message to escape detection. As we mentioned earlier, the input to the HMAC includes a sequence number that is used to prevent replay attacks.

Figure 6.3 depicts the HMAC as being a 160-bit SHA1 digest. It is also possible to use an MD5 128-bit digest.

The SSL 3 HMAC is slightly different from that described in RFC 2104 [Krawczyk, Bellare, and Canetti 1997], being based on an earlier draft of that RFC. TLS uses the standard HMAC as defined in RFC 2104.

The padding and pad length fields are used to pad the message to the block size of the encryption algorithm when a block encryption cipher, such as 3DES, is used. When a stream cipher, such as RC4, is used, these fields are not present.

During the first part of the handshake, before the ChangeCipherSpec message, the NULL cipher suite is used, in which case there is no HMAC or padding. This is because the keys are not yet available to generate the HMACs or encrypt the message.