Introduction to TLS v1.2

Some notes about how TLS v1.2 (Transport Layer Security) works. The goal explain what is going on in a network traffic dump, the role of the different TLS extensions, the impact of the different cipher suites on security, etc. It includes several diagrams and many references.

The post starts with a summarizing sequence diagram. The next section describes in details an example of a typical TLS connection. Then, some more details are provided. The next section explains how session resumption works. The following section describes some less used features: mutual authentication, alternative certificate formats and TLS-PSK.

You may want to check as well “The Illustrated TLS Connection” for examples of the mesages on the wire.

Update 2023-05-25: Fix error about protocol stack diagrams where “EAP-TLS” was used instead of “EAP-TTLS”.

Update 2020-02-01: added some notes about DNS rebinding for HTTPS in appendix.

Table of content

Overview

TLS works on top of a reliable transport (usually TCP) and provides:

• a stream-oriented transport;
• privacy/confidentiality (using encryption);
• data integrity (protection against tampering);
• server authentication (usually);
• client authentication (optionally);
• negotiation of the cipher suite, protocol version, etc.;
• protection of the handshake against downgrade attacks.

[ HTTP/1.x ] [ HTTP/2 ] [ IMAP ] [ SMTP ] [ DNS ] [ LDAP ]
[ TLS      ] [ TLS    ] [ TLS  ] [ TLS  ] [ TLS ] [ TLS  ]
[ TCP      ] [ TCP    ] [ TCP  ] [ TCP  ] [ TCP ] [ TCP  ]
[ IP       ] [ IP     ] [ IP   ] [ IP   ] [ IP  ] [ IP   ]
  HTTPS        HTTPS     IMAPS    SMTPS    DoT     LDAPS    etc.

The major phases of the TLS connections are:

1. The client and the server negotiates which version of the TLS protocol to use, which encryption scheme to use, which key exchange algorithm to use, etc. This is done is the ClientHello and ServerHello messages.
2. The client and the server proceed with the negotiated key exchange algorithm, typically a Diffie-Hellman (DH) key exchange. The purpose of the key exchange is to establish some shared secret (the master secret) between the client and the server without revealing this secret to eavesdroppers. The server and optionally the client are authenticated as part of the key exchange algorithm.
3. The master secret is used to derive some key material (for encryption and authentication).
4. Then the client and the server confirm to each other that they have computed the same master secret and observed the same TLS handshake. The latter should prevent an attacker from successfully tampering with the TLS handshake (eg. forcing the usage of weaker options during the negotiation). This is done in the ChangeCipherSpec and Finished messages.
5. At this point, the TLS handshake is over. The client and the server can exchange encrypted and authenticated application protocol data.

[ Handshake | ChangecipherSpec | Alert | Application (eg. HTTP) ] (TLS subprotocols)
[ TLS Record Protocol: fragmentation                            ] (multiplexing and framing)
[ TLS Record Protocol: compression                              ] (if negotiated)
[ TLS Record Protocol: record protection                        ] (encryption and message auth)
[ Transport layer (eg. TCP)                                     ]

Summary

The following sequence diagram summarized a huge part of TLS v1.2. This might be somewhat overwhelming for now. The next sections should explain it all. Some bits (such as session resumption and TLS-PSK) are omitted for now but are discussed in following sections. A simplified version of this diagram presenting the typical case is presented in the next section: you might want to skip there at first reading.

Example

This section describes in details a typical (real) TLS connection. This is what used to happens when connecting to this web site using Firefox 78.12.0esr^[1]. This example, uses the ECDHE_RSA key exchange algorithm:

• an ephemeral Diffie-Hellman key exchange is used to generate key material (such as encryption keys);
• the server authenticates itself by including a RSA (Rivest–Shamir–Adlema) signature in the key exchange.

The differents steps are explained in more details afterwards.

Negotiation

The client and the server starts by exchanging hello messages (ClientHello and ServerHello). These messages are used for negotiating:

• the TLS version (in this case TLS v1.2);
• the cipher suite;
• the compression (if any) method^[2];
• the usage of extensions.

These messages include extensions which can be used to:

• negotiate or announce support for other features such as the supported signature methods, application layer protocol to be used, etc.;
• exchange other important information necessary for the handshake (eg. server name).

The client and the server exchange random values (nonces) as well in the hello messages which are used for deriving key material later on. Exchanging nonces ensures that each participant contributes some ephemeral value to the exchange (even if, for example, they are using a static Diffie-Hellman key pair): this can be used to prevent replay attacks.

struct {
    ProtocolVersion client_version;
    Random random;
    SessionID session_id;
    CipherSuite cipher_suites<2..2^16-2>;
    CompressionMethod compression_methods<1..2^8-1>;
    select (extensions_present) {
        case false:
            struct {};
        case true:
            Extension extensions<0..2^16-1>;
    };
} ClientHello;

struct {
    ProtocolVersion server_version;
    Random random;
    SessionID session_id;
    CipherSuite cipher_suite;
    CompressionMethod compression_method;
    select (extensions_present) {
        case false:
            struct {};
        case true:
            Extension extensions<0..2^16-1>;
    };
} ServerHello;

CipherSuite

The cipher suite used is negotiated in the TLS hello messages. In TLS v1.2, a cipher suite is roughly a combination of:

• a key exchange method (how to bootstrap the security and authenticate the server);
• a secret-key encryption method;
• a cryptographic hash algorithm.

Ciphersuites are named using a pattern of the form TLS_{key_exchange}_WITH_{encryption}_{hash}.

Extensions

Both ClientHello and ServerHello messages can include TLS extensions. However, the server can only include extensions which have been sent by the client: some extensions are only sent by the client in order to advertise that it has support for them.

Some important extensions are explained below.

enum {
    signature_algorithms(13), (65535)
} ExtensionType;

struct {
    ExtensionType extension_type;
    opaque extension_data<0..2^16-1>;
} Extension;

SNI extension

The client can use the Server Name Indication (SNI) extension to announce the server name it intends to connect to. For example, the server can use this information to select which certificate chain to send, whether to propose mutual authentication, etc.

struct {
    NameType name_type;
    select (name_type) {
        case host_name: HostName;
    } name;
} ServerName;

enum {
    host_name(0), (255)
} NameType;

opaque HostName<1..2^16-1>;

struct {
    ServerName server_name_list<1..2^16-1>
} ServerNameList;

ALPN extension

The peers can use the Application-Layer Protocol Negotiation (ALPN) to negotiate which application protocol will be tansported by the TLS connection^[5]. In this example, the client announces support for HTTP/2 (h2) and HTTP/1.1 (http/1.1) and the server chooses to use HTTP/1.1.

opaque ProtocolName<1..2^8-1>;

struct {
    ProtocolName protocol_name_list<2..2^16-1>
} ProtocolNameList;

Signature algorithms extension

The client may use the signature algorithms extension to indicate which signature methods are supported by the client:

• in certificate signatures;
• when signing ephemeral Diffie-Hellman public keys.

enum {
    none(0), md5(1), sha1(2), sha224(3), sha256(4), sha384(5),
    sha512(6), (255)
} HashAlgorithm;

enum { anonymous(0), rsa(1), dsa(2), ecdsa(3), (255) }
SignatureAlgorithm;

struct {
    HashAlgorithm hash;
    SignatureAlgorithm signature;
} SignatureAndHashAlgorithm;

SignatureAndHashAlgorithm supported_signature_algorithms<2..2^16-2>;

Supported groups

The supported_groups extension (formerly elliptic_curves) can be used by the client to advertise which Diffie-Hellman groups it handles:

• values like ffdhe2048 for finite field Diffie-Hellman (FFDH, eg. DHE_* key exchange algorithms);
• values like x25519, secpp256r1, brainpoolP256r1 for Elliptic-Curve Diffie-Hellman (ECDH, eg. ECDHE_* key exchange algorithms).

This extension is not sent by the server: the server advertises the chosen group in the ServerKeyExchange message.

Server certificate chain

At this point, the server sends a certificate chain in the Certificate message. This certificate chain is used to associate a static public key to its identity (its domain name)^[6]. This public key will then be used in the following steps to authenticate the TLS handshake.

By default and in most cases, X.509 certificates are used. The usage of other types of certificates can be negotiated using the client_certificate_type and server_certificate_type TLS extensions.

opaque ASN.1Cert<1..2^24-1>;

struct {
    ASN.1Cert certificate_list<0..2^24-1>;
} Certificate;

At this point, the client validates the certificate chain (see appendix). If the validation succeeds, the client can trust that the public key found in the server certificate actually belongs to the server and can be relied on for authenticating the server.

The type of leaf certificate (type of public key, key usage) usable depends on which key exchange algorithm is used which type of key and which certificate may be used. For example, for ECDHE_RSA, the certificate must contain a RSA public key and the key usage extension (if present) must include digitalSignature.

Key exchange

At this point, the client and the server proceed with the negotiated key exchange. Several key exchange algorithms are available in TLS v1.2. In any case, the role of this step is to:

• establish a shared secret (the pre master secret) only known by the client and server, which is going to be used to derive encryption and MAC (message authentication code) keys;
• and (for most key exchange algorithms) authenticate the server.

In our example, the ECDHE_RSA key exchange algorithm is used. This is an ephemeral Elliptic Curve Diffie-Hellamn key exchange (ECDHE) with RSA signature for server authentication:

1. the server generates a new (ephemeral) ECDH key pair (different for each TLS session)^[7];
2. the server signs this public key (alongside with the random nonces) using its RSA private key in order to authenticate the key exchange;
3. the server sends the ephemeral key and the signature to the client (ServerKeyExchange message);
4. the client can now check the digital signature by using the server public key in order to ensure that the ECDH public key is sent by the server;
5. the client generates an ephemeral ECDH key pair as well and sends the ephemeral public key to the server in the ClientKeyExchange message.

struct {
    select (KeyExchangeAlgorithm) {
        case ecdhe_rsa:
            ServerECDHParams    params;
            digitally-signed struct {
                opaque client_random[32];
                opaque server_random[32];
                ServerDHParams params;
            } signed_params;
    };
} ServerKeyExchange;

struct {
    select (KeyExchangeAlgorithm) {
        case ecdhe_rsa:
            ClientECDiffieHellmanPublic;
    } exchange_keys;
} ClientKeyExchange;

struct {
    ECParameters    curve_params;
    ECPoint         public;
} ServerECDHParams;

struct {
    ECPoint ecdh_Yc;
} ClientECDiffieHellmanPublic;

At this point, both the client and the server (see the details of the key derivation in appendix):

1. compute a (the same) Diffie-Hellman shared secret (the pre master secret) using its own Diffie-Hellman private key and the other peer Diffie-Hellman public key;
2. derive a shared master secret from this pre master secret;
3. derive key material (such as encryption keys) from the master secret.

A passive attacker only has access to the Diffie-Hellman public keys. Therefore, it cannot compute the pre master secret, the master secret and the key material. See the previous episode for explanations about the DH key exchange.

An active attacker cannot contribute an ephemeral public key on behalf of the server because of the digital signature.

Vulnerability: Logjam attack

This weakness in the TLS v1.2 protocol was exploited in the Logjam attack. In this attack, the client is negotiating a DHE key exchange but a man-in-the-middle (MITM) negotiates a DHE export^[8] key exchange with the server instead:

1. the server generates a ServerKeyExchange which contains a DH public key on a weak (export) DH group;
2. the attacker forwards this message to the client;
3. as the signature in the messages does not cover the cipher suite, the signature is acceptable by the client and the client may continue the handshake using the weak DH group.

The Finished messages are supposed to prevent this type of manipulation by ensuring that both the client and the server have the same view of the handshake. However, if the attacker is able to break either of the DH public keys on this weak DH group fast enough, he can compute the premaster secret in time in order to spoof the Finished message.

The client can protect against this attack by checking that the DH group chosen by the server is big enough.

Enabling encryption

At this point, the client peer sends:

1. a ChangeCipherSpec message which indicates that the negotiated encryption method (in our case AES_128_GCM) will be used for all of its subsequence messages;
2. a Finished message which confirms that both peers have the same master secret (key confirmation) and observed the same the same handshake (protection against downgrade attacks).

Then the server sends a ChangeCipherSpec message and a Finished message as well.

struct {
    enum { change_cipher_spec(1), (255) } type;
} ChangeCipherSpec;

struct {
    opaque verify_data[verify_data_length];
} Finished;

In our case, we are using AES_128_GCM for encryption. Two encryption secret keys (one for each direction) are derived from the master secret. In addition, two initialization vectors are derived from the master secret as well (one for each direction). As this is an authenticated encryption with associated data (AEAD) algorithm, it already protects against tampering: there is not need for a separate MAC and there is no need to derived MAC keys from the master secret.

The Finished message includes a verify_data field which contains a a MAC^[9] tag of all the previous handshake messages using the master secret:

verify_data
         PRF(master_secret, finished_label, Hash(handshake_messages))
            [0..verify_data_length-1];

The other peer verifies this value in order to ensure that the handshake has not been tampered with (eg. downgrading the TLS version, forcing usage of weaker cipher suites, disabling TLS extensions, etc.).

Application Data

The client and the server can now exchange secured (encrypted and protected) messages over the TLS channel.

When one side does not need to send data on the TLS connection anymore, it sends a Alert close_notify messsage to the other side with the warning level to signal end-of-data.

Details

Key exchange methods

The following key exchange algorithms support server authentication using public-key cryptography:

• RSA (RSA transport), the pre master secret is randomly generated by the client and sent encrypted using the server RSA public key^[10];
• DH_DSS/DH_RSA, Finite Field Diffie-Hellman (FFDH) key exchange with static server key;
• ECDH_ECDSA/ECDH_RSA, Elliptic Curve Diffie-Hellman (ECDH) key exchange with static server key;
• DHE_DSS,FFDH key exchange with ephemeral server key authenticated using a DSA digital signature;
• DHE_RSA, FFDH key exchange with ephemeral server key authenticated using a RSA digital signature;
• ECDHE_ECDSA, ECDH key exchange with ephemeral server DH key authenticated using ECDSA digital signature;
• ECDHE_RSA, ECDH key exchange with ephemeral server DH key authenticated using RSA digital signature.

The following key exchange algorithms are anonymous (unauthenticated ephemeral Diffie-Hellman exchange). They do not authenticate the server:

• DH_anon, anonymous FFDH key exchange;
• ECDH_anon, anonymous ECDH key exchange.

Other key exchange algorithms are used for providing mutual authentication (i.e., both the server and the client are authenticated) using a shared-secret (instead of public-key cryptography or in addition of it):

• PSK, RSA_PSK, DHE_PSK, ECDHE_PSK for TLS-PSK (see appendix);
• SRP for TLS-SRP;
• ECCPWD for TLS-PWD.

struct {
    select (KeyExchangeAlgorithm) {
        case dh_anon:
            ServerDHParams params;
        case dhe_dss:
        case dhe_rsa:
            ServerDHParams params;
            digitally-signed struct {
                opaque client_random[32];
                opaque server_random[32];
                ServerDHParams params;
            } signed_params;
        case rsa:
        case dh_dss:
        case dh_rsa:
        case ecdh_dss:
        case ecdh_rsa:
            struct {} ;
            /* message is omitted for rsa, dh_dss, and dh_rsa */
        case ecdhe_dss:
        case ecdhe_rsa:
        case ecdh_anon:
            ServerECDHParams    params;
            digitally-signed struct {
                opaque client_random[32];
                opaque server_random[32];
                ServerDHParams params;
            } signed_params;
    };
} ServerKeyExchange;

struct {
    select (KeyExchangeAlgorithm) {
        case rsa:
            EncryptedPreMasterSecret;
        case dhe_dss:
        case dhe_rsa:
        case dh_dss:
        case dh_rsa:
        case dh_anon:
            ClientDiffieHellmanPublic;
        case ecdh_dss:
        case ecdh_rsa:
        case ecdhe_dss:
        case ecdhe_rsa:
        case ecdh_anon:
            ClientECDiffieHellmanPublic;
    } exchange_keys;
} ClientKeyExchange;

struct {
    public-key-encrypted PreMasterSecret pre_master_secret;
} EncryptedPreMasterSecret;

struct {
    opaque dh_p<1..2^16-1>;
    opaque dh_g<1..2^16-1>;
    opaque dh_Ys<1..2^16-1>;
} ServerDHParams;     /* Ephemeral DH parameters */

struct {
    select (PublicValueEncoding) {
        case implicit: struct { };
        case explicit: opaque dh_Yc<1..2^16-1>;
    } dh_public;
} ClientDiffieHellmanPublic;

struct {
    ECParameters    curve_params;
    ECPoint         public;
} ServerECDHParams;

struct {
    ECPoint ecdh_Yc;
} ClientECDiffieHellmanPublic;

Payload protection

Message authentication is always used in addition to encryption. This prevents a man-in-the-middle from tampering with the protected data.

Some encryption schemes use authenticated encryption with additional data (AEAD): they already protect against tampering. This is the case for example for AES_128_GCM, AES_128_GCM and CHACHA20_POLY1305.

Other encryption schemes (eg. AES_256_CBC) do not protect against tampering. In this case, an HMAC is included with each encrypted TLS records in order to provide message authentication: TLS uses MAC-then-encrypt by default for this but the encrypt_then_mac extension can be used to negotiate the usage of encrypt-then-MAC instead. Encrypt-then-MAC is considered to be more robust (see “The Order of Encryption and Authentication for Protecting Communications” and “The Cryptographic Doom Principle”).

In all cases, the payload protection covers some additional authenticated data (AAD) in addition to the payload:

additional_data = seq_num + TLSCompressed.type +
                        TLSCompressed.version + TLSCompressed.length;

In particular, the sequence number (seq_num) is incremented for each TLS record: this prevents an attacker from replaying a TLS record.

Alerts

If some error happens at any time (eg. in the TLS handshake), the peer sends an Alert message with some other alert code.

enum { warning(1), fatal(2), (255) } AlertLevel;

enum {
    close_notify(0),
    unexpected_message(10),
    ...
} AlertDescription;

struct {
    AlertLevel level;
    AlertDescription description;
} Alert;

The close_notify alert is sent by one peer when it does no have any mesasge to send on the TLS connection.

Session resumption

In order to speed up/optimize the connection establishment, TLS supports the resumption of an existing TLS session.

This can have two benefits:

• it reduces the number of round trips necessary to establish the TLS connection;
• it reduces the computation overhead of the TLS handshake (signatures, DH key echange, etc.).

Two different mechanisms may be used for session resumption in TLS v1.2:

• classic session resumption based on session identifiers;
• stateless (from the point of view of the server) session resumption based on session tickets.

In both cases, the session state (cipher, MAC, master secret, etc.) established in a previous TLS connection are reused for the new TLS connection. As the random nonces are different in the new connection, the generated key material is different as well.

Session identifiers

If the server supports classic session resumption,

1. it returns a session identifier in the ServerHello message while initializing a full handshake;
2. it stores the session state (cipher, MAC, master secret, etc.) associated with the session ID in a session cache;
3. the client may choose to include the session state in a cache of its own.

opaque SessionID<0..32>;

Later, the client may try to resume the session in order to avoid the overhead associated with a full handshake:

1. the client includes the session identifier in the ClientHello;
2. if the server accepts the session, it sends back the session identifier in the ServerHello (otherwise, it uses a fresh session identifier);
3. the client and server reuse the negotiated chosen session state;
4. they derive new key material from the master secret and the new random nonces.

Session ticket

A downside of the classic session resumption mechanism is that it requires a server-side state: the server needs to store the state of each sessions in some cache. The session ticket extension can be used to avoid the need for server-side cache. It is used instead of session identifiers.

When the session_ticket, is used the server may send an opaque session ticket in the NewSessionTicket message as the result of a TLS handshake. The client keeps this ticket and associates it with the master secret. This ticket can then sent by the client in a in a new TLS connection (in the session_ticket extensions of the ClientHello) in order to resume the connection.

struct {
    uint32 ticket_lifetime_hint;
    opaque ticket<0..2^16-1>;
} NewSessionTicket;

The ticket is design to contain the session state (cipher, MAC, master secret, etc.). Therefore, it needs to be be much larger than the session identifier. The session state are encrypted^[11] and authenticated using some key material known by the server. When the server receives the ticket, it checks that it is genuine (and still valid), decrypts it and uses the session state in the ticket for this new connection.

The message NewSessionTicket message is sent before the ChangeCipherSpec and is therefore not encrypted. An attacker can obtain the session ticket but cannot use it to resume a connection on behalf of the client as it does not know the master secret.

struct {
    opaque key_name[16];
    opaque iv[16];
    opaque encrypted_state<0..2^16-1>;
    opaque mac[32];
} ticket;

struct {
    ProtocolVersion protocol_version;
    CipherSuite cipher_suite;
    CompressionMethod compression_method;
    opaque master_secret[48];
    ClientIdentity client_identity;
    uint32 timestamp;
} StatePlaintext;

enum {
    anonymous(0),
    certificate_based(1),
    psk(2)
} ClientAuthenticationType;

struct {
    ClientAuthenticationType client_authentication_type;
    select (ClientAuthenticationType) {
        case anonymous: struct {};
        case certificate_based:
            ASN.1Cert certificate_list<0..2^24-1>;
        case psk:
            opaque psk_identity<0..2^16-1>;   /* from [RFC 4279] */
    };
} ClientIdentity;

ASN1_SEQUENCE(SSL_SESSION_ASN1) = {
    ASN1_EMBED(SSL_SESSION_ASN1, version, UINT32),
    ASN1_EMBED(SSL_SESSION_ASN1, ssl_version, INT32),
    ASN1_SIMPLE(SSL_SESSION_ASN1, cipher, ASN1_OCTET_STRING),
    ASN1_SIMPLE(SSL_SESSION_ASN1, session_id, ASN1_OCTET_STRING),
    ASN1_SIMPLE(SSL_SESSION_ASN1, master_key, ASN1_OCTET_STRING),
    ASN1_IMP_OPT(SSL_SESSION_ASN1, key_arg, ASN1_OCTET_STRING, 0),
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, time, ZINT64, 1),
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, timeout, ZINT64, 2),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, peer, X509, 3),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, session_id_context, ASN1_OCTET_STRING, 4),
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, verify_result, ZINT32, 5),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, tlsext_hostname, ASN1_OCTET_STRING, 6),
#ifndef OPENSSL_NO_PSK
    ASN1_EXP_OPT(SSL_SESSION_ASN1, psk_identity_hint, ASN1_OCTET_STRING, 7),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, psk_identity, ASN1_OCTET_STRING, 8),
#endif
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, tlsext_tick_lifetime_hint, ZUINT64, 9),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, tlsext_tick, ASN1_OCTET_STRING, 10),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, comp_id, ASN1_OCTET_STRING, 11),
#ifndef OPENSSL_NO_SRP
    ASN1_EXP_OPT(SSL_SESSION_ASN1, srp_username, ASN1_OCTET_STRING, 12),
#endif
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, flags, ZUINT64, 13),
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, tlsext_tick_age_add, ZUINT32, 14),
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, max_early_data, ZUINT32, 15),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, alpn_selected, ASN1_OCTET_STRING, 16),
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, tlsext_max_fragment_len_mode, ZUINT32, 17),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, ticket_appdata, ASN1_OCTET_STRING, 18),
    ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, kex_group, UINT32, 19),
    ASN1_EXP_OPT(SSL_SESSION_ASN1, peer_rpk, ASN1_OCTET_STRING, 20)
} static_ASN1_SEQUENCE_END(SSL_SESSION_ASN1)

Impact of session resumption on security

Extra features

Mutual TLS authentication

Usually, only the server is authenticated at the TLS level. However, TLS supports authenticating the client as well^[12] using public-key cryptography. This is called mutual (or client) TLS authentication (mTLS).

When the server supports mutual authentication, it sends a CertificateRequest message. This message can include information about which certificates are accepted by the server: the certificate_authorities field can list the distinguished names (DN) of the certificate authorities (CA) (either root or intermediate) accepted by the server.

enum {
    rsa_sign(1), dss_sign(2), rsa_fixed_dh(3), dss_fixed_dh(4),
    rsa_ephemeral_dh_RESERVED(5), dss_ephemeral_dh_RESERVED(6),
    fortezza_dms_RESERVED(20),
    ecdsa_sign(64), rsa_fixed_ecdh(65), ecdsa_fixed_ecdh(66),
    (255)
} ClientCertificateType;

opaque DistinguishedName<1..2^16-1>;

struct {
    ClientCertificateType certificate_types<1..2^8-1>;
    SignatureAndHashAlgorithm supported_signature_algorithms<2^16-1>;
    DistinguishedName certificate_authorities<0..2^16-1>;
} CertificateRequest;

The server advertizes which type(s) of client certificate it is willing to accept:

• rsa_sign, client authentication with RSA signature;
• dss_sign, client authentication with DSA signature;
• ecdsa_sign, client authentication with ECDSA signature;
• rsa_fixed_dh and dss_fixed_dh, client authentication with fixed FFDH key (both are identical in TLS v1.2);
• rsa_fixed_ecdh and ecdsa_fixed_ecdh, client authentication with fixed ECDH key (deprecated in TLS v1.2).

If the client chooses not to authenticate itself, it sends an empty Certificate message: the server may choose to either accept or reject (by terminating the TLS connection with a handshake_failure error) non-authenticated clients.

If the client chooses to authenticate itself, it first sends its certificate chain in a Certificate message:

• If the client certificate contains a signing public key (*_sign) the client then sends a CertificateVerify message which contains a signature of all the previous handshake messages.
• If the client certificate contains a (static) Diffie-Hellman public key (*_fixed_dh or *_fixed_ecdh), the client does not generate an ephemeral public key but uses the public key in the certificate. This is only possible if the cipher suite uses a compatible Diffie-Hellman key exchange (FFDH vs. ECDH) and the same Diffie-Hellman parameters. The rsa_fixed_ecdh and ecdsa_fixed_ecdh client certificate types are supposed to be only usable using ECDH_ECDSA and ECDH_RSA (i.e., non-ephemeral) key exchanges (but I did not find the associated requirement for rsa_fixed_dh and dss_fixed_dh).

struct {
    digitally-signed struct {
        opaque handshake_messages[handshake_messages_length];
    }
} CertificateVerify;

Renegotiation

The client may start a new handshake in order to renegotiate the security parameters of a connection at any time after the initial handshake.

Why? Reasons for using TLS renegotiation may include:

• for rekeying. There is a maximum amount of data that can be safely encrypted/MACed with one encryption/MAC key (depending on the encryption scheme). For long-live TLS sessions, it may be necessary to change encryption keys. TLS renegotiation may be used for this purpose.
• to provide confidentiality for the client authentication. The client Certificate message is transmitted in the handshake before the CipherSpecChaneg message. It is therefore sent in cleartext in the initial handshake. Doing the mutual client authentication in a renegotiation handshake can be used to avoid sending the user Certificate in cleartext.
• for mid-session client-authentication. Some application way want to propose client authentication mid-session. For example, some HTTPS applications may want to propose client authentication only when accessing certain restricted resources. In this case, the server may request a renegotiation and propose a TLS client authentication (but see the restrictions in the previous subsection).

How? In order to do this, the client sends a ClientHello message and proceeds with a new handshake. The client may start the TLS renegotiation unilaterally or at the request of the server (HelloRequest message). As the renegotiation handshake happens after the ChangecipherSpec, it is protected (encrypted) using the key material of the existing connection.

struct { } HelloRequest

Secure renegotiation is a fix at the TLS protocol level for CVE-2009-3555. In introduces the renegotiation_info TLS extension. In the initial handshake, this extension is only used to negotiate the usage of secure renegotiation. In the renegotiation handshake, these extension contain:

• for the client, the client verify_data of the original handshake;
• for the server, the client verify_data and the server verify_data of the original handshake.

Alternative certificates

By default, TLS uses X.509 certificates. Alternative public keys formats can be used instead of X.509 certificates using the client_certificate_type and server_certificate_type TLS extensions.

This can be used to use exchange raw public keys instead of X.509 certificates. In this case, the Certificate messages contain the raw public key of the peer. Raw public keys may for example, be used for IoT applications in order to avoid relying on a public key infrastructure (PKI). The devices may for example be identified with a hash of their public keys. Alternatively, a similar result could be achieved by exchanging self-signed X.509 certificates and only process the public keys.

opaque ASN.1Cert<1..2^24-1>;

struct {
    select(certificate_type){

        // certificate type defined in this document.
        case RawPublicKey:
            opaque ASN.1_subjectPublicKeyInfo<1..2^24-1>;

        // X.509 certificate defined in RFC 5246
        case X.509:
            ASN.1Cert certificate_list<0..2^24-1>;

        // Additional certificate type based on
        // "TLS Certificate Types" subregistry
    };
} Certificate;

TLS-PSK

TLS-PSK includes key exchange algorithms which provide mutual authentication based on a (per-client) shared secret, the pre shared key (PSK). This usually (with the exception of RSA_PSK) does not use certificates for server or client authentication at all. This can be useful for low-performance environments such as IoT applications.

PSK key exchange algorithms:

• PSK, mutual authentication based on a PSK;
• RSA_PSK, mutual authentication based on a PSK combined with a RSA encrypted secret;
• DHE_PSK, mutual authentication based on a PSK combined with an ephemeral FFDH key exchange;
• ECDHE_PSK mutual authentication based on a PSK combined with an ephemeral ECDH key exchange.

A psk_identity is included in the ClientKeyExchange. It has the same role as login, identifying the client that is trying to authenticate.

struct {
    select (KeyExchangeAlgorithm) {
        case psk:
            opaque psk_identity_hint<0..2^16-1>;
        case diffie_hellman_psk:
            opaque psk_identity_hint<0..2^16-1>;
            ServerDHParams params;
        case rsa_psk:
            opaque psk_identity_hint<0..2^16-1>;
        case ec_diffie_hellman_psk:
            opaque psk_identity_hint<0..2^16-1>;
            ServerECDHParams params;
    };
} ServerKeyExchange;

struct {
    select (KeyExchangeAlgorithm) {
        case psk:
            opaque psk_identity<0..2^16-1>;
        case diffie_hellman_psk:
            opaque psk_identity<0..2^16-1>;
            ClientDiffieHellmanPublic public;
        case rsa_psk:
            opaque psk_identity<0..2^16-1>;
            EncryptedPreMasterSecret;
        case ec_diffie_hellman_psk:
            opaque psk_identity<0..2^16-1>;
            ClientECDiffieHellmanPublic public;
    } exchange_keys;
} ClientKeyExchange;

FAQ

How are used the random values in the hello message?

The random values in the hello messages are used for:

• deriving the master secret from the pre master secret;
• deriving the key material from the master secret.

This prevents some forms of replay attacks by ensuring that both participants contribute some ephemeral value to the master secret. This is especially important when non non-DHE key exchange is used or for session resumption.

The random values are used as well in the digital signatures used for authentication (for example in in the DHE_* and ECDHE_* key exchange algorithms):

struct {
    select (KeyExchangeAlgorithm) {
        case dhe_dss:
        case dhe_rsa:
            ServerDHParams params;
            digitally-signed struct {
                opaque client_random[32];
                opaque server_random[32];
                ServerDHParams params;
            } signed_params;
        case ecdhe_dss:
        case ecdhe_rsa:
        case ecdh_anon:
            ServerECDHParams    params;
            digitally-signed struct {
                opaque client_random[32];
                opaque server_random[32];
                ServerDHParams params;
            } signed_params;
    };
} ServerKeyExchange;

By including a new random for each TLS connection, the client can be sure that the server ephemeral DH public key is not being replayed.

How is the hash function used in the cipher suite?

The hash function included in the cipher suite is used:

• for payload protection (using HMAC) when the encryption method is not an AEAD;
• for defining a pseudorandom function (PRF)^[13], the TLS PRF.

The TLS PRF is used:

• for deriving the verify_data field in the Finished messages (working as a MAC);
• for deriving the master secret (from the pre master secret);
• for deriving the key material (from the master secret).

Appendix, key hierarchy

Notes:

• When using session resumption, the derivation of master secret uses the random values from the initial handshake but the derivation of key block uses the random values from the current handshake.
• When using the extended master secret extensions, the master secret is using the transcript hash instead of the client and server random values.

Pre Master Secret

The pre master secret is obtained as a result of the key exchange algorithm. Depending on the key exchange algorithm it may:

• be chosen by the client (RSA key exchange);
• be computed as the result of a Diffie-Hellman key exchange;
• be computed from a pre shared key (PSK);
• or a combination of those.

Master Secret

The master secret is derived from pre master secret and the two nonces (client and server random). The master secret is reused in case of session resumption. It is computed as:

master_secret = PRF(pre_master_secret, "master secret",
                    ClientHello.random + ServerHello.random)
                    [0..47];

where the PRF depends on the cipher suite.

The inclusion of the server and client nonces prevents some form of replay attacks by making sure that both participants contribute some ephemeral value to the master secret.

When the extended_master_secret extension is used, the hash of all the previous handshake messages is used instead of only using the server and client random values. This binds the master secret with the full TLS handshake which fixes the triple handshake attack.

master_secret = PRF(pre_master_secret, "extended master secret",
                    session_hash)
                    [0..47];

When using session resumption, the same master secret is reused for all connections associated with the same TLS session.

Key Material

The key material is computed as:

key_block = PRF(SecurityParameters.master_secret,
                      "key expansion",
                      SecurityParameters.server_random +
                      SecurityParameters.client_random);

This key material is decomposed in client and server MAC keys, encryption keys and initialization vectors (IV):

client_write_MAC_key[SecurityParameters.mac_key_length]
server_write_MAC_key[SecurityParameters.mac_key_length]
client_write_key[SecurityParameters.enc_key_length]
server_write_key[SecurityParameters.enc_key_length]
client_write_IV[SecurityParameters.fixed_iv_length]
server_write_IV[SecurityParameters.fixed_iv_length]

As the master secret is reused in case of session resumption, the inclusion of the server and client random values ensures that the key material changes is different from different TLS connections in the same TLS session: this prevents replay attacks through session resumption.

Keying Material Exporters

The Keying Material Exporters mechanism defines a way to generate key material as a result of the TLS connection to be used in other protocols.

This can either be:

PRF(SecurityParameters.master_secret, label,
               SecurityParameters.client_random +
               SecurityParameters.server_random
               )[length]
PRF(SecurityParameters.master_secret, label,
               SecurityParameters.client_random +
               SecurityParameters.server_random +
               context_value_length + context_value
               )[length]

where the label depends on the consuming application/protocol.

A IANA registry of existing labels can be used to find some usages of this feature.

Appendix, using TLS for authentication only

In some applications, no application protocol data is transported on top of the TLS connection. Instead, the TLS protocl is only used for the secure handshake and the authentication it provides. The secrets established in the TLS handshake may be used to generate cryptograpic material for another protocol (see the Keying Material Exporters) which does not sit on top of TLS. This approach is for example used in OpenVPN, WPA-EAP (WPA-entreprise) with EAP-TLS, and QUIC^[14]).

                     [ TLS     | IP   ]
[ TLS | IP or Eth. ] [ EAP-TLS | SNAP ] [ TLS     ]      [ TLS | HTTP3 ]
[ OpenVPN          ] [ EAP     | LLC  ] [ EAP-TLS ]      [ QUIC        ]
[ TCP or UDP       ] [ EAPOL   | CCMP ] [ EAP     | IP ] [ UDP         ]
[ IP               ] [ Wifi           ] [ PPP          ] [ IP          ]
     OpenVPN              WPA2-EAP           PPP             HTTPS
                            with             with           (HTTP/3)
                           EAP-TLS          EAP-TLS

Some EAP methods (EAP-TTLS, PEAP and TEAP) use TLS to secure (“tunnel”) the traffic of another authentication method. As before, the secrets established in the TLS handshake may be used to derive exported key material for other protocols (eg. in WPA-EAP).

            [ ...      ]
            [ EAP      ] [ PAP      ] [ CHAP     ] [ MS-CHAP  ] [ ...  ] [ ...  ]
            [ AVP      ] [ AVP      ] [ AVP      ] [ AVP      ] [ EAP  ] [ EAP  ]
[ mTLS    ] [ TLS      ] [ TLS      ] [ TLS      ] [ TLS      ] [ TLS  ] [ TLS  ]
[ EAP-TLS ] [ EAP-TTLS ] [ EAP-TTLS ] [ EAP-TTLS ] [ EAP-TTLS ] [ PEAP ] [ TEAP ]
[ EAP     ] [ EAP      ] [ EAP      ] [ EAP      ] [ EAP      ] [ EAP  ] [ EAP  ]
[ ...     ] [ ...      ] [ ...      ] [ ...      ] [ ...      ] [ ...  ] [ ...  ]
  EAP-TLS       EAP           PAP          CHAP       MS-CHAP     EAP      EAP
                over          over         over        over       over     over
             EAP-TTLS       EAP-TTLS     EAP-TTLS     EAP-TLS     PEAP     TEAP

See the EAP-TTLS AVP Usage subregistry for list of authentication methods compatible with EAP-TTLS.

Appendix, cleartext client identity

In TLS v1.2, the client sends its certificate chain before the cipher change. As a consequence, the certificate chain is sent in cleartext (unless it is sent as part of the TLS renegotiation). For example, some VPN solutions (eg. OpenvPN) provide support for mTLS based client authentication. In this case, the identity of the client might be leaked in cleartext.

This can be seen in the following screenshot of Wireshark inspecting an OpenVPN handshake.

Text version:

Frame 15: 1264 bytes on wire (10112 bits), 1264 bytes captured (10112 bits)
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 55934, Dst Port: 1194, Seq: 259, Ack: 1596, Len: 1198
OpenVPN Protocol
Transport Layer Security
    TLSv1.2 Record Layer: Handshake Protocol: Certificate
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 999
        Handshake Protocol: Certificate
            Handshake Type: Certificate (11)
            Length: 995
            Certificates Length: 992
            Certificates (992 bytes)
                Certificate Length: 989
                Certificate: 308203d9308202c1a003020102021414e338472bac1eece342df5fc60e83779784a25830… (pkcs-9-at-emailAddress=john.doe@example.com,id-at-commonName=Jon Doe,id-at-organizationName=Internet Widgits Pty Ltd,id-at-stateOrProvinceName=Some-S
                    signedCertificate
                        version: v3 (2)
                        serialNumber: 0x14e338472bac1eece342df5fc60e83779784a258
                        signature (sha256WithRSAEncryption)
                        issuer: rdnSequence (0)
                        validity
                        subject: rdnSequence (0)
                            rdnSequence: 5 items (pkcs-9-at-emailAddress=john.doe@example.com,id-at-commonName=Jon Doe,id-at-organizationName=Internet Widgits Pty Ltd,id-at-stateOrProvinceName=Some-State,id-at-countryName=AU)
                                RDNSequence item: 1 item (id-at-countryName=AU)
                                RDNSequence item: 1 item (id-at-stateOrProvinceName=Some-State)
                                RDNSequence item: 1 item (id-at-organizationName=Internet Widgits Pty Ltd)
                                RDNSequence item: 1 item (id-at-commonName=Jon Doe)
                                RDNSequence item: 1 item (pkcs-9-at-emailAddress=john.doe@example.com)
                        subjectPublicKeyInfo
                        extensions: 3 items
                    algorithmIdentifier (sha256WithRSAEncryption)
                    Padding: 0
                    encrypted: 342782ae2f94bf0123ade8d88117423e9bfeaefe69f9f93fc9f6a2104d4595f7bc47f6a3…
    TLSv1.2 Record Layer: Handshake Protocol: Client Key Exchange

This is especially problematic if the OpenVPN tunnel is intended to protect your privacy. You probably should not include personally identifiable information in the client certificate if you are using TLS v1.2 or below.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout client.key -out client.crt
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout client.key -out client.crt
openssl dhparam -out dh2048.pem 2048
sudo openvpn --mode server --port 1194 --local 127.0.0.1 --dev tun0 \
    --proto tcp-server --tls-server --ca client.crt --cert server.crt \
    --dh dh2048.pem --key server.key \
    --tls-version-min 1.2 --tls-version-max 1.2 --verify-client-cert require
sudo tcpdump "tcp port 1194" -w openvpn-tls12.pcap -i lo
sudo openvpn --remote 127.0.0.1 1194 tcp-client --dev tun1 --client \
    --ca server.crt --cert client.crt --key client.key \
    --tls-version-min 1.2 --tls-version-max 1.2
wireshark openvpn-tls12.pcap

OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Apr 28 2021
library versions: OpenSSL 1.1.1d  10 Sep 2019, LZO 2.10

Appendix, KCI TLS vulnerability

We have seen that authentication based on static DH is vulnerable to Key Compromise Impersonation (KCI). The KCI TLS vulnerability (CVE-2015-8960) is a practical vulnerability of this. If an attacker tricks the user into installing a client certificate (and associated private key) chosen by the attacker into his web browser, the attacker can impersonate any web site which uses a compatible certificate (in the same DH group) even if this web site does not support static DH (and client authentication) at all.

The attack may go as follows:

1. the attacker generates a ECDH key pair and generates an associated (possibly self-signed) TLS client certificate;
2. the attacker tricks the user info installing a PKCS#12 file containing the chosen client certificate and private key to the user,
3. the user browser tries to connect to a website (eg. https://example.com) which has a certificate with a EC public key which can be used for ECDH (either no key usage extension or keyAgreement in the key usage extension);
4. the attacker intercepts comunications and TLS handshake starts between the client and the attacker;
5. the attacker negotiates the usage of a cipher suite with static ECDH server key (ECDH_* key exchange);
6. the attacker sends the certificate for https://example.com (containing the server ECDSA signing key);
7. the client interprets this EC public key as an ECDH private key (key usage confusion);
8. the attacker requests client authentication;
9. the browser proposes using the client certificate for client authentication;
10. the user accepts;
11. key material is derived from a static-static ECDH key exchange;
12. the attacker has the client static ECDH private key (because he has generated it) and the server static ECDH public key and can compute the pre master secret, master secret and key material.

The following conditions must be met for this attack (see the KCI TLS slides):

• the client must support TLS v1.2 or below;
• the client must accept to use fixed DH (FFDH or ECDH) authentication;
• the target server certificate must,
  • contain either an EC public key (which could be intended to be used for ECDH or ECDSA) or a FFDH public key (matching with the client certificate);
  • either no no key usage certificate extension or a key usage extension which includes the keyAgreement usage.

Mitigations:

• many implementation have removed (or disabled) support for client authentication based on static ECDH key (this is deprecated since TLS v1.2);
• fixed DH is disabled is many implementations (eg. recent browsers) anyway;
• the ECDH vs. ECDSA key usage confusion can be prevented by including the digitalSignature key usage (but not keyAgreement) in certificates which are expected to be used for signing.

Appendix, DNS-rebinding for HTTPS using session resumption and RSA transport

An interesting vulnerability, uses session resumption for DNS-rebinding attacks against certain HTTPS services.

TLS-based services can be protected against DNS-rebinding by enforcing the value of the TLS-level server name (SNI extension). Application-layer server name (eg. the Host HTTP header) can be enforced as well for the same effect.

Even if neither TLS-level server name nor application-layer server name is enforced, DNS-rebinding attacks are normally prevented by TLS: this is because, when the client tries to establish a TLS connection to the target.example server instead of attacker.example, the client expects a certificate chain for attacker.example but receives a certificate chain for target.example it should reject the TLS connection.

However, the attacker can manage to avoid server certificates by using session resumption. In order for this to work, the attacker has to:

1. associate the master secret established in the client-attacker TLS session with a session in the target;
2. forward the target session ID to the client.

This can easily be done using the RSA transport key exchange algorithm as explained in the following diagram.

The consequence of this attack is that the attacker can issue arbitrary requests to the target website from the client and access the responses. This can be useful for several reasons:

• the malicious requests to the target server is coming from the client IP address (the attacker IP address is typically not even logged in the access log because not HTTP request is made by the attacker);
• this can be used to bypass IP-address restrictions or rate-limiting.

Prerequisites:

• TLS v1.2 support in client and target server;
• session resumption support in client and target server;
• RSA transport support in client and target server;
• default virtual host in server.

Several workarounds have been implemented in different implementations. This should be fixed by using the extended master secret extension if it is implemented as specified:

If the original session did not use the "extended_master_secret" extension but the new ClientHello contains the extension, then the server MUST NOT perform the abbreviated handshake.

This is fixed in TLS v1.3.

Appendix, certificate validation

Without going into details, the validation of the certificate chain may include:

1. verification of the validity of each certificate (notBefore, notAfter);
2. verification of the signature of each certificate in the chain up to a trusted root CA;
3. verification that the server host name matches with one of the names in the server certificate (SubjectAltName extension);
4. processing of other certificate extensions such as:
   • validating the key usage (eg. digitalSignature)
   • validating the Extended Key Usage (i.e., Server Authentication vs. Client Authentication);
   • validating Basic Constraints extensions (CA flag);
   • check certificate revocation using Certification Revocation Lists (CRL), OCSP or OCSP stapling;
5. enforcing certificate pinning, which may
   • be configured using HTTP Public Key Pinning (HPKP) (deprecated);
   • hardcoded in the browser;
   • hardcoded in the application (often implemented in mobile applications);
6. enforcing DANE/TLSA contraints;
7. enforcing the presence of Signed Certificate Timestamps (SCTs) (which may be in the signed_certificate_timestamp TLS extension, in the SignedCertificateTimestampList X.509/OCSP extension) and validating them (for certificate transparency).

References

RFCs:

• RFC 5248, TLS v1.2
• RFC 6066, some extensions (including SNI)
• RFC 4492, Elliptic Curves Cryptography (ECC)
• RFC 7301, ALPN
• RFC 7366, Encrypt-then-MAC
• RFC 7905, ChaCha20-Poly1305
• RFC 7250, raw public keys
• RFC 5077, tickets (for session resumption)
• RFC 5288, AES GCM for TLS
• RFC 7366, Encrypt-then-MAC extension
• RFC 7919, Supported Groups extension
• RFC 5705, Keying Material Exporters
• RFC 3436, TLS over SCTP
• RFC 4279, TLS-PSK
• RFC 4785, TLS-PSK with NULL encryption
• RFC 5487, TLS-PSK with SHA-256/384 and AES GCM
• RFC 5489, ECDHE_PSK (TLS-PSK)
• RFC 5054, TLS-SRP
• RFC 7925, TLS profile for IoT
• RFC 5280, PKIX Certificate and CRL Profile
• RFC 4159, X.509 Certification Path Building
• RFC 5746, TLS Renegotiation Indication Extension
• RFC 9000, QUIC
• RFC 9001, Using TLS to secure QUIC
• RFC 7525, Recommendations for Secure Use of TLS and DTLS
• RFC 9325, Recommendations for Secure Use of TLS and DTLS

Registries:

• TLS registry
• Extensions Registry
• X.509 PKIX registry

Misc:

• Ciphersuite, description of TLS cipher suites
• The Illustrated TLS Connection for TLS v1.2
• The Order of Encryption and Authentication for Protecting Communications
• The Cryptographic Doom Principle
• Triple Handshakes Considered Harmful: Breaking and Fixing Authentication over TLS
• Imperfect Forward Secrecy: How Diffie-Hellman Fails in Practice, David Adrian et al
• Weak Diffie-Hellman and the Logjam Attack
• Prying open Pandora’s box: KCI attacks against TLS, Clemens Hlauschek et al
• Lecture Notes on Cryptography, Shafi Goldwasser and Mihir Bellare
• Practical Issues with TLS Client Certificate Authentication, Arnis Parsovs
• DNS rebinding for HTTPS
• TLS “secrets“ What everyone forgot to tell you... ~ Florent Daignièere, Matta Consulting Ltd, Blackhat USA, July 2014