{"version": "https://jsonfeed.org/version/1", "title": "/dev/posts/ - Archive for 2017", "home_page_url": "https://www.gabriel.urdhr.fr", "feed_url": "/2017/feed.json", "items": [{"id": "http://www.gabriel.urdhr.fr/2017/08/02/foo-over-ssh/", "title": "Foo over SSH", "url": "https://www.gabriel.urdhr.fr/2017/08/02/foo-over-ssh/", "date_published": "2017-08-02T00:00:00+02:00", "date_modified": "2017-08-02T00:00:00+02:00", "tags": ["computer", "network", "ssh", "unix"], "content_html": "
A comparison of the different solutions for using SSH2 as a secured\ntransport for protocols/services/applications.
\nThe SSH-2 protocol uses its\nTransport Layer Protocol to provide\nencryption, confidentiality, server authentication and integrity over a\n(potentially) unsafe reliable bidirectional data stream (usually TCP port 22):
\nThe transport layer transports SSH packets.\nIt handles:
\nEach packet starts with a message number and can belong to:
\nTypical protocol stack (assuming TCP/IP):
\n\n [Session | Forwarding]\n[SSH Authn. |SSH Connection ]\n[SSH Transport ]\n[TCP ]\n[IP ]\n\n
The Connection Protocol is used\nto manage channels\nand transfers data over them. Each channel is (roughly) a bidirectionnal\ndata stream:
\nSSH_MSG_CHANNEL_EXTENDED_DATA
) in addition of the main data stream\n(SSH_MSG_CHANNEL_DATA
) [4];SSH_MSG_CHANNEL_OPEN
)\nwhich may be accepted (SSH_MSG_CHANNEL_OPEN_CONFIRMATION
)\nor rejected (SSH_MSG_CHANNEL_OPEN_FAILURE
) by the other side;SSH_MSG_CHANNEL_EOF
);Multiple channels can be multiplexed over the same SSH connection:
\n\nC \u2192 S SSH CHANNEL_DATA(1, \"whoami\\n\")\nC \u2192 S SSH CHANNEL_DATA(2, \"GET / HTTP/1.1\\r\\nHost: foo.example.com\\r\\n\\r\\n\")\nC \u2190 S SSH CHANNEL_DATA(5, \"root\\n\")\nC \u2190 S SSH CHANNEL_DATA(6, \"HTTP/1.1 200 OK\\r\\nContent-Type:text/plain\\r\\n\")\nC \u2190 S SSH CHANNEL_DATA(6, \"Content-Length: 11\\r\\n\\r\\nHello World!\")\n\n
A session
channel is used to start:
SSH_MSG_CHANNEL_REQUEST(chan, \"shell\", \u2026)
;SSH_MSG_CHANNEL_REQUEST(chan, \"exec\", \u2026, command)
) which is\nusually passed to the user shell;SSH_MSG_CHANNEL_REQUEST(chan, \"subsystem\", \u2026, subsystem_name)
).For session channels, the protocol has support for setting environment variables,\nallocating a server-side TTY, enabling X11 forwarding, notifying of the terminal\nsize modification (see SIGWINCH
), sending signals, reporting the exit\nstatus or exit signal.
\nC \u2192 S SSH CHANNEL_OPEN(\"session\", 2, \u2026)\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(3, 6)\nC \u2192 S SSH CHANNEL_REQUEST(6, \"pty-req\", TRUE, \"xterm\", 80, 120, \u2026)\nC \u2190 S SSH CHANNEL_SUCCESS(3)\nC \u2192 S SSH CHANNEL_REQUEST(6, \"env\", TRUE, \"LANG\", \"fr_FR.utf8\")\nC \u2190 S SSH CHANNEL_SUCCESS(3)\nC \u2192 S SSH CHANNEL_REQUEST(6, \"exec\", TRUE, \"ls /usr/\")\nC \u2190 S SSH CHANNEL_SUCCESS(3)\nC \u2190 S SSH CHANNEL_DATA(3, \"bin\\ngames\\ninclude\\nlib\\nlocal\\sbin\\nshare\\nsrc\\n\")\nC \u2190 S SSH CHANNEL_EOF(3)\nC \u2190 S SSH CHANNEL_REQUEST(3, \"exit-status\", FALSE, 0)\nC \u2190 S SSH CHANNEL_CLOSE(3)\nC \u2192 S SSH CHANNEL_CLOSE(6)\n\n
Shell session channels are used for interactive session are not really\nuseful for protocol encapsulation.
\nIn SSH, a command is a single string.\nThis is not an array of strings (argv
).\nOn a UNIX-ish system, the command is usually expected to be called by the user's\nshell (\"$SHELL\" -c \"$command\"
): variable expansions, globbing are applied\nby the server-side shell.
ssh foo.example.com 'ls *'\nssh foo.example.com 'echo $LANG'\nssh foo.example.com 'while true; do uptime ; sleep 60 ; done'\n
\nA subsystem is a \u201cwell-known\u201d service running on top of SSH. It is\nidentified by a string which makes it system independent: it does not\ndepend on the user/system shell, environment (PATH
), etc.
With the OpenSSH client, a subsystem can be invoked with\nssh -S $subsystem_name
.
Subsystem names come in\ntwo forms:
\nservice_name@domain
;Well-known subsystem names include:
\nsftp
is used to connect a local SFTP client to a remote SFTP\nserver[5];publickey
is used for the\nSSH Public Key Substem which\ncan be used by clients to manage their SSH public keys;snmp
is used for\nSNMP over SSH;netconf
for\nNETCONF over SSH;rpki-rtr
for\nRPKI-Router over SSH.When using a subsystem:
\nWith the OpenSSH server, a command can be associated with a given\nsubsystem name with a configuration entry such as:
\nSubsystem sftp /usr/lib/openssh/sftp-server\n
\nThe command is run under the identity of the user with its own shell\n(\"$SHELL\" -c \"$command\"
).
If you want to connect to a socket you might use:
\nSubsystem http socat STDIO TCP:localhost:80\nSubsystem hello@example.com socat STDIO UNIX:/var/run/hello\n
\nIt is possible to use exec
to avoid keeping a shell process[6]:
Subsystem http exec socat STDIO TCP:localhost:80\nSubsystem hello@example.com exec socat STDIO UNIX:/var/run/hello\n
\nThis works but OpenSSH complains because it checks for the existence of\nan exec
executable file.
The SSH has support for forwarding (either incoming or outgoing)\nTCP connections.
\nLocal forwarding is used to forward a local connection (or any\nother local stream) to a remote TCP endpoint. A channel of type\ndirect-tcpip
is opened to initiate a TCP connection on the remote\nside. This is used by ssh -L
, ssh -W
and ssh -D
\nC \u2192 S SSH CHANNEL_OPEN(\"direct-tcpip\", chan, \u2026, \"foo.example.com\", 9000, \"\", 0);\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2, \u2026)\nC \u2192 S SSH CHANNEL_DATA(chan2, \"aaa\")\n\n
Remote forwarding is used to request to forward all incoming\nconnections on a remote port over the SSH connection. The remote side\nthen opens a new forwarded-tcpip
channel for each connection. This\nis used by ssh -R
.
\nC \u2192 S SSH GLOBAL_REQUEST(\"tcpip-forward\", remote_addr, remote_port)\nC \u2190 S SSH REQUEST_SUCCESS(remote_port)\n S \u2190 X Incoming connection\nC \u2190 S SSH CHANNEL_OPEN(\"forwarded-tcpip\", chan, \u2026, address, port, peer_address, peer_port)\nC \u2192 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2, \u2026)\n S \u2190 X TCP Payload \"aaa\"\nS \u2190 X SSH CHANNEL_DATA(chan2, \"aaa\")\n\n
Since OpenSSH 6.7, it is\npossible to involve (either local or remote) UNIX sockets in forwards\n(ssh -L
, ssh -R
, ssh -W
):
When the UNIX socket is on the client-side,\nclient support is needed but server-side support is not needed.
\nWhen the UNIX socket is on the server-side, both client\nand server support is needed. This is using an (OpenSSH) protocol extension\nwhich works similarly to the TCP/IP forwarding:
\nSSH_MSG_CHANNEL_OPEN(\"direct-streamlocal@openssh.com\", \u2026, path)
\nfor initiating a connection to a remote UNIX stream socket (local\nforwarding);SSH2_MSG_GLOBAL_REQUEST(\"streamlocal-forward@openssh.com\", TRUE, path)
is used to request a remote forwarding and each new\nconnection opens a channel with\nSSH_MSG_CHANNEL_OPEN(\"forwarded-streamlocal@openssh.com\", \u2026, path, \u2026)
.As an extension, OpenSSH has support for tunnel forwarding. A tunnel\ncan be either Ethernet-based (TUN devices) or IP based (TAP devices).\nAs channels do not preserve message boundaries, a header is prepended\nto each message (Ethernet frame or IP packet respectively): this\nheader contains the message length\n(and for IP based tunnels, the address family i.e. IPv4 or IPv6).
\nThis is used by ssh -w
.
Messages for an IP tunnel:
\n\nC \u2192 S SSH CHANNEL_OPEN(\"tun@openssh.com\", chan, \u2026, POINTOPOINT, \u2026)\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2)\nC \u2192 S SSH CHANNEL_DATA(chan2, encapsulation + ip_packet)\n\n
and the packets use the form:
\n4B packet length\n4B address family (SSH_TUN_AF_INET or SSH_TUN_AF_INET6)\nvar data\n
\nMessages for an Ethernet tunnel:
\n\nC \u2192 S SSH CHANNEL_OPEN(\"tun@openssh.com\", chan, \u2026, ETHERNET, \u2026)\nC \u2190 S SSH CHANNEL_OPEN_CONFIRMATION(chan, chan2)\nC \u2192 S SSH CHANNEL_DATA(chan2, encapsulation + ethernet_frame)\n\n
and the packets use the form:
\n4B packet length\nvar data\n
\nThe x11
channel type is used for\nX11 forwarding.
scp
uses SSH to spawn a remote-side scp
process. This remote scp
\nprocess communicates with the local instance using its stdin
and\nstdout
.
When the local scp
sends data, it spawns:
scp -t /some_path/\n
\nWhen the local scp
receives data, it spawns:
scp -f /some_path/some_file\n
\nrsync can work over SSH. In this mode of operation, it uses SSH to\nspawn a server rsync process which communicates with its stdin
and\nstdout
.
The local rsync
spawns something like in the remote side:
rsync --server -e.Lsfx . /some_path/\n
\nSFTP is a file transfer protocol.\nIt is expected to to work on top of SSH\nusing the sftp
subsystem. However it can work on top of other streams\n(see sftp -S $program
and sftp -D $program
).
This is not FTP running over SSH.
\nFISH\n(Files transferred over Shell)\nis another solution for file system operation over a\nremote shell (such as rsh
or ssh
): it uses exec
sessions to\nexecute standard UNIX commands on the remote side in order to do the\noperations. This first approach will not work if the remote side is\nnot a UNIXish system: in order to have support for non UNIX, it\nencodes the same requests as special comments at the beginning of the\ncommand.
Git spawns a remote git-upload-pack /some_repo/
which communicates\nwith the local instance using its standard I/O.
Many systemd *ctl
tools (hostnamectl
, busctl
, localectl
,\ntimedatectl
, loginctl
, systemctl
) have builtin support for\nconnecting to a remote host. They use a ssh -xT $user@$host
\nsystemd-stdio-bridge. This\ntool connects to the D-Bus\nsystem bus\n(i.e. ${DBUS_SYSTEM_BUS_ADDRESS:-/var/run/dbus/system_bus_socket}
).
Program | \nSolution | \n
---|---|
scp | \nCommand (scp ) | \n
rsync | \nCommand (rsync ) | \n
sftp | \nSubsystem (sftp ) | \n
FISH | \nCommands / special comments | \n
git | \nCommand (git-upload-pack ) | \n
systemd | \nCommand (systemd-stdio-bridge ) | \n
Which solution should be used to export your own\nprotocol over SSH? The shell, X11 forwarding and TUN/TAP forwarding\nare not really relevant in this context so we are left with:
\nUsing a dedicated subsystem is the cleaner solution.\nThe subsystem feature of SSH has been designed for this kind of application:\nit is supposed to hide implementation details such as the shell,\nPATH
, whether the service is exposed as a socket or a command,\nwhat is the location of the socket,\nwhether socat
is installed on the system, etc.\nHowever with OpenSSH, installing a new subsystem is done by adding a new entry\nin the /etc/ssh/sshd_config
file which is not so convenient for packaging\nand not necessarily ideal for configuration management.\nAn Include
directive has been included for ssh_config
\n(client configuration) in OpenSSH 7.3: the same directive for sshd_config
\nwould probably be useful in this context.\nIn practice, the subsystem feature seems to be mostly used by sftp
.
Using a command is the simpler solution: the only requirement is to\nadd a suitable executable, preferably in the PATH
. Moreover, the\nuser can add their own commands (or override the system ones) for their\nown purpose by adding executables in its own PATH
.
These two solutions have a few extra features which are not really\nnecessary when used as a pure stream transport protocol but might be\nhandy:
\nAccept-Language
) for free with LANG
, and\nLC_*
;stderr
which is not really useful\nfor encapsulating protocols.The two forwarding solutions have fewer features which are more in\nline with what is expected of a stream transport but:
\nmyservice@example.com
for custom non-standard protocols.The command and subsystem solutions run code with the user's identity\nand will by default run with the user permissions. The setuid
and\nsetgid
bits might be used if this is not suitable.
Another solution is to use socat
or netcat to connect to a socket and get\nthe same behavior as socket forwarding (security-wise).
For Unix socket forwarding, OpenSSH uses the user identity to connect\nto the socket. The daemon can use SO_PEERCRED
(on Linux, OpenBSD),\ngetpeereid()
\n(on BSD),\ngetpeerucred()
\n(Solaris) to get the user UID, GID in order to avoid a second\nauthentication. On Linux, file-system permissions can be used to\nrestrict the access to the socket as well.
For TCP socket forwarding, OpenSSH uses the user identity to connect to\nthe socket and ident
(on localhost) might be used in order to get\nthe user identity but this solution is not very pretty.
I kind-of like the subsystem feature even if it is not used that much.
\nThe addition of an Include
directive in sshd_config
might help deploying\nsuch services. Another interesting feature would be an option to associate a\nsubsystem with a Unix socket (without having to rely on socat
).
The random padding is used to make the whole Binary Packet Protocol message\na multiple of the cipher block size (or 8 if the block size is smaller). \u21a9\ufe0e
\nThe receiver uses the\nSSH_MSG_CHANNEL_WINDOW_ADJUST
\nmessage to request more data. \u21a9\ufe0e
Each channel is associated with two integer IDs, one for each side\nof the connection. \u21a9\ufe0e
\nThis is used to transport both stdout
(SSH_MSG_CHANNEL_DATA(channel, data)
)\nand stderr
(SSH_MSG_CHANNEL_EXTENDED_DATA(channel, SSH_EXTENDED_DATA_STDERR, data)
)\nover the same session channel. \u21a9\ufe0e
It is currently not yet registered but it is described in the SFTP\ndrafts\nand widely deployed. \u21a9\ufe0e
\nbash
already does an implicit exec
when bash -c \"$a_single_command\"
is used. \u21a9\ufe0e