Document custom connections

[pyhedgehog]

Please document how can I initiate connection of ssh client via some non-standard transport (not via tcp or ssh-forwarding). Right now I need to use socks, but it can be almost everything (http "CONNECT", websocket, unix fifo, etc.). Maybe I've missed already-existing reference?

[ronf]

I looked into supporting this at one point by allowing an already-connected socket object to be passed into asyncssh.create_connection(), similar to what is supported by asyncio.create_connection(). However, SSH would still require you to still provide things like the host & port, so it could use those to do a lookup in the known_hosts file. The object would also have to support all the methods of the socket class, and would need to be compatible with the asyncio event loop, which would require it to expose something like a file descriptor that could be waited on.

This could probably be made to work for something like SOCKS or HTTP CONNECT where you do a handshake on a regular TCP socket at the beginning and then can just pass that socket on to be used directly after that handshake is complete. It would be more difficult to support something like a web socket, though, where every read & write has to worry about adding additional framing bytes. There's also the question of how you'd handle the multiplexing that something like a web socket would support.

UNIX fifos would break some things, like the way AsyncSSH currently tries to query peer information from the socket and expects to get back IPv4/IPv6 endpoint information. It would also cause problems when it tried to set socket options like SO_KEEPALIVE and TCP_NODELAY.

If you can open an established socket-like object which supports the asyncio event loop, it wouldn't be a difficult change to support it in asyncssh.create_connection(). Basically, instead of calling loop.create_connection() with host, port, family, flags, and local_addr, you'd call loop.create_connection() with just the conn_factory and a "sock" argument that was your already-established socket object. I could potentially support this as long as you also still provided host & port, but it would only work in the limited set of cases where the object was still basically a TCP socket (or pretended really well to be one).

[pyhedgehog]

Also I'm looking for something like ProxyCommand. I mean openssh config directive that allows to use external process as an ssh connector. There can't be sane "peer".

[ronf]

Unfortunately, there's nothing like that available right now. With the change describe above, it would be possible to do something like this by creating a socketpair(), though, and then having Python start a subprocess which ran your proxy command with stdin/stdout redirected to one side of the socketpair and with the other side passed to AsyncSSH.

That said, one of the main goals of AsyncSSH was to avoid having a separate process for each SSH connection you open. If you use something like ProxyCommand, you end up paying that cost. Cases like that might be best handled by just executing the regular OpenSSH code as a subprocess.

[pyhedgehog]

Will using fake SSHClientConnection as a tunnel work? If yes, what methods should be implemented?

[ronf]

Hmm, that's an interesting idea. If you were to implement a class had a create_connection() method on it that took a protocol_factory, host, and port as arguments and returned a channel and transport, that could work. If you already had an open socket, your create_connection() call could probably just call loop.create_connection() on that socket. For instance:

import asyncio, asyncssh, socket

class SocketTunnel:
    """A wrapper which opens a socket you can run an SSH connection over"""

    async def create_connection(self, protocol_factory, host, port):
        """Return a channel and transport to run SSH over"""

        loop = asyncio.get_event_loop()

        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.connect((host, port))

        # Do other operations on sock here, like a SOCKS or HTTP CONNECT

        return (await loop.create_connection(protocol_factory, sock=sock))

async def run_client():
    async with asyncssh.connect('localhost', tunnel=SocketTunnel()) as conn:
        print((await conn.run('ls')).stdout, end='')

asyncio.get_event_loop().run_until_complete(run_client())

The one "gotcha" here is that you can't easily do async I/O on the socket you open before you return it, as you need to use the protocol_factory passed in by AsyncSSH in the loop.create_connection() call.

I suppose you might be able to play games with socket.fromfd(), where you open your own async socket transport and do some I/O for a while and then create a new socket object using fromfd() on the original socket's fileno(). Then, after you close your transport, you'd still be able to use the new socket object as an argument to the loop.create_connection() call that starts up the AsyncSSH transport.

[pyhedgehog]

I've got as least one running example:

import asyncio, asyncssh, aiosocks

class SocksFakeSSHClientConnection:
    def __init__(self, proxy, proxy_auth=None, host_map=None):
        self.proxy = proxy
        self.proxy_auth = proxy_auth
        self.host_map = host_map or {}

    @asyncio.coroutine
    def create_connection(self, session_factory, remote_host, remote_port, orig_host='', orig_port=0, *, encoding=None, window=2097152, max_pktsize=32768):
        dst = self.host_map.get((remote_host, remote_port), (self.host_map.get(remote_host, remote_host), remote_port))
        transport, protocol = yield from aiosocks.create_connection(session_factory, proxy=self.proxy, proxy_auth=self.proxy_auth, dst=dst)
        return transport, protocol

def connect(host, port=22, *, proxy=aiosocks.Socks4Addr('127.0.0.1',9050), proxy_auth=None, host_map=None, **kwargs):
    tunnel = SocksFakeSSHClientConnection(proxy, proxy_auth, host_map)
    return asyncssh.connect(host, port, tunnel=tunnel, **kwargs)

def test():
    fakehost = 'myhomecomp'
    username = 'myuser'
    torhost = 'abcdefghijklmnopq.onion'
    loop = asyncio.get_event_loop()
    loop.run_until_complete(connect(fakehost, username=username, host_map={fakehost:torhost}))
    loop.close()

I'll inform you when other samples will be ready.

[ronf]

Nice!

You don't actually need all the arguments you have on your create_connection() method. In order to build a class compatible with using your object as a value for "tunnel", you need only the session_factory, host, and port. None of the other args are ever passed in from asyncssh.create_connection().

I've been thinking that it might make sense to allow "tunnel=" to take a coroutine as an argument in addition to a class with a create_connection() method. That way, you could just pass in a callable or coroutine that did what your create_connection() method does here, without needing to wrap a fake connection class around it. You'd still be free to create a class if you needed to pass in other state like you are here, but in the simpler case where you don't need to pass any other arguments in, a simple coroutine could be used without a class.

I'll also probably consider allowing you to pass in a socket object as a "sock" argument similar to asyncio.create_connection(), in cases where you want to pass in an already established socket which you're ready to begin an SSH handshake on. However, unlike asyncio.create_connection(), it'll still be necessary to pass in a host & port argument in this case for the purposes of the known_hosts lookup.

[ronf]

After thinking about this a bit more, I'm not sure how useful it would be to pass in an existing socket to asyncssh.create_connection(). As noted above, it would be difficult to create such a connected socket from within an asyncio-based application without blocking the event loop or trying to extract the connected socket from an existing async transport object. So, I think I'll hold off on that change.

Also, while I could allow a coroutine to be passed in instead of an instance of a class which has a create_connection() method, the difficulty in passing custom arguments into such a coroutine might limit how useful that is., so perhaps it's cleaner to always take an object like it does now.

One other minor thing I noticed: In your example @pyhedgehog, you didn't decorate your connect() method as an asyncio coroutine. While it works without doing this, it'd be more correct to add the decorator for that and then change your "return" to do a "yield from" of the asyncssh.connect().