The best SSH library is ssh
Nearhost opens a dev server running on another machine at a local https://myapp.test. Everything it does on that machine travels over SSH: the port forwards, the file reads that discover a project's ports, and the socket a coding agent uses to reach it. So one of the first decisions was how a Go program should speak SSH.
Go makes the answer look obvious. golang.org/x/crypto/ssh is mature and well maintained, and it has forwarding built in. A library keeps everything in one process, gives you typed errors and spares you from parsing another program's output. I didn't use it. Nearhost starts your own ssh binary and talks to it, and the whole contract with the user fits in one line of the docs: if ssh <host> works without asking you anything, Nearhost works.
Your ssh config is a program
Open the ~/.ssh/config of anyone who has used SSH for a few years and you'll find a small program:
Include ~/.ssh/config.d/*
Host mini
HostName mini.local
User ecs
IdentityAgent "~/Library/Group Containers/2BUA8C4S2C.com.1password/t/agent.sock"
Host prod-*
ProxyJump bastion
CertificateFile ~/.ssh/prod-cert.pub
Match host *.corp exec "corp-vpn-up"
GSSAPIAuthentication yesIt pulls in other files by glob, and a Match block runs a shell command to decide whether it applies. One host jumps through another. Keys sit in 1Password, Secretive or a YubiKey behind an agent socket, and on a Mac UseKeychain holds the passphrases. Some hosts want a signed certificate, and some want Kerberos.
A library implements the SSH protocol, and your config describes OpenSSH's client, which is a much bigger thing. To honour that file I would have had to rebuild ssh_config evaluation, agent discovery, jump-host chaining and known-hosts handling, then chase OpenSSH as it changed. Every gap would surface as a user whose ssh mini works and whose Nearhost doesn't. That's the worst bug report a tool like this can get, because the user has already proved the machine is reachable.
Running ssh turns that whole class of bug into OpenSSH's problem, and OpenSSH has had a lot of practice. Nearhost stores one thing per server, the host string you'd type after ssh, and keeps no keys, passwords or ports.
Nearhost does parse your ssh config, using kevinburke/ssh_config, to list your Host aliases in the add-server form. It skips wildcards and Match blocks, and the list never decides how a connection gets made. If the parser misreads your config, you get a wrong suggestion in a form and type the host yourself.
One master per server
OpenSSH can multiplex many sessions over one connection. You start a master that owns the TCP connection and a control socket, and later ssh invocations talk to the master through the socket instead of logging in again. Nearhost runs one master per server, in the foreground, under a supervisor:
ssh -N -M -S ~/.config/nearhost/run/mini.sock \
-o BatchMode=yes \
-o ExitOnForwardFailure=yes \
-o ServerAliveInterval=15 -o ServerAliveCountMax=3 \
-o ControlPersist=no \
-o ClearAllForwardings=yes \
-o RemoteCommand=none \
-- mini-N runs no command and -M makes it a master. BatchMode=yes matters most. It stops ssh from prompting, because a prompt would print over the full-screen interface that owns the terminal. When a host needs a passphrase or a host key confirmation, the master fails with OpenSSH's own error, and Nearhost shows you that error with the ssh command to run once by hand.
With the master up, everything else is a short request over its socket. Serving a project adds a forward to the running connection, and stopping it takes the forward away:
ssh -S mini.sock -O forward -L 127.0.0.1:52731:localhost:80 -- mini
ssh -S mini.sock -O cancel -L 127.0.0.1:52731:localhost:80 -- mini
ssh -S mini.sock -O check -- miniThe master stays connected throughout, so the other projects on that server keep their connections. -O check answers whether the master is alive, and the supervisor polls it after launch to know when the socket is ready.
File reads ride the same connection. Discovery reads a project's .env, Compose file and Vite config to propose its routes, and it does that over SFTP. Nearhost starts ssh's SFTP subsystem through the master and hands the process's pipes to pkg/sftp, which speaks the protocol on top:
cmd := s.longLived(context.Background(), "-S", s.SocketPath(), "-o", "BatchMode=yes",
"-o", "RemoteCommand=none",
"-o", "ClearAllForwardings=yes",
"-o", "ControlMaster=no",
"-s", "--", s.Host, "sftp")
stdin, _ := cmd.StdinPipe()
stdout, _ := cmd.StdoutPipe()
cmd.Start()
client, err := sftp.NewClientPipe(stdout, stdin)That's the split I'd recommend to anyone in the same position. OpenSSH owns authentication and the connection, and a library speaks a protocol over a pipe OpenSSH hands you. Nearhost never runs a shell command on the server, and you install nothing there.
The coding agent support works the same way, in reverse. Each master gets a -R forward of a Unix socket, so ~/.nearhost/<id>.sock on the server leads back to Nearhost on your laptop. An agent working on the server talks to that socket to propose a project, and Nearhost opens the add form for you to accept. That directory is the only thing Nearhost creates on a server.
The parts that bite
Shelling out moved the hard work somewhere else. A library makes you reimplement OpenSSH. Running OpenSSH makes you live with its behaviour, and I didn't see most of that coming.
Your config applies to Nearhost's ssh too
The feature I wanted, OpenSSH honouring your whole config, applies to every ssh Nearhost starts. Your config can carry options that break Nearhost's job:
- A
LocalForwardfor the host. The master would try to bind that port, and withExitOnForwardFailure=yesa port already in use kills the connection.ClearAllForwardings=yesdrops them, and Nearhost loses nothing, because it asks for its own forwards later over the socket. - A
RemoteCommand. ssh refuses to combine one with a subsystem like SFTP.RemoteCommand=noneclears it. ControlMaster autowithControlPersist, the common setup for fast logins.ControlPersistwould send Nearhost's master into the background, out of reach of its supervisor, so the master getsControlPersist=noand the SFTP client getsControlMaster=no.
The command line beats the config file, so each invocation switches off what would change its job. You can't apply that fix to the control commands, though. ClearAllForwardings on an -O forward would drop the very -L it names. And a mux client sends the master every LocalForward and RemoteForward the config lists for the host, along with the one you asked for. So control commands skip the config:
// control is command for one request to the running master over its socket:
// -O check, forward, cancel or exit. It reads no configuration file.
func (s *Session) control(ctx context.Context, args ...string) *exec.Cmd {
return s.command(ctx, append([]string{"-F", "/dev/null", "-S", s.SocketPath()}, args...)...)
}They need the socket and nothing else. -F /dev/null with -S gives them that, and the host on the end serves as a label.
The health check has the reverse problem. If you already have a master open to mini, a plain ssh mini true borrows it and succeeds, which tells you nothing about whether Nearhost's own master can log in. So the check sets ControlMaster=no and ControlPath=none and makes a connection of its own.
A host is an argument
Nearhost passes the host string to ssh as an argument, and a host that starts with a dash is an option. A config edited by hand, or a value pasted from somewhere else, could name a server -oProxyCommand=... and run a command the moment Nearhost connected. The config now refuses an ssh_host that starts with -. Every invocation also puts -- before the host, so ssh stops reading options there whatever the config says.
Unix sockets have a length limit
A Unix socket path fits in 104 bytes on macOS and 108 on Linux, counting the final NUL. OpenSSH binds the control socket at its path plus a 17-byte temporary suffix and then renames it, so the usable budget is smaller than it looks:
const (
socketAddrSize = 104
sshBindSuffix = 17
maxSocketPath = socketAddrSize - sshBindSuffix - 1
hashedNameChars = 18 // "h-" and 16 hex digits
)A server name that would push the path over that limit gets a short hash instead. Names with path separators or % get one too, because ssh expands % tokens in a control path and a / could put the socket outside the run directory. If the run directory alone is too long, Nearhost refuses to start the master and tells you to set a shorter XDG_CONFIG_HOME. Without that check, you'd get an error from deep inside ssh about a socket you never asked for.
The agent socket on the server has the same limit with nothing Nearhost can shorten, since the user's home directory sets most of the path. When sshd refuses the forward and the path runs over 103 bytes, Nearhost names the length as the reason. The mux client only says the forward failed.
Ctrl-C reaches everyone
A terminal sends Ctrl-C to the whole foreground process group, which includes every ssh Nearhost started. The interface wants to catch that, tell each master to exit and close the forwards in order. Children dying at the same instant make that impossible. So the two long-lived processes, the master and SFTP, start in a process group of their own and end when Nearhost ends them. nearhost add does the opposite. It handles no signals and dies on Ctrl-C, so the ssh it starts to read a project stays in its group and dies with it. A master in a group of its own would outlive it.
Errors are strings
ssh reports failure as an exit code and some text on stderr. The supervisor reconnects with backoff from one second up to thirty, but some failures won't go away by waiting, and it has to tell those apart by reading:
for _, needle := range []string{
"Permission denied",
"Host key verification failed",
"REMOTE HOST IDENTIFICATION HAS CHANGED",
"Too many authentication failures",
} {After three of those in a row it stops trying, and shows the message and the fix. I don't love matching strings in another program's output. In exchange, you see OpenSSH's own words, and you can paste them into a search engine.
Reconnects forget forwards
When a master dies and the supervisor starts a new one, the new master has none of the forwards. Nearhost keeps the forwards it wants in a table, each stamped with the launch it was applied on. After a reconnect, it re-applies every forward whose stamp is older than the current launch, and only then reports the server connected. Anyone who sees "connected" sees every forward settled, and a forward that failed to re-apply shows as failed instead of looking fine.
What it costs
Nearhost needs an OpenSSH client on your laptop. Every forward change goes through a subprocess and costs a fork, and errors arrive as text. The transport tests run a fake ssh, a shell script that records its arguments and acts out the flags Nearhost uses. Those tests check the commands Nearhost sends, and they never see a real handshake.
In return, Nearhost has no authentication code. It holds no keys, so it can't leak any. It never asks for an SSH password or a key's passphrase, and it never prompts about host keys. The only password it asks for is your laptop's, when sudo needs it to update /etc/hosts or trust the local CA. The SSH setup you've built over the years works unchanged: the jump host your company requires, the hardware key you tap to log in, and the parts I've never heard of.
Many of the comments in the transport code explain which option undoes which part of your config. Each one marks a feature I'd otherwise have built into a library and kept in step with OpenSSH.