SSH Jumping With Jump Hosts

Introduction

I run lots of virtual machines in a Proxmox host on Hetzner. I do this with them sitting behind one of them that acts as a NAT. Thus, to be able to SSH into them directly I had created a GitLab pipeline for updating the NAT with port forwarding rules easily, so that I could SSH into each machine by using a custom port. One could have manually connected to the NAT and then manually connected to the VPS from there over the private IP, without creating these port-forwarding rules, but that would have been very time-consuming.

This tutorial will show you how to use the "jump hosts" parameter/option in SSH, SCP, and SSHFS, in order to be able to do this automatically as the middle ground. Thus, one does not need to create a port forwarding rule for each host, and one does not need to manually connect from the NAT.

Steps

Update/Create SSH Config File

The first thing we need to do is plug all of the details for each of the servers into our SSH config file. I am going to call them "gateway" and "destination" for simplification purposes of indicating their role in this tutorial, but you would likely have different names.

For this tutorial, I am going to create a temporary SSH configuration file for this, as I will actually be updating my SSH management tool to create this dynamically on the fly, and I do not wish to pollute my existing default SSH config file, but you could just enter the details in your $HOME/.ssh/config file.

Host gateway
    HostName 123.45.67.89
    User programster
    port 2222
    IdentityFile /path/to/my/keys/2025-programster

Host destination
    HostName 192.168.0.3
    User programster
    port 22
    IdentityFile /path/to/my/keys/2024-programster

The HostName of the destination server should be the IP address as it appears from the gateway server. E.g. this will likely be your internal LAN/private IP address which is going to be something like 192.168.x.x or 10.x.x.x. if on something like AWS.

Now all you need to do is run the following command from your terminal, in order to connect to the underlying server:

ssh -J gateway destination

If you are using a temporary/custom config file like I am, then one just needs to provide its filepath after using the -F parameter like so:

ssh -F temp-config -J stuart@gateway stuart@destination

SCP

If wishing to use scp to copy files through a jumphost, then it is exactly the same as before, with simply using the additional `-J parameter:

GATEWAY_SERVER="gateway"
DESTINATION_SERVER="destination"

scp \
  -F temp-config \
  -J $GATEWAY_SERVER:/path/to/file \
  $DESTINATION_SERVER:/path/to/local/file

Entering Passphrases

This resulted in me having to enter each of my keys passphrases which was annoying. To work around this, I just added the keys to my ssh agent with:

ssh-add /path/to/my/keys/2025-programster
ssh-add /path/to/my/keys/2024-programster

SSHFS

If one wishes to mount the underlying VPS with SSHFS through the jump host, then one can do so using -o ProxyJump=gateway like so:

sshfs \
  -F $SSH_CONFIG_FILEPATH \
  -o ProxyJump=$GATEWAY_SERVER_NAME \
  $UNDERLYING_SERVER_NAME:$REMOTE_FOLDER \
  $LOCAL_MOUNT_POINT