Configure KVM Private NAT With Guest Gateway
This tutorial arose out of necessity. Hetzner only allows you a certain amount of "public" IPs to each of your dedicated servers. This means that if you follow the simple/traditional route of giving each of your guests their own public IP (like if you used DigitalOcean hosting etc), then you will quickly run out of IPs and won' be able to host very many services on your server.
What we need to do is create a private network for the majority of our guests to use, and set up a single guest as a gateway that they all connect to, in order to connect with the outside world. We will also be setting up a reverse proxy and port forwarding rules on this gateway so that our guests can be reached from the outside world, not just the other way around.
Create Private Network
The first thing we need to do is set up a private internal network that all of the guests will be connected to such that they can communicate with each other. This is unlike the default KVM network, which does not allow KVM guests to communicate with each other. This network will use the standard private IP address set, so we don't need to worry about running out of IP addresses.
Add/append this to your KVM host's network/interfaces file (don't change any existing blocks):
auto privatebr0 iface privatebr0 inet static address 192.168.0.1 netmask 255.255.0.0 pre-up brctl addbr privatebr0 post-down brctl delbr privatebr0
Now use virsh to define the private network that relates to the block we just added to the interfaces file:
echo '<network> <name>privatenet</name> <bridge name="privatebr0" /> </network>' >> /tmp/net.xml virsh net-define /tmp/net.xml virsh net-start privatenet virsh net-autostart privatenet
Now if you run
virsh net-list --all
You should see the new network named
Name State Autostart Persistent ----------------------------------------------- default active yes yes privatenet active yes yes
Create NAT/Gateway Guest
Next, we need to create the NAT/gateway guest.
If you haven't already deployed a Debian guest to fulfill this role, do so now.
I will assume that they have already been set up against a normal bridged setup with a public IP address.
Run the following command to add a network interface to the guest that is connected to this new private network:
virsh attach-interface \ --domain $GUEST_ID \ --type network \ --source privatenet \ --model virtio \ --config \ --live
Edit the guest in order to see the mac addresses that are assigned to it. If you need to manually set one to a whitelisted MAC address, now is the time to do it.
sudo virsh edit $GUEST_ID
Log into the guest
sudo virsh console $GUEST_ID
Use the following command to get details of your interfaces:
Make a note of the MAC address to interface name pairs. We need to make sure we use the name of the interface that is connected to the whitelisted MAC address, for the public IP, and the other interface name for the internal private network that the other KVM guests will be on.
Now, using the names of the networks, set your
/etc/network/interfaces file accordingly. Here is mine:
source-directory /etc/network/interfaces.d auto enp0s2 allow-hotplug enp0s2 # Interface for public IP that can access the outside world iface enp0s2 inet static address xxx.xxx.xxx.xxx netmask 255.255.255.255 pointopoint xxx.xxx.xxx.xxx gateway xxx.xxx.xxx.xxx dns-nameservers 18.104.22.168 22.214.171.124 # internal private network interface auto enp1s0 iface enp1s0 inet static address 192.168.0.1 netmask 255.255.0.0 gateway 192.168.0.1 dns-nameservers 126.96.36.199 188.8.131.52
We now need to configure the guest to allow forwarding of packets from the guests on the private network, as if they were this server, so they can get through the MAC address filtering.
First we need to tell your server to accept forwarding:
echo 1 > /proc/sys/net/ipv4/ip_forward sudo sysctl -p
...and uncomment this line so there is no
Now we need to set up iptables rules so that the server will forward packets. Whenever you wish to allow the outside world to gain access to one of your servers, you will need to tweak these rules to forward the relevant port. The only case where you wont need to do this is for websites with ports 80/443 because we will also set this server up as a reverse proxy later.
Create a file with the following contents (tweaking the variables as necessary), and save it somewhere like
#!/bin/bash # Specify the name of the interface that has access to the outside world # the one with your public IP PUBLIC_INTERFACE="enp0s2" # Specify the name of the interface that is connected to the internal private # LAN. INTERNAL_INTERFACE="enp1s0" # Flushing all rules /sbin/iptables -F /sbin/iptables -X # Setting default filter policy /sbin/iptables -P INPUT ACCEPT /sbin/iptables -P OUTPUT ACCEPT /sbin/iptables -P FORWARD ACCEPT # Allow unlimited traffic on loopback /sbin/iptables -A INPUT -i lo -j ACCEPT /sbin/iptables -A OUTPUT -o lo -j ACCEPT /sbin/iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT # Allow new incoming SSH connections from anywhere for this server /sbin/iptables -I INPUT -p tcp --dport 22 -j ACCEPT # Allow all new incoming connections from our internal network /sbin/iptables -I INPUT -p tcp --in-interface $INTERNAL_INTERFACE -j ACCEPT # Example for rorwarding ftp requests #HOST="192.168.0.1" #DEST_SERVER="192.168.0.2" #/sbin/iptables -t nat -A PREROUTING -d $HOST/32 -p tcp --dport 20 -j DNAT --to-destination $DEST_SERVER:20 #/sbin/iptables -t nat -A PREROUTING -d $HOST/32 -p tcp --dport 21 -j DNAT --to-destination $DEST_SERVER:21 #/sbin/iptables -A FORWARD -p tcp -d $DEST_SERVER --dport 20 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT #/sbin/iptables -A FORWARD -p tcp -d $DEST_SERVER --dport 21 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT # All packets being forwarded should look like they are coming from this server. sudo iptables --table nat --append POSTROUTING --out-interface $PUBLIC_INTERFACE -j MASQUERADE # Accept the forwarding of all packets that came in on the internal internal # private network for KVM guests. sudo iptables --append FORWARD --in-interface $INTERNAL_INTERFACE -j ACCEPT
Make sure that file gets executed on boot, either by adding the relevant cron (e.g.
@reboot), or adding to your
/etc/rc.local file etc.
Finally, port forwarding only helps us when we wish to provide services that use different ports. However, I find that most services are just websites that all want to use ports 80 and port 443. Thus we need to set up a reverse proxy. Luckily I already have a post for that, which you can use.
Make sure that your other guests have a single network interface, and they are configured to use the KVM private network we set up earlier.
sudo virsh edit $GUEST_ID
<interface type='network'> <mac address='52:xx:xx:xx:xx:xx'/> <source network='privatenet'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface>
Then be sure to assign them private static IPs. You will need to keep track of which IPs are already being used and assigned to which server. This is what you will need to set up your reverse proxy and port-forwarding rules against, otherwise we would have set up DHCP.
Below is an example
/etc/network/interfaces file of such a guest:
# Include files from /etc/network/interfaces.d: source-directory /etc/network/interfaces.d auto enp1s0 iface enp1s0 inet static address 192.168.0.8 netmask 255.255.0.0 gateway 192.168.0.1 dns-nameservers 184.108.40.206 220.127.116.11
... and below is a netplan configuration for that guest (in case you are using Ubuntu etc)
# This file describes the network interfaces available on your system # For more information, see netplan(5). network: version: 2 renderer: networkd ethernets: ens6: addresses: - 192.168.0.2/16 gateway4: 192.168.0.1 nameservers: addresses: - 18.104.22.168 - 22.214.171.124
IPTables Reset Rules
If you ever mess up when setting up iptables, it's good to have this to hand to perform a reset.
/usr/sbin/iptables --policy INPUT ACCEPT; /usr/sbin/iptables --policy OUPUT ACCEPT; /usr/sbin/iptables --policy FORWARD ACCEPT; /usr/sbin/iptables -Z; /usr/sbin/iptables -F; /usr/sbin/iptables -X;
First published: 19th July 2021