Creating Full ZFS Clones in Proxmox

Introduction

I absolutely love snapshots. They have saved my bacon on more than one occasion, and it's great to be able to go back to any point in time that one took a snapshot for. Unfortunately, when one uses Proxmox to clone a virtual machine, the snapshots aren't copied across. If you wish to retain those snapshots in the clone, then you can do this by manually performing actions in the CLI, which this tutorial will show you how to do.

This is also particularly useful for working around the fact that rolling back snapshots in ZFS also "loses" any snapshots that were created after the restored snapshot, and that Proxmox doesn't appear to be able to create clones from snapshots that aren't the latest. Thus if one is not sure which snapshot one needs, one can use this method to create a clone and go back one snapshot at a time, knowing that the original VM is still there with all the snapshots retained.

Steps

For this tutorial, I created a virtual machine (VM), with ID 905, for which I created three "stateful" (memory included) snapshots that I called snapshot1, snapshot2, and snapshot3.

Clone The Disk

The first thing we need to do is create a full copy of the original "disk", including all of it's snapshots. This can be done through the use of ZFS send and recieve as shown below:

# Set the VM IDs as yours are likely to be different.
SOURCE_VM_ID=905
DEST_VM_ID=906

zfs send -R \
  rpool/data/vm-$SOURCE_VM_ID-disk-0 \
  | zfs receive -F rpool/data/vm-$DEST_VM_ID-disk-0

This needs to be run as root, and the VM we are cloning from must not be running because we need the dataset to not be mounted.

• The -R flag tells ZFS to perform a full replication stream, which cuases all of the snapshots to be sent without us having to perform incremental send/recieve for each one.

• The -F forces zfs to overwrite the destination, which already exists (from us cloning the VM).

Copy Snapshot Memory States

The previous command copied the disk with all of its snapshots, however, when we took the snapshots, we told proxmox to store that state as well. Proxmox keeps the state in additional (separate) ZFS datasets that use a naming convention so that one can easily tell which virtual machine they belong to.

Since my snapshots were called snapshot1, snapshot2, and snapshot3, the datasets representing the dataset's memory state at that point in time are called:

• vm-$DEST_VM_ID-disk-0-state-snapshot1
• vm-$DEST_VM_ID-disk-0-state-snapshot2
• vm-$DEST_VM_ID-disk-0-state-snapshot3

Thus, I am going to run the three commands below in order to copy them overt for the clone:

zfs send \
  rpool/data/vm-$SOURCE_VM_ID-state-snapshot1 \
  | zfs receive rpool/data/vm-$DEST_VM_ID-disk-0-state-snapshot1

zfs send \
  rpool/data/vm-$SOURCE_VM_ID-state-snapshot2 \
  | zfs receive rpool/data/vm-$DEST_VM_ID-state-snapshot2

zfs send \
  rpool/data/vm-$SOURCE_VM_ID-state-snapshot3 \
  | zfs receive rpool/data/vm-$DEST_VM_ID-state-snapshot3 

Now we have our snapshots, but they wont show up in the proxmox GUI. To resolve this we need to manually create or update the clone's Proxmox config file, in order to specify to tell Proxmox about the snapshots. This is easily done by copying the configuration file for the original VM over to the clone like so:

cp \
  /etc/pve/qemu-server/$SOURCE_VM_ID.conf \
  /etc/pve/qemu-server/$DEST_VM_ID.conf

... and simply updating the clone's configration file content to specify the clone's virtual machine ID rather than the one we copied from. We can do this by running a search and replace operation changing:

vm-$SOURCE_VM_ID

to:

vm-$DEST_VM_ID

This search and replace operation can be achieved with the sed tool by running the following set of commands:

SEARCH="vm-$SOURCE_VM_ID"
REPLACE="vm-$DEST_VM_ID"
FILEPATH="/etc/pve/qemu-server/$DEST_VM_ID.conf"

sed -i "s;$SEARCH;$REPLACE;" $FILEPATH

We have now created our ZFS clone with the corresponding snapshots. This is a "full clone" that is not tied to the original in any way. Rolling back or deleting a snapshot on the parent or the original will have no impact on the other.

Possible Further Manual Actions - Too Much Of A Clone?

This steps above created a absolute copy of the original virtual machine. At first that sounds like exactly what you want, but actually people usually prefer the additional default behaviours that Proxmox usually automatically performs for you when creating a clone, such as changing the MAC address, virtual machine ID etc. You will need to perform any such actions manually.

References

• Serverfault - zfs send multiple snapshots