KVM - Offline Migration

You are unlikely to be able to migrate between Intel and AMD servers. Refer here. You can change server distribution with an extra step.

1. (Optional) Save the guest.

If the machine is currently running and you can't just shut it down for the migration, then save the machine's state which we will use later.

virsh save $GUEST_ID $FILENAME

e.g.

virsh save dns.technostu.com dns.technostu.com.state

Note: I just chose the state suffix to makes sense for me. The img suffix is already being used for the disk image. You can use any suffix you like.

2. Grab the Config

GUEST_ID=my-vm-id
virsh dumpxml --migratable $GUEST_ID > $GUEST_ID.xml

e.g.

virsh dumpxml --migratable dns.programster.org > dns.programster.org.xml

Moving Data: If you are not using shared storage between the two hosts, and are actually moving the data of the guest as well, you will need to update the source file part of the xml file to point to where the .img will now be.

Networking: Your KVM guest will be configured exactly the same, including any network bridges etc. Thus you may need to either alter the xml file, or ensure that the host you are transferring to has the same bridge setup etc.

3. Copy the XML file to the new host.

4. Create a guest with the guest.xml file

virsh create $GUEST_ID.xml

e.g.

virsh create dns.programster.org.xml

If you get any error responses at this point, please refer to the debugging section down below.

Ubuntu users

When trying to perform the previous step, you may run into permission issues, if that is the case then try running the following commands from this posted workaround.

sudo apt-get install apparmor-profiles apparmor-utils -y
sudo aa-complain /usr/lib/libvirt/virt-aa-helper

5. Optional Define the Guest

You need to "define" the guest on the new system if you want to be able to:

• start the guest by name/ID, and not by using the xml file.
• See guests as "offline" when they are not running, rather than not being listed at all, with the command sudo virsh list --all.

sudo virsh define $GUEST_ID.xml

6. Start The Guest

If you didn't save the machine state, then you can simply run the command sudo virsh start [guest-id] or sudo virsh start [guest-id].xml if you didnt bother to define the guest.

If you did have a running guest, then run the following command to restore the guest from save.

virsh restore $GUEST_ID.state

e.g.

virsh restore dns.programster.org.state

Debugging

Migrating Across Distros

If you are migrating from one distro to another, you may get a message like below:

error: internal error: process exited while connecting to monitor: 2017-08-22T10:00:32.179446Z qemu-system-x86_64: -msg timestamp=on: unsupported machine type
Use -machine help to list supported machines

Luckily this is easy to resolve. All you need to do is edit your xml definition file and change the OS section to reflect the change. For example, when I was migrating from ubuntu yakkety to xenial (16.10 to 16.04), I needed to change:

  <os>
    <type arch='x86_64' machine='pc-i440fx-yakkety'>hvm</type>
    <boot dev='hd'/>
  </os>

to

  <os>
    <type arch='x86_64' machine='pc-i440fx-xenial'>hvm</type>
    <boot dev='hd'/>
  </os>

If you have having no luck, you can try using the the genric value of pc instead, or look up all the possible machine types your KVM hypervisor supports with:

kvm -M ?

... and then pic the one that is closest to the one you are replacing.

Migrating Across Distro 2

I got the following error message when migrating from Ubuntu16.04 to Debian 9:

error: Cannot check QEMU binary /usr/bin/kvm-spice: No such file or directory

Simply change:

<emulator>/usr/bin/kvm-spice</emulator>

to:

<emulator>/usr/bin/kvm</emulator>

Migrating CPU (Even Intel <--> AMD)

If you get the message:

Host CPU does not provide required features ...

Then look in your configuration file and find something like:

<os>
    <type arch='x86_64' machine='pc-i440fx-xenial'>hvm</type>

... and just change it to pc:

<os>
    <type arch='x86_64' machine='pc'>hvm</type>

Alternatively, if that doesn't work, you might have something like:

  <cpu mode='custom' match='exact' check='partial'>                                    
    <model fallback='forbid'>Opteron_G5</model>                                        
  </cpu>

In which case, change this to:

  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>

Apparmor Label

If you get the following error message:

error: unsupported configuration: Unable to find security driver for model apparmor

... then look at your configuration and search for apparmor. You will probably find the following line:

<seclabel type='dynamic' model='apparmor' relabel='yes'/>

Just delete it and you're sorted.

Notice - Migrating to ZFS

Please be aware that when I was migrating my guests from an ext4 mdadm RAID system to ZFS, I had to remove the cache=none from the xml configs for the disk settings before the machines could start.

References

• IBM Knowledge Center - Migrating a KVM guest offline with dumpxml
• CentOS Docs - Chapter 20 Managing Guests with Virsh
• Ask Ubuntu - Ubuntu 14.04 KVM - error: internal error...
• ServerFault - Unable to import Ubuntu virtual machine on Centos 7 kvm host
• superuser - kvm Error starting domain: unsupported configuration: Unable to find security driver for model selinux