Programster's Blog

Tutorials focusing on Linux, programming, and open-source

KVM - Creating Thinly Provisioned Guests

KVM

When you create external snapshots, you create a read-only file that represents your virtual machine at a certain point in time. Since the data is read-only, we can use this file as a base for other guests we create in future. In order to operate, the guests will have all their changes go to a new empty file that we allocate them. Since we only need to create an empty qcow2 file for each guest, rather than copy the guest we are cloning, creating a guest takes seconds to perform (it was instant for me). Also, because the guests are all sharing the same base image, this saves a lot of storage and is called thin-provisioning.

An example diagram is shown below in which 3 guests are cloned from a base image, which is then updated, and a 4th guest is then cloned off the updated base image. With all 5 virtual machines, the storage needs is only about 4.4 GB (the size of the base image).

7Vzfc6o4FP5rnNl9aIcQQHy8dnvv7sOd2Znuj3sfI0TMFIkTYrX7128iAQlBxArYVu1D5RCS8J3vnHxJwBF8WG6/MbRafKchjke2FW5H8LeRbQPxEf+k5TWzjG0vM0SMhKrQ3vBE/sPKaCnrmoQ41QpySmNOVroxoEmCA67ZEGN0oxeb01hvdYUibBieAhSb1n9JyBeZ1c/vQtp/xyRa5C0Db5KdmaHgOWJ0naj2Rjac7z7Z6SXK61I3mi5QSDclE3wcwQdGKc++LbcPOJbY5rBl1309cLboN8MJb3OBnV3wguK1uvW/Z+uEr4UNePeWI/5/W+OUy+OR7aHlagSnySyV/8RxLBqZhuRFfI3kV/9e1va9OCUaLp1Vt8xfc5g53kr7gi9jYQDia8oZfcYPNKZMWBKaiJLTOYnjignFJErEYSDuEwv79AUzToQDv6gTSxKGspnpZkE4flqhQLa5EXSVFdKEf0VLEkt6/oNZiBKkzIqLwM6PVW+Vv2QzeHsQbFC4UIQGpkvM2asooi5wVSUqKHw3O9zsGTZRJRYlcuVXIcXpqKh371fxRbm23s2wvZvtm1vPcyvInTiAX532foU3v57nV2gNF6+u4VfDGXKUkeAwyhEnVIIoegOnYkjkiCQS5Z2LDgKhxk40y2u0TgVo4urEhybxaxGyYQcQee0h6hQS9xTO3Lk1kDh1kPgdQDI+Dom4QGioQ4FWiuxyEEsR4wc4CIyIF2dmvuu4kngRQyERMJWvyqSPSdJC9jS6owy71wg7gL5ORWDi7tXA7nWAut8f6iHC/rwWdS/w8Wx+KdQHQHXSI6ou9kOnDlXfnkHP6xO5C/Azb6qXtODKv1qC7j4DQFkk6SGgNPXWFAnYbMuQXX/h5SpGXJ77Yylnm8cnT7849zvBNv31WvXYkSRv6XrDNfUYgK7p+S4EGThBkQ2owICjQ+IPKTdACwn2YfUGaNZ5lxQcoEedd3HFcQj3IXDtU8kNoTkOYXcJjvao3wYRHUewHFJ11Kze3VRHl6tAR1L9BWVHzQLfJxpiG2G3faiPsDXLqr1FXAu193FH2Ga2H1eUvY0ZoAXqSfhF7u/JLBKjNCWBDra4S/b6Q4b+vZsf/txlAnGwJbx0Shz9zK8KI5znDMr4gkY0QfHj3joN1uwFhxrM8qJmkEXP6ZoFWI9ljliEeZlojWrHrUE7tzEskj150btR5wLVwp+UJLwUYkD39Z1d8WLWfXXV3pFmRZ5eUXV9N7tlox7hSfRaKraSBdKG/kK9GWNz80j5XADt+Zj1YM/OwgXt0kSblZUbYW+EfTeE7XHGPswM56Sxa0jJAOzrywXNy3V2RTeDmilTU644IS24lTCrhnPbtACqO5rV/HIgL7wlFK9Q7Nz4cgZf+tzcHWRB5aTUDa0BU7e5vvLpQ3Hc7I0WT3a881BsK+neEopXONTf+HIGX9rsK7xjvrSbE560tlcd6mt2Jrrhy7jyPBaw3soXu1JTtS+fdcrYZiPnI5O3eKS/LXk9nQhOX9QtgiL3bLWKttR16zvcfaLLU+in5cr4VK5UoK95GvR9c8Wop0OymBPg267mYM+2G47Oh5EhdjVhGwF9HbuazoDzXNjpPFdLzkVC/pGXU8k5O/PGfJvzpD28EOjw9phxPf0BMKc6rLbeigB6RXDSX8o1nyZYr0KRWNNrT4fFclPD8lNv7+b1+LDBfI69+mwYjiczy7rAyt6gGc/c7zr4fpxzez/uPHFb59jeYuYzv+p0moJwbVO59RZPF1vJaqEgcrWgKYizl6ac3pYyq3ND8EYBASpjl1P1dIcCotPFoI4VZK3/T56xQ7uiIL2+/H+wpVMJUK3IqTKpOwLkIrdEAIZReEeTWGKYJmiVLqj6tYjrHh2h8TDKcK+PO+baygE3mfF75W5yBnzL3zHXQYI12/XctkKSPsuy2YKXtWFkN0+zIipzHL2/OU6fN7s1A3VHjhOH+1/kybLm/meP4OP/">
Ubuntu 16.04 Guest 1
8.2 MB
[Not supported by viewer]
Ubuntu 16.04 Guest 2
8.2 MB
[Not supported by viewer]
Ubuntu 16.04 Guest 3
8.2 MB
[Not supported by viewer]
Base Ubuntu 16.04 Template Image
(4.4 GB)
[Not supported by viewer]
updates
updates
Ubuntu 16.04 Guest 4
8.2 MB
[Not supported by viewer]
read-only snapshot 1
read-only snapshot 1
read-only snapshot 2
read-only snapshot 2
current disk image writes go to.
current disk image writes go to.

Below is a script I created for creating thinly provisioned guest called "new-guest" from a base image called "template-ubuntu-16.04-docker". Just tweak the variables at the top to change it to your needs.

#!/bin/bash

# Specify settings here.
NEW_GUEST_DOMAIN="new-guest"
BASE_GUEST_DOMAIN="template-ubuntu-16.04-docker"
BASE_GUEST_DISK="/vms/base-images/ubuntu-16.04.qcow2"
NEW_GUEST_FOLDER="/vms/$NEW_GUEST_DOMAIN"
NEW_GUEST_DISK="`echo $NEW_GUEST_FOLDER`/`echo $NEW_GUEST_DOMAIN`.qcow2"
NEW_GUEST_XML_PATH="`echo $NEW_GUEST_FOLDER`/`echo $NEW_GUEST_DOMAIN`.xml"

# Create a directory for anything related 
# to our guest such as snapshots
mkdir -p $NEW_GUEST_FOLDER

# Create a qcow2 disk file for our new guest.
sudo qemu-img create \
  -f qcow2 \
  -b $BASE_GUEST_DISK \
  $NEW_GUEST_DISK

# Generate a new xml file for the guest.
# This takes care of changing the MAC address for us.
sudo virt-clone \
  --original $BASE_GUEST_DOMAIN \
  --name $NEW_GUEST_DOMAIN \
  --file=$NEW_GUEST_DISK \
  --preserve-data \
  --print-xml > $NEW_GUEST_XML_PATH

# Use the xml file to define the new guest
sudo virsh define $NEW_GUEST_XML_PATH
sudo rm $NEW_GUEST_XML_PATH

After creating a guest, it's a good idea to start it up and run the following commands from inside the guest to give it a new unique identity that is different from the machine it was cloned from.

sudo /bin/rm -v /etc/ssh/ssh_host_*
sudo dpkg-reconfigure openssh-server

Taking It Further

In a future post, I plan to see if I can live migrate guests without shared storage, and see if using external snapshots can be useful in the process.

References