OverlayFS

OverlayFS provides a great way to merge directories or filesystems such that one of the filesystems (called the "lower" one) never gets written to, but all changes are made to the "upper" one. This is great for systems that rely on having read-only data, but the view needs to be editable, such as Live CD's and Docker containers/images (image is read only). This also allows you to quickly add some storage to an existing filesystem that is running out of space, without having to alter any structures. It could also be a useful component of a backup/snapshot system.

[Image courtesy of Docker Docs]

To get a much better understanding of what overlayFS is and how it works, please read this article provided by Docker. The rest of this tutorial will focus on using OverlayFS.

Simplest Case - Directories Only

The command below creates directories for the lower filesystem, lower filesystem and the merged result which goes into "overlay" (but you may wish to rename to "merged").

cd /tmp
mkdir lower upper workdir overlay

sudo mount -t overlay -o \
  lowerdir=/tmp/lower,\
  upperdir=/tmp/upper,\
  workdir=/tmp/workdir \
  none /tmp/overlay

• When you create files in lower or upper, they will appear in the overlay folder.
• Any files you create in the overlay folder are only created in the upper dir.
• Changes you make to existing files in the overlay folder only appear in the upper dir.
• If you delete a file in the overlay folder that was in the lower dir, then a "whiteout" is created so that it does not appear in the overlay folder, but still remains in the lower dir. The same applies to folders.

Combining with A Lower Filesystem

This example demonstrates how we can throw a read-only filesystem into the mix by extending the above example:

cd /tmp
cd /tmp

# Create the necessary directories.
mkdir lower upper workdir overlay

# Lets create a fake block device to hold our "lower" filesystem
dd if=/dev/zero of=lower-fs.img bs=4096 count=102400

# Give this block device an ext4 filesystem.
mkfs -t ext4 lower-fs.img

# Mount the filesystem we just created and give it a file
sudo mount lower-fs.img /tmp/lower
sudo chown $USER:$USER /tmp/lower
echo "hello world" >> /tmp/lower/lower-file.txt

# Remount the lower filesystem as read only just for giggles
sudo mount -o remount,ro lower-fs.img /tmp/lower

# Create our overlayfs mount
sudo mount -t overlay -o \
  lowerdir=/tmp/lower,\
  upperdir=/tmp/upper,\
  workdir=/tmp/workdir \
  none /tmp/overlay

Now you can still write to lower-file.txt in the overlay filesystem even though it only appears because it is in the lower filesystem. This is because any changes you make are actually made in the upper filesystem.

Combining with An Upper Filesystem

This example further extends the example above by making the upper filesystem another filesystem on its own virtual block device.

cd /tmp
cd /tmp

# Create the necessary directories.
mkdir lower upper overlay

# Lets create a fake block device to hold our "lower" filesystem
dd if=/dev/zero of=lower-fs.img bs=4096 count=102400
dd if=/dev/zero of=upper-fs.img bs=4096 count=102400

# Give this block device an ext4 filesystem.
mkfs -t ext4 lower-fs.img
mkfs -t ext4 upper-fs.img

# Mount the filesystem we just created and give it a file
sudo mount lower-fs.img /tmp/lower
sudo chown $USER:$USER /tmp/lower
echo "hello world" >> /tmp/lower/lower-file.txt

# Remount the lower filesystem as read only just for giggles
sudo mount -o remount,ro lower-fs.img /tmp/lower

# Mount the upper filesystem
sudo mount upper-fs.img /tmp/upper
sudo chown $USER:$USER /tmp/upper

# Create the workdir in the upper filesystem and the 
# directory in the upper filesystem that will act as the upper
# directory (they both have to be in the same filesystem)
mkdir /tmp/upper/upper
mkdir /tmp/upper/workdir

# Create our overlayfs mount
sudo mount -t overlay -o \
  lowerdir=/tmp/lower,\
  upperdir=/tmp/upper/upper,\
  workdir=/tmp/upper/workdir \
  none /tmp/overlay

Pay careful attention to the changes we had to make to the workdir in this example. The workdir has to be a directory in the upper filesystem so now /tmp/upper is the mount point instead of the upper directory, and /tmp/upper/upper is the upper directory, and /tmp/upper/workdir is the new location of the workdir.

Nesting OverlayFS

You can nest OverlayFS mounts, but it would probably be better just to specify multiple lower directories in one go using : as a separator. The lower directories are layered from right to left.

cd /tmp
sudo mount -t overlay -o \
  lowerdir=/tmp/lower:/tmp/lowest,\
  upperdir=/tmp/upper,\
  workdir=/tmp/workdir \
  none /tmp/overlay

References

• Docker and OverlayFS in practice
• Unix & Linux - Querying an overlayfs
• Arch Linux - Overlay filesystem
• Ask Ubuntu - How do I use OverlayFS?
• Ask Ubuntu - Example OverlayFS Usage