Duplicity - Backing up to A Remote Computer Using GPG Encryption And Over An SSH Connection

This tutorial will show you how to back up your files to a remote computer in an encrypted format using duplicity.

Steps

First, generate a gpg key if you don't have one already that you wish to use.

gpg --gen-key

I would go with the the default options, a 2048 bit RSA key that never expires.

When it has finished being generated, export the private key and send it to yourself or another server in case you ever need to use it to restore the files and the source server becomes unavailable.

gpg --export-secret-key -a "blah@gmail.com" > [filename].asc

Now Generate an SSH key with no passphrase. It is important that this key is only used for passwordless logging into our backup account on the remote server, so make sure it is a non-sudo user with only access to its home.

ssh-keygen
ssh-copy-id host1@backups.programster.org

Install duplicity if you haven't already

sudo apt-get install duplicity -y

Below is a really simple tutorial for creating a backup on your local netowork. I would not use this to perform backups over the internet, because there is no encryption in place, either in transit or for the backup files. If you do wish to use encrypted backups, and use an encrypted connection (SSH), then please refer to

Steps

Copy and paste the following BASH backup script onto the server and edit your settings/variables appropriately.

#!/bin/bash
# A simple backup script wrapper for duplicity.
# Author - nixCraft  under GPL v2+
# Edited by Programster
# -----------------------------------------------------

## Define VARS ##
# Use gpg2 --list-keys to get your keys
GPG_KEY='9817322C'

# The path to where the files that we want to backup are
SOURCE='/home/stuart/code'

# The path to where we want to store the backup
TARGET='scp://host1@backups.programster.org//home/host1/backup'

# The full path to the duplicity program
# you shouldn't need to change this if you are using Ubuntu
DUPLICITY_CMD='/usr/bin/duplicity'

# Specify how far back we want to be able to restore to
# e.g. how long a backup lives in days before it is removed.
BACKUP_LIFETIME="30D"

# Specify how many days of iterative backups are allowed to
# be taken before a full backup has to be taken
DAYS_BETWEEN_FULL_BACKUPS="10D"

####################


## Removing old backups##
echo "Removing old backups..."
$DUPLICITY_CMD remove-older-than ${BACKUP_LIFETIME} \
  --encrypt-key="${GPG_KEY}" \
  ${TARGET}

## Backup our files ##
echo "Backing up files..."
$DUPLICITY_CMD \
  --full-if-older-than="${DAYS_BETWEEN_FULL_BACKUPS}"
  --force ${SOURCE} \
  --encrypt-key="${GPG_KEY}" \
  ${TARGET}

echo "done!"

On ubuntu 16.04 servers, if you were to run the script now, you would probably get the following error message:

BackendException: Could not initialise backend: No module named paramiko

We need to install the following package for duplicity to work over scp.

sudo apt-get install python-paramiko -y

Now run your backup script and you should get output similar to below:

Last full backup date: none
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1494703857.24 (Sat May 13 14:30:57 2017)
EndTime 1494703857.25 (Sat May 13 14:30:57 2017)
ElapsedTime 0.02 (0.02 seconds)
this is another file
SourceFiles 2
SourceFileSize 4108 (4.01 KB)
NewFiles 2
NewFileSize 4108 (4.01 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 2
RawDeltaSize 12 (12 bytes)
TotalDestinationSizeChange 521 (521 bytes)
Errors 0
-------------------------------------------------

Congratulations, you now have a remote backup! If you ssh into the remote server's backup area, you should now see a bunch of encrypted gpg files like below:

duplicity-full.20170513T193057Z.manifest.gpg
duplicity-full.20170513T193057Z.vol1.difftar.gpg
duplicity-full-signatures.20170513T193057Z.sigtar.gpg
duplicity-inc.20170513T193057Z.to.20170513T193126Z.manifest.gpg
duplicity-inc.20170513T193057Z.to.20170513T193126Z.vol1.difftar.gpg
duplicity-new-signatures.20170513T193057Z.to.20170513T193126Z.sigtar.gpg

Restoring files

If you ever need to restore the files, use the script below, making sure to set your variables/settings appropriately. You will need to run the script on a server that has access to the private key, and you will have to enter the passphrase.

#!/bin/bash
GPG_KEY='9817322C'

# The path to where the files that we want to backup are
TARGET='/home/stuart/restore'

# The path to where we want to store the backup
SOURCE='scp://host1@backups.programster.org//home/host1/backup'

# The full path to the duplicity program
# you shouldn't need to change this if you are using Ubuntu
DUPLICITY_CMD='/usr/bin/duplicity'

$DUPLICITY_CMD --force ${SOURCE} --encrypt-key="${GPG_KEY}" ${TARGET}

Risks

You may have spotted that there is one risk here, which is that the source server becomes compromised by a malicious user that wipes the source content, and removes the backup files from the remote backup server. I believe the best answer to this is to utilize automated snapshots of the backup server's disk, in a way that is out of the control of the backup user on that server. E.g. by a hypervisor hosting the backup server, or something else that provides the disk, such as root snapshotting a ZFS dataset. Another option is to have another user on the backup server have access to the files and rsync them to another area that only it has access to. I'm sure you can come up with something that works for you.

References

• Duplicity Docs