Set Up A Debian Mirror Using Rsync

When apt-mirror has problems, one can always fall back to the good old days of setting up mirrors using rsync.

Related Posts

• Deploy A Local Ubuntu Mirror using Rsync

Steps

Firstly, we need to set install apache and a storage path. The storage path should have over 1 TiB of space.

sudo apt install apache2 -y

STORAGE_PATH="/mirror-data/debian"
sudo mkdir -p $STORAGE_PATH
sudo ln -s $STORAGE_PATH /var/www/debian

Configure Apache

Run the following command to configure apache.

sudo echo '<VirtualHost *:80>
    ServerAdmin webmaster@hostname.com

    DocumentRoot /var/www
        <Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>' | sudo tee /etc/apache2/sites-enabled/000-default.conf

Now we need to create a user that will be executing the sync, yet the webserver can access all the files of. It's nice to be able to log in as this user to manually run the sync from time to time.

MIRROR_USER=syncer
sudo adduser $MIRROR_USER
sudo adduser www-data $MIRROR_USER

Now switch to that user and go to the home directory.

su $MIRROR_USER
cd $HOME

Create and execute the following script. I called it sync.sh which will be referenced later in a cron.

#!/bin/bash

STORAGE_PATH="/mirror-data/debian"

rsync \
  --recursive \
  --links \
  --perms \
  --times \
  --compress \
  --progress \
  --delete \
  rsync://ftp.uk.debian.org/debian \
  $STORAGE_PATH

You can add --bwlimit=1000 to limit the bandwidth to 1000 Kilobytes per second. You will need to do the math to convert from kilobytes to Megabits.

Cron

Now we need the mirror to automatically stay up to date. We will run the task at midnight.

Make sure you only do this after you have completed the initial sync which will likely take many hours.

If you haven't already, log in as your mirror user:

su $MIRROR_USER

Edit the crontab file for that user.

crontab -e

Add the following line:

0 0 * * * $HOME/sync.sh

Use Your Mirror

You can now configure your Debian computers to use this mirror for updates.

References

• Rsync Manual
• Ubuntu Docs - Rsyncmirror