Simple rotated backups with rsnapshot

rsnapshot is a lightweight backup solution which creates rotated backups of local or remote directories. One of the key benefits of rsnapshot is its extreme simplicity. As a bonus backups are created using hardlinks to reduce the space used upon the backup host.

Like many backup solutions rsnapshot is a script which is built upon a foundation of OpenSSH and Rsync - the latter being used to synchronise directory contents without using excessive bandwidth, and the former to ensure the communication is encrypted and secure.

The script takes care of rotation automatically allowing you to setup the directories you wish backed up and then forget it.

You will most likely wish to run it in one of two ways:

• Give it a list of local directories to be backed up.
• Give it a list of remote directories to be backed up, and the host from which they should be taken.

The installation is straightforward, as you would expect:

apt-get install rsnapshot

Once installed you'll need to make several changes to the configuration file /etc/rsnapshot.conf:

• Setup the rotation periods.
• Setup the list of directories/files to backup.
• Optionally exclude files from the backups.

The only thing to remember is that when setting up keys and values in the configuration file you must specify them as Tab-separated. Why this choice was made I have no idea - syntactically significant whitespace should be regarded as evil IMHO.

Moving on the first thing you'll need to do is uncomment the various rotation periods:

interval        hourly  6
interval        daily   7
interval        weekly  4
interval        monthly 6

If these are left commented out (Comments are lines beginning with the '#' character) then nothing will happen.

The numbers listed against the names will control how often the backups are rotated, so by default you'll receive:

• 6 hourly backups.
• 7 daily backups.
• 4 weekly backups.
• 6 monthly backups.

Once these have been uncommented you'll need to setup the directories to backup. How this is done depends on the nature of the directories, local vs. remote.

A local directory will look like this:

# LOCALHOST
backup /home/          localhost/
backup /etc/           localhost/
backup /usr/local/     localhost/

(Remember that tabs are used to separate these values).

A remote system will instead look like this:

backup  root@hostname:/home/            hostname/
backup  root@hostname:/etc/             hostname/
backup  root@hostname:/usr/local/bin/   hostname/

The difference between these two samples is the second section of the lines; for local systems it is just a directory name with a trailing "/" character. For remote hosts you should use "login@full.hostname.com:/path".

The last field in the samples is the backup directory to use upon the localhost - by default the backups will be stored beneath "/var/cache/rsnapshot". You can change this by setting:

#
# Change the location where backups are stored.
#
snapshot_root   /mnt/nfs/backupstore

If you are setting up remote backups the system assumes the there is a password-less login setup between the local machine and the remote machine specified (root@hostname in this example).

If you've not setup key-based OpenSSH logins then now is a good time to do it.

Note that you don't need to use remote root logins unless the files you wish to backup are unreadable to ordinary users upon the machine you're backing up. (Although it's fair to say if you're backing up /etc you'll almost certainly wish to do so as root - as otherwise important files such as /etc/shadow will not be read).

Once you've configured your logins for remote machines, and setup the filepaths you wish to backup you only need to enable the cronjob used to run the backups.

To do that remove the "#" comment characters from the relevant lines in /etc/cron.d/rsnapshot file leaving you with :

# This is a sample cron file for rsnapshot. 
# The values used correspond to the examples in /etc/rsnapshot.conf.
# There you can also set the backup points and many other things.
#
# To activate this cron file you have to uncomment the lines below.
# Feel free to adapt it to your needs.

0 */4   * * *           root    /usr/bin/rsnapshot hourly
30 3    * * *           root    /usr/bin/rsnapshot daily
0  3    * * 1           root    /usr/bin/rsnapshot weekly
30 2    1 * *           root    /usr/bin/rsnapshot monthly

The names used here "hourly", "daily", etc, match those used in the rsnapshot.conf file - which is important.

Once you've enabled the cronjobs you should be ready to make your first backup - which will take a while.

To do so run:

rsnapshot hourly

If all goes well you'll receive your backup files beneath "backup_root"/hourly.0/hostname, and any errors will be reported in the file /var/log/rsnapshot.log.

After a few days you'll notice that you have several new files such as:

root@scratchy:/mnt/backup# ls -l
total 68
drwxr-xr-x  4 root root  4096 2005-08-16 04:00 daily.0
drwxr-xr-x  4 root root  4096 2005-08-15 04:00 daily.1
drwxr-xr-x  4 root root  4096 2005-08-14 04:00 daily.2
drwxr-xr-x  4 root root  4096 2005-08-13 04:01 daily.3
drwxr-xr-x  4 root root  4096 2005-08-12 04:00 daily.4
drwxr-xr-x  4 root root  4096 2005-08-11 04:00 daily.5
drwxr-xr-x  4 root root  4096 2005-08-10 04:01 daily.6
drwxr-xr-x  4 root root  4096 2005-08-17 20:01 hourly.0
drwxr-xr-x  4 root root  4096 2005-08-17 16:00 hourly.1
drwxr-xr-x  4 root root  4096 2005-08-17 12:00 hourly.2
drwxr-xr-x  4 root root  4096 2005-08-17 08:01 hourly.3
drwxr-xr-x  4 root root  4096 2005-08-17 04:00 hourly.4
drwxr-xr-x  4 root root  4096 2005-08-17 00:01 hourly.5

These are the rotated backups which we've saved, the initial backups will be the largest - the later ones will only contain changes made since the previous run.

To see this you can view the amount of disk space taken up by each backup with "rsnapshot du":

root@scratchy:/mnt/backup# rsnapshot du
8.1G    /mnt/backup/hourly.0/
124M    /mnt/backup/hourly.1/
164M    /mnt/backup/hourly.2/
90M     /mnt/backup/hourly.3/
90M     /mnt/backup/hourly.4/
79M     /mnt/backup/hourly.5/
194M    /mnt/backup/daily.0/
199M    /mnt/backup/daily.1/
216M    /mnt/backup/daily.2/
189M    /mnt/backup/daily.3/
229M    /mnt/backup/daily.4/
224M    /mnt/backup/daily.5/
324M    /mnt/backup/daily.6/
11G     total

As the output shows the initial backup consumes a significant amount of disk space, and later ones only show the small amount of changes since then.

To reduce the space of the backups you can exclude files you don't care about, such as Apache logfiles. Simply add the filenames, or patterns, to the configuration file:

exclude access.log
exclude access.log.*
exclude error.log
exclude error.log.*
exclude referer.log
exclude referer.log.*
exclude agent.log
exclude agent.log.*
exclude Mail/backup/*
exclude /home/qemu
exclude Cache
exclude history.dat
exclude .bash_history
exclude viminfo

Note that any filename which matches the patterns will be excluded - so don't be too permissive or you might accidentally include a filename you did care about.