This site is now 100% read-only, and retired.

Encrypted remote backups made easy

Posted by JohanJentell on Mon 27 Nov 2006 at 06:25

This is an easy way to take backups and putting them on a remote site. I invented this script in order to put backups on a reliable remote site where I unfortunatley only have a user account.

The prerequisites for this script is:

  • Public key authentication over SSH set up with the remote site. If you dont have this, follow this guide.
  • Gnu-PG installed

This is the entire script. Everything written in capital letters should be replaced by your own information. If you want an explanation of each section of the script, look a bit further down...

touch /tmp/$datetime
chmod 0600 /tmp/$datetime

find /var/www/ -name "*.html" -print >> /tmp/$datum

touch /tmp/backup_sql_$datetime.sql
chmod 0600 /tmp/backup_sql_$datetime.sql
mysqldump --databases YOUR_DATABASE_NAME -u YOUR_USERNAME -pYOUR_PASSWORD >> /tmp/backup_sql_$datetime.sql
echo /tmp/backup_sql_$datetime.sql >> /tmp/$datetime

tar cjfT /tmp/backup_$datetime.tar.bz2 /tmp/$datetime 2> /dev/null

echo SECRET_PASSWORD| gpg --batch --yes --passphrase-fd 0 --symmetric --cipher-algo AES256 /tmp/backup_$datetime.tar.bz2

scp -2 -p -B -q /tmp/backup_$datetime.tar.bz2.gpg REMOTE_USERNAME@REMOTE_HOST:REMOTE_FOLDER/
ssh -2 -lREMOTE_USERNAME REMOTE_HOST chmod 0600 REMOTE_FOLDER/backup_$datetime.tar.bz2.gpg

rm -f /tmp/$datetime /tmp/backup_sql_$datetime.sql /tmp/backup_$datetime.tar.bz2 /tmp/backup_aik_$datetime.tar.bz2.gpg

Ok here goes for explaining it:

touch /tmp/$datetime
chmod 0600 /tmp/$datetime
Creates a timestamp of when the backup is created.

touch /tmp/$datetime
chmod 0600 /tmp/$datetime
Creates a file in which to list all files to back up.

find /var/www/ -name "*.html" -print >> /tmp/$datum
Finds all .html documents in the /var/www folder and add to the backup-list.

touch /tmp/backup_sql_$datetime.sql
chmod 0600 /tmp/backup_sql_$datetime.sql
mysqldump --databases YOUR_DATABASE_NAME -u YOUR_USERNAME -pYOUR_PASSWORD >> /tmp/backup_sql_$datetime.sql
echo /tmp/backup_sql_$datetime.sql >> /tmp/$datetime
Takes a backup of your mysql database and add to the backup-list.

tar cjfT /tmp/backup_$datetime.tar.bz2 /tmp/$datetime 2> /dev/null
Creates a compressed file of all files in the backup-list using the bz2 algorithm (you might need support for bz2).

echo SECRET_PASSWORD| gpg --batch --yes --passphrase-fd 0 --symmetric --cipher-algo AES256 /tmp/backup_$datetime.tar.bz2
Encrypts the compressed file using AES encryption.

scp -2 -p -B -q /tmp/backup_$datetime.tar.bz2.gpg REMOTE_USERNAME@REMOTE_HOST:REMOTE_FOLDER/
Copy the encrypted file to a remote location

ssh -2 -lREMOTE_USERNAME REMOTE_HOST chmod 0600 REMOTE_FOLDER/backup_$datetime.tar.bz2.gpg
Make sure the remote file does not have any read/write privileges to the backupfile.

rm -f /tmp/$datetime /tmp/backup_sql_$datetime.sql /tmp/backup_$datetime.tar.bz2 /tmp/backup_aik_$datetime.tar.bz2.gpg
Cleans upp all locally used documents during the backup procedure

The bad things with this script: It will allow anyone with local root privileges to see the passwords you have entered into the script file. It will also reveal MySQL and GPG passwords used if someone is monitoring the processes on the local machine.

The good things: It is darn easy to use. Seen from a remote perspective, it is a very safe way of taking backups. All transmissions are using SSH v2. The remote backup using AES-256 and your choice of a strong password should keep your backups safe from evil decrypters during the next 20 years at least (unless any huge technical breakthrough or exploit found in the encryption algorithm)

Thoughts about the script: All STD Error output from tar is piped to /dev/null. This is only because tar otherwise always outputs "removing leading /", which is good that the program does, but is a bit irritating in your logfiles as it looks as something actually went wrong in the script. There should be a nicer solution to this.

 

 


Re: Encrypted remote backups made easy
Posted by Anonymous (139.140.xx.xx) on Mon 27 Nov 2006 at 06:45
You might be interested in duplicity:

"Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server."

(from the Debian package description)

[ Parent ]

Re: Encrypted remote backups made easy
Posted by Steve (62.30.xx.xx) on Mon 27 Nov 2006 at 07:09
[ View Weblogs ]

I'm a big fan of using rsnapshot since it uses rsync to create diffs it means that the backups are very space-efficient.

Although on the downside there is no explicit encryption at least the files are transferred via openssh.

I just export all databases prior to backing up to get them via the same process.

(For reference we've covered using duplicity in the past.)

Steve

[ Parent ]

Re: Encrypted remote backups made easy
Posted by Anonymous (134.84.xx.xx) on Tue 28 Nov 2006 at 16:58
I'm using rsnapshot to backup to an encfs encrypted filesystem on an NFS mount. It seems to work pretty well so far.

[ Parent ]

Re: Encrypted remote backups made easy
Posted by Anonymous (24.43.xx.xx) on Mon 27 Nov 2006 at 06:49
Is there a particular reason why you are specifying a passphrase in your script instead of just encrypting to a public key?

[ Parent ]

Re: Encrypted remote backups made easy
Posted by reluctant (65.78.xx.xx) on Mon 27 Nov 2006 at 15:39
aptitude install duplicity # works very well for me.

Description: encrypted bandwidth-efficient backup
Duplicity backs directories by producing encrypted tar-format volumes
and uploading them to a remote or local file server. Because duplicity
uses librsync, the incremental archives are space efficient and only
record the parts of files that have changed since the last backup.
Because duplicity uses GnuPG to encrypt and/or sign these archives, they
will be safe from spying and/or modification by the server.

[ Parent ]

Re: Encrypted remote backups made easy
Posted by sgeiger (67.78.xx.xx) on Mon 27 Nov 2006 at 23:03
hdup and duplicity are both remote backup encryption tools that are built on gnupg and rsync.

[ Parent ]

Re: Encrypted remote backups made easy
Posted by dkg (216.254.xx.xx) on Mon 27 Nov 2006 at 23:20
[ View Weblogs ]
I think you have a typo in the third line: shouldn't
find /var/www/ -name "*.html" -print >> /tmp/$datum
be
find /var/www/ -name "*.html" -print >> /tmp/$datetime
?

You might want to make sure that you don't open yourself to symlink attacks from predictable filenames, too. For example, another user on your source host might be malicious. That malicious user might write a script to watch the contents of /tmp and trigger an event as soon as something that looks like /tmp/$datetime. If it catches it before your find invocation is complete, the malicious script could create a dangerous symlink like this:

ln -s /home/youruser/yourimportantfile /tmp/backup_sql_$datetime.sql
Then the mysqldump process could actually overwrite the contents of /home/youruser/yourimportantfile on your behalf (hey, you wanted to redirect output to this symlink, right?), clobbering your important data.

Even more devious attacks can be crafted which exploit predictably-named files in world-writable directories like /tmp, potentially even forcing arbitrary users to execute arbitrary code.

A good way to avoid this sort of thing, when you need to work with temporary files is to use something standard like mktemp. here's a typical invocation:

## make a working directory, readable/writable only by me:
WD=$(mktemp -d)

## stuff some data into a file in that working directory:
find /var/www/ -name "*.html" -print >> "$WD/find.out"
Hope this is helpful!

[ Parent ]

Re: Encrypted remote backups made easy
Posted by Anonymous (213.239.xx.xx) on Tue 28 Nov 2006 at 06:39
I like backup-manager:

http://www.backup-manager.org/

"Backup Manager is a command line backup tool for GNU/Linux, designed to help you make daily archives of your file system.

Written in bash and perl, it can make tar, tar.gz, tar.bz2, and zip archives and can be run in a parallel mode with different configuration files.

Archives are kept for a given number of days and the upload system can use ftp or scp to transfer the generated archives to a list of remote hosts. The configuration file is very simple and basic and gettext is used for internationalization."

It uses gnupg for encryption.

[ Parent ]

Typo?
Posted by Anonymous (62.172.xx.xx) on Tue 28 Nov 2006 at 08:04
When you're backing up the .html files, you list them in /tmp/$datum, whereas the other lists of files to backup are in /tmp/$datetime.

I think the $datum is a typo...

Cheers,
Neil Greenwood.

[ Parent ]

Re: Encrypted remote backups made easy
Posted by Anonymous (66.129.xx.xx) on Fri 1 Dec 2006 at 00:19
Another solution to consider, espeically if you need to make incremental backups, is duplicity, as detailed in this previous article.

[ Parent ]