This site is now 100% read-only, and retired.

Making a bootable backup Debian system disk

Posted by rossen on Thu 17 Jan 2008 at 11:13

Making a backup disk is usually an obvious task, but making the disk ready to boot so that it can immediately be used as a replacement can be tricky. Here I give a recipe for making a bootable disk that can be adapted to whatever block device you may be using as a backup disk. This could be an internal or external hard disk, a USB key, etc.

I discuss some of the tricks that will avoid creation of a backup system disk that only gets as far as displaying

        GRUB

before it freezes. A number of other pitfalls are mentioned.

ASSUMPTIONS

For this discussion, I will assume that the system disk, also containing /boot, is a CF (Compact Flash) connected via an IDE interface (identified as hdc) and the backup disk is another CF (not necessarily the same size) connected to a USB reader. The reason for this unusual choice of media is that this procedure was developed for backing up MiniITX firewalls which often use a CF as their system disk.

It is assumed that the system disk that you wish to backup already boots correctly and uses GRUB as its bootloader. I also assume that the system is a Debian, but the procedure should be fairly easy to adapt to most GNU/Linux distributions.

The backup disk can be created using a second machine (call it "bob") or directly using the same machine that one wishes to backup (call it "alice"). Since the procedure was developed for backing up firewalls that were remotely accessible via SSH, I will assume that a second machine is used and that the file-system creation tools (e.g. mke2fs) are of the same version on both machines.

SUMMARY

Here is a summary of the procedure:

  1. Partition the backup CF on bob.
  2. Create an empty file-system image file on bob and mount it via the loopback device.
  3. Backup alice's root file-system to the loopback device on bob.
  4. Umount the loopback device and copy the file-system image file to the backup CF.
  5. Mount the backup CF, chroot into it, and run GRUB to make it bootable.

HINTS AND WARNINGS

For the following procedure, you want to have a very strict control of whether devices are mounted or not. Unfortunately, the default setting for the Gnome (and KDE?) desktop is to automount and browse hot-plugged devices. Instead of remembering to do a "umount /dev/sdb1; umount /dev/sdb2; ..." each time you plug in a USB disk, you can deactivate automounting. In the Gnome desktop, go to Desktop->Preferences->Removable Devices and Media and toggle off any of the automount options.

The procedure makes extensive use of low-level disk manipulation tools. If you do not understand what you are doing, or if you make a typing mistake, or if you have a weird BIOS or disk, you can easily render your system temporarily unbootable and maybe even wipe out your disks. If this warning was news to you, you better Read The Fine Manual first. You should read "info grub", "man cfdisk", "info parted", "man fdisk", "man dd", "man mke2fs", "man losetup", and "man rsync" to understand the purpose for every command and option used in this procedure.

PROCEDURE

1. The first pitfall to avoid is the creation of a backup disk with an odd or inconsistent disk geometry.

Linux does not care how many heads, sectors per track, and cylinders a disk has, but a lot of BIOSes, bootloaders, and disk manipulation tools which were designed to cope with DOS's restrictions do care. In addition some BIOSes detect USB keys as USB floppy drives, USB zip drives, or USB hard drives, depending on their size and (I suspect) their geometry. This makes it complicated to configure such a BIOS to boot the right device. fdisk and sfdisk will complain if they encounter block devices with inconsistent physical and logical geometries. DOS and Windows can be uninstallable on a disk that has an odd geometry.

The trick is to never leave disk geometry to chance. Wipe out the partition table and the first few megabytes of the backup disk and rebuild it with cfdisk, forcing a geometry of 255 heads and 63 tracks per sector. This geometry will signal BIOSes, bootloaders, and disk manipulation tools to use LBA addressing and to assume that the device is a big, modern hard disk (or USB HDD).

Note that parted and utilities based on libparted such as gparted do not (yet) give one sufficient control to specify disk geometry. The only utility that I have found that allows one to easily set the geometry is cfdisk.

Assuming that the backup disk was recognised as sdb on bob, (type "dmesg" and/or "cat /proc/partitions" and/or "lshw -short" to verify), use cfdisk to create a first partition, /dev/sdb1, big enough to hold all of alice's root file-system data and toggle it bootable. You can verify the result with parted and fdisk as indicated.

        dd if=/dev/zero of=/dev/sdb bs=1M count=10
        cfdisk -z -h 255 -s 63 /dev/sdb
        parted /dev/sdb print
        fdisk -l /dev/sdb

2. Creation of an empty image file for storing alice's root file system starts by deciding how big it should be. It should be no bigger than the partition created in the previous step. Assuming alice's root file system is comfortably stored on a 1GB CF, we will make a 950MB loopback image file on bob (since a 1GB CF is never exactly 1GB), give it an ext3 file-system with no reserved blocks, and mount it with the loopback device.

        mkdir -p /media/alice-hdc1 /scratch
        dd if=/dev/zero of=/scratch/alice-hdc1.img bs=1M count=950
        losetup /dev/loop/0 /scratch/alice-hdc1.img 
        mke2fs -j -m0 /dev/loop/0
        mount /dev/loop/0 /media/alice-hdc1

3. We will use rsync to backup alice's root file-system to the image file on bob. Using the "-x" option forces rsync to transfer only the root file-system, avoiding /proc, /sys, or any other file systems that might be mounted alice. The "--exclude" option of rsync can be used with the same effect if you wish to avoid copying some files to the backup. I like to remove the /etc/mtab file so that the backup system does not falsely detect an improperly umounted file-system and fsck it when it is first booted.

        rsync -ax root@alice:/ /media/alice-hdc1
        rm /media/alice-hdc1/etc/mtab
        umount /media/alice-hdc1
        losetup -d /dev/loop/0

4. At this point bob:/scratch/alice-hdc1.img contains an image file of an ext3 file-system containing all of alice's root file-system files. The advantage of the image file is that everything needed to create a backup of alice is in one easy-to-copy file in case you want to experiment with different backup media.

If you feel like it, you can avoid using the loopback image file by rsyncing directly to a directory on bob or even mounting the backup disk on bob and rsyncing directly to it. In fact, this may be your only choice if the root file-system of alice is bigger than the amount of free space on bob:/scratch.

Copy the image file to the backup disk on bob. Running "sync" is necessary to make sure that all data copied to the backup disk is really written to the disk. The sync might actually last a few minutes if /dev/sdb is a slow USB device.

        cat /scratch/alice-hdc1.img > /dev/sdb1
        sync

5. Finally, it is time to make the backup disk bootable. The procedure is to mount the disk and use it as a chroot environment where we run GRUB interactively to install GRUB's boot files into the disk's master boot record and following sectors with the correct parameters embedded. The chroot environment must have a /dev containing disk device files, so we bind-mount bob's /dev into the chroot. On bob, we do the following to setup and enter the chroot environment and run GRUB:

        mount /dev/sdb1 /media/alice-hdc1
        mount --bind /dev /media/alice-hdc1/dev
        chroot /media/alice-hdc1 /bin/bash
        grub

GRUB does not use the same disk and partition naming system as Linux since it is meant to be a Universal bootloader. We need to find GRUB's name for the backup disk. To do this, there is a convenient command called "find" that will tell us where a given file is. For example:

        grub> find /boot/grub/stage1
         (hd0,5)
         (hd1,0)

GRUB has found two instances of the stage1 file, one on bob's normal root disk and one on the backup disk meant for alice. Which is which?

        grub> cat (hd0,5)/etc/hostname
        bob

        grub> cat (hd1,0)/etc/hostname
        alice

It appears that (hd1,0) is the backup disk that we want to work on. At this point we could run the GRUB installation procedure on (hd1) and it would seem to succeed...

        grub> root (hd1,0)
         Filesystem type is ext2fs, partition type 0x83
        grub> setup (hd1)
         Checking if "/boot/grub/stage1" exists... yes
         Checking if "/boot/grub/stage2" exists... yes
         Checking if "/boot/grub/e2fs_stage1_5" exists... yes
         Running "embed /boot/grub/e2fs_stage1_5 (hd1)"...  16 sectors are embedded.  succeeded
         Running "install /boot/grub/stage1 d (hd1) (hd1)1+16 p (hd1,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
        Done.

...but it would NOT boot if we tried to replace alice's system disk with alice's backup system disk. All it would do is display "GRUB" and then stop. The reason is that the disk was programmed to look for stage2 of GRUB on (hd1,0) whereas the backup disk is now (hd0), not (hd1).

The trick is to use the "device" command to temporarily remap /dev/sdb to (hd0) before installing GRUB. Here is how to use "device" and check that it is working as wished.

        grub> device (hd0) /dev/sdb
        grub> cat (hd0,0)/etc/hostname
        alice

Now we can properly install to the temporarily redefined (hd0), quit GRUB, quit the chroot environment, and umount the backup disk.

        grub> root (hd0,0)
         Filesystem type is ext2fs, partition type 0x83
        grub> setup (hd0)
         Checking if "/boot/grub/stage1" exists... yes
         Checking if "/boot/grub/stage2" exists... yes
         Checking if "/boot/grub/e2fs_stage1_5" exists... yes
         Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  16 sectors are embedded.  succeeded
         Running "install /boot/grub/stage1 d (hd0) (hd0)1+16 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
        Done.
        grub> quit
        exit
        umount /media/hdc1/dev
        umount /media/hdc1

The backup disk is now bootable. Test it if you have the opportunity!

A QUICK-AND-DIRTY METHOD

If the system disk (e.g. sda) can be taken offline and the backup disk (e.g. sdb) is EXACTLY the same size, there is a far simpler quick-and-dirty method for creating a bootable backup system disk.

  1. Verify that neither disk has partitions that are mounted by typing "mount".
  2. Do a low-level copy of sda to sdb and do not forget to sync to make sure that all data is flushed to the destination disk.
        cat /dev/sda > /dev/sdb
        sync

Unfortunately, CFs seem to rarely have exactly the same size, even if you buy the same model from the same manufacturer. I suspect that this is due to a random number of bad blocks that are removed from each CF during the manufacturing process. Hard disks usually do not have this problem and two "300GB" drives from two different manufacturers often have identical geometries which makes them ideal candidates for this quick backup method.

Creative Commons License
This work is licensed under a Creative Commons Attribution-Share Alike 2.5 Switzerland License.

 

 


Re: Making a bootable backup Debian system disk
Posted by suspended user tong (69.63.xx.xx) on Sat 2 Feb 2008 at 16:36
[ View Weblogs ]
Nice write up. Very informative and insightful.

I have one question,

As a summary, the procedure in the article is:

1. Backup alice's root file-system into file-system image file on bob.
2. Run GRUB to make the backup CF bootable.

But to me, there is still once piece missing -- How to boot the backup bootable CF with the file-system image file?

Thanks

[ Parent ]

Re: Making a bootable backup Debian system disk
Posted by rossen (84.72.xx.xx) on Sat 2 Feb 2008 at 17:53
[ View Weblogs ]
Your summary is not quite complete. You seem to have missed a step after 1.: block-copying the image file from bob onto the CF (or whatever backup media you happen to be using). THEN you need to run GRUB to make the backup media bootable.

[ Parent ]

Re: Making a bootable backup Debian system disk
Posted by suspended user tong (69.63.xx.xx) on Sun 3 Feb 2008 at 23:55
[ View Weblogs ]
I meant to summarize the "missing" step in my step 1. Ok, so my step 1 should read:

1. Backup alice's root file-system into file-system image file on bob' CF card (or whatever backup media).

but then, my question still holds: How to boot the backup bootable CF with the file-system image file?

thanks

[ Parent ]

Re: Making a bootable backup Debian system disk
Posted by rossen (84.72.xx.xx) on Mon 4 Feb 2008 at 07:55
[ View Weblogs ]
Remove alice's old (dead) media and put the backup media in its place.

The goal of the procedure was to create a drop-in, bootable replacement for alice's old media without taking alice offline or requiring a backup media that was exactly the same size as the original media.

If you want to boot the backup media *without* replacing alice's old media (and alice is still bootable), you can configure the GRUB on the old media to boot the backup media (or do a one-time, customised boot using the GRUB command prompt), but you are likely to run into difficulties since /etc/fstab is identical on both media.

[ Parent ]

Re: Making a bootable backup Debian system disk
Posted by suspended user tong (69.63.xx.xx) on Mon 4 Feb 2008 at 14:58
[ View Weblogs ]
thanks

[ Parent ]

Re: Making a bootable backup Debian system disk
Posted by quanguniten (211.24.xx.xx) on Sun 4 Dec 2011 at 09:35
This is very interesting and I want to do something similar. I have a PC with Debian 5.0 running. I connected 1 CF card to IDE (secondary) and it is detected as /dev/hdc1. Then I connected the 2nd CF card using USB reader and it is detected as /dev/sda1. Everything is on the same PC. When I typed the command: rsync -ax root@nguyen:/ /media/quang-hdc1, I got the error message: could not resolve host name nguyen... nguyen is the hostname of the CF card connected to IDE. So do you have any idea what I have done wrong? Thanks in advance, really appreciate your help.

[ Parent ]

Re: Making a bootable backup Debian system disk
Posted by Anonymous (72.37.xx.xx) on Fri 15 Jan 2010 at 19:31
Hoo-freakin-aray! I have nearly the same issue - and it was perplexing me into the twilight zone as to why some CF devices could be copied and other not. This works -- and now all my travails make sense. Time to add a browser favorites. Thanks for the help!

[ Parent ]

Re: Making a bootable backup Debian system disk
Posted by Zoanie (72.37.xx.xx) on Fri 9 Apr 2010 at 20:31
I used your notes to make a number of backup CF disk Linux systems. It always works what ever the CF disk sizes. That was in the bucolic days before grub2. Now i'm back to square one trying to backup a CF Linux system. Can you possibly take a look at what is required to change in your note to support grub2? It looks like the 'device' command was removed and 'root' is changed as well. Many thanks for your orginal note.

[ Parent ]

What about restoring it back to its computer?
Posted by locke (83.49.xx.xx) on Wed 8 Sep 2010 at 08:59
Hi, nice paper this one, got it working in no more than a testing morning.
But now, as the clumsy one I am, I wonder, how can I use such pen-drive (or what ever) to restore it to an analogue computer. I mean, i've made my "perfect" system image and now I want to use it to flush it to all servers like the source one (partitions already made in each brand new one)..... how can i? please

[ Parent ]