This site is now 100% read-only, and retired.

Backing Up/Restoring Package Information?

Posted by mthaddon on Fri 28 Oct 2005 at 20:02

I'm wondering if anyone can help me out with the best way to backup/restore debian package information? I'm trying to write a script which will allow for "seamless" server migration (I realize this is something of an impossibility), and obviously one of the key pieces is being able to match the packages and the package configuration of one server to another.

I know I can do "dpkg --get-selections" on the source server and then "dpkg --set-selections" followed by "apt-get -u dselect-upgrade" to load the correct packages, but how do I ensure that the configuration options that were chosen on the original server are preserved on the target server?

Thanks, Tom

 

 


Re: Backing Up/Restoring Package Information?
Posted by antonh (129.13.xx.xx) on Fri 28 Oct 2005 at 21:04

Hi

I once had the same probelem. I backed up the following:

  • dpkg --get-selections
  • /var/cache/debconf/
  • /etc
  • /home
  • programm specific data from /var (samba, mail, ...)
  • my kernel
  • additionaly installed debs
  • self build debs
  • ...

On the target system I did this:

  • a minimal installation
  • load the backup of
    • /etc
    • /var/...
    • /home
  • dpkg --set-selections
  • apt-get dselect-upgrade
  • installation of my kernel
  • installation of other debs

I hope I haven't forgotten anything important.

Good backuping Anton

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by lpenz (200.102.xx.xx) on Fri 28 Oct 2005 at 21:21
There is supposed to be an easier way using debsums and cruft. Anyone?

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by Anonymous (82.250.xx.xx) on Sat 29 Oct 2005 at 15:15

May-be using dpkg-repack on each installed package in order to build a local deb server ?

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by Anonymous (85.75.xx.xx) on Sat 29 Oct 2005 at 19:37
Backup2l used this function (/etc/backup2l.conf) to backup the system configuration.


PRE_BACKUP ()
{
#echo " pre-backup: nothing to do"

# e. g., shut down some mail/db servers if their files are to be backup'ed

# On a Debian system, the following statements dump a machine-readable list
of
# all installed packages to a file.
echo " writing dpkg selections to /root/dpkg-selections.log..."
dpkg --get-selections | diff - /root/dpkg-selections.log > /dev/null || dpkg
--get-selections > /root/dpkg-selections.log
}


Backup2l is a nice incremental backup tool. I am still waiting for a feature in the Debian install CD, which allows me to specify the location of a backup2l (I sure there are other similar tools) space and then restores w/o any user interaction the complete system with this incremental backup information.

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by Anonymous (83.161.xx.xx) on Sat 29 Oct 2005 at 21:01
you may also consider to simply copy everything from your old server to the new one. I managed to switch to new server hardware with a downtime less than 3 minutes. I wrote a howto, calling this 'hotcloning', which can be found here:
http://olivier.sessink.nl/publications/hotcloning/index.html

if your new server is supposed to be identical to the old one, this method is the best thing you can do.
regards,
Olivier Sessink

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by suspended user mthaddon (66.134.xx.xx) on Sat 29 Oct 2005 at 21:08
Hi Olivier,

That sounds pretty cool. I was originally thinking of this option, but was warned off it because of issues with synchronizing debian package information, databases and such. Do you know of any conflicts with MySQL, PostgreSQL or any other applications with this approach, or can you confirm that these work fine. I'd much prefer your approach if possible, because it's so much cleaner, and I want to be able to do this on a recurring basis (essentially keeping a "slave" server in sync with a "master" server).

Thanks, Tom

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by Anonymous (83.161.xx.xx) on Sun 30 Oct 2005 at 21:43
Mysql works fine BUT: during the final sync, you should stop all services. You cannot do this hotcloning with mysql running. Same for postgresql, samba, nfs or any other service that might change files. During the final sync any process that changes critical files should be stopped.

There might be one reason why this could not work: if you switch to hardware of a different platform. If you go from 32bit to 64bit, don't count on hotcloning, but I suppose that is obvious because you need other debian packages in such a setup as well.

(note: if you go to a 64 bit amd64 machine, but you run it in 32bit compatibility mode the method will work fine, but not if you go to native 64bit)

regards,
Olivier Sessink

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by Chris2000 (62.204.xx.xx) on Wed 2 Nov 2005 at 09:10
Hi,

I've also been trying this syncing approach. It works fine to some extent but I do wonder about those same problems Tom mentions.
I'm looking for a way to run a group of remote servers (client_server) by just mantaining one master_server.

Let's say we are pushing the master_server copy out to the client_server.
The ssh keys should not be cloned to the client_server so we need to exclude
/etc/ssh/ssh_host_*

How many other examples like this exist on a linux system? Is it possible to list them? Would the list be too unstable to maintain easily?

How about updating postfix on the master_server and then pushing the changes onto the client_server.
If I list /etc/postfix I get
dynamicmaps.cf master.cf postfix-script sasl sender_canonical.db main.cf postfix-files post-install sender_canonical

Here, there are files that I don't want overwritten. So let's say I choose to exclude /etc/postfix from the copy.
But what happens if a postfix update on the master_server changes the format of the config files? (updates often do that sort of thing). Do I get inconsistency between the config files and the rest of the package?

Like Tom, I would prefer and easy solution to this.

I've started to put a list together.

**** files and directories to exclude when pushing an update ****
/etc/hosts
/etc/resolve.conf
/etc/dhcpd.conf
/etc/firewall
/etc/network
/etc/passwd*
/etc/shadow*
/etc/group*
/etc/samba
/var/spool
/opt/ltsp/i386/etc/lts.conf
/boot
/vmlinuz
/initrd.img
/lib/modules
/etc/fstab
/etc/mtab
/etc/mdadm
/etc/rsync
/etc/hostname
/etc/ssh/ssh_host_*
/etc/ssl
/var/tmp
/var/log
/var/run
/var/lock
/root/
/mount
/media
/proc
/tmp
/dev
/usr/src
/home
************************

Chris.

[ Parent ]

Re: Backing Up/Restoring Package Information?
Posted by i_mac (205.150.xx.xx) on Wed 2 Nov 2005 at 17:19
Hey there,

Rsync, with Olivier's traditional approach, and caveats for specific apps/mounts/sesions is great.

There is something more advanced, and may not be ready for non-power users of Debian. Official Debian sources are not up to date, but I expect Etch to have v2.X support, if not 3.x. I run 2.0.7 on Sarge currently, and now farm out virtual machines for all sorts of tasks/friends/etc.

It provides a <10ms transfer of of a running box to new iron on the same subnet if you have a network card that can swich MAC addresses (i.e. DEC/Tulip) to allow for the instant layer-2 switch transistion of live traffic.

For people with NIC's that can't change their MAC, the transfer requires a pause equivalent to a bridge topology change, or in the case of slower interconnects, how fast you can sync up the memory. You can Pause -> Transfer -> Start for a worst case scenario. Syncing the data depends on your setup (disk image from a file, shared storage/SAN, nfs mount)

Anyhow, its the Xen virtual machine monitor. (xen.sf.net). I have multiple webservers running on xen domains, and I can move 'em around as I please. Run your servers in a Xen domain (which is identical package-wise to a real machine) and your hot failover is a synch.

HTML Blurb on how the Live transfer of hot iron here:
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user. html#SECTION02430000000000000000

Information on Debian packages here:
http://wiki.xensource.com/xenwiki/DebianPackageMaintenance

A Google summer of code project created the latest debs (Up to kernel-soruce-2.6.11 for sarge); I hope to see linux-patch-xen for the linux-source-2.6.14 series kernels soon.

Well that's enough info to get interested parties going. Back to my GNU/Solaris project...

[ Parent ]