Posted by Steve on Thu 3 Aug 2006 at 09:39
When it comes to installing new installations of Debian GNU/Linux there is one tool which should not be ignored. Whether you're dealing with a real system, or a virtualised one, the debootstrap tool is ideal for quickly installing new Debian environments.
Put simply the debootstrap package allows you to install a fresh copy of Debian GNU/Linux into a directory. This new installation will have all the basic packages and binaries which you'd expect to be present such as:
Using debootstrap might seem a little bit strange if you're only used to the Debian installer, but it is a very useful tool for performing installations if you've got something like a LiveCD which supports your hardware. The process would be go something like this:
If you already have a system installed then you can use debootstrap to setup a "chroot" installation of Debian for testing purposes.
Chroot?A chroot on Unix operating systems is an operation which changes the root directory. It affects only the current process and its children. The term "chroot" itself can refer either to the chroot(2) system call or the chroot(8) wrapper program.
A program that is re-rooted to another directory cannot access files outside that directory. This provides a convenient way to sandbox an untrusted, untested or otherwise dangerous program. It is also a simple kind of "jail" mechanism.
To get started with debootstrap you must first install it:
root@desktop:~# apt-get install debootstrap
Once installed you can start experimenting with it. A simple invocation would look like this:
root@desktop:# mkdir /tmp/blah root@desktop:# debootstrap sarge /tmp/blah/
Here we're telling debootstrap that we wish to have a fresh installation of Sarge made into the directory /tmp/blah. The process will take a while since it will involve downloading packages from the Debian mirrors, as you can see from the following output:
root@desktop:/tmp# debootstrap sarge /tmp/blah/ I: Retrieving Release I: Retrieving Packages I: Validating Packages I: Resolving dependencies of required packages... I: Resolving dependencies of base packages... I: Found additional required dependencies: libtext-iconv-perl zlib1g I: Checking component main on http://ftp.debian.org/debian... I: Retrieving adduser I: Validating adduser I: Retrieving apt I: Validating apt ....
Once the system has finished you can use the chroot command to "jail" yourself inside it:
root@desktop:~# chroot /tmp/blah root@desktop:/# root@desktop:/# root@desktop:/# ls bin dev home lib mnt proc sbin sys usr boot etc initrd media opt root srv tmp var root@desktop:/# root@desktop:/# apt-get clean root@desktop:/# root@desktop:/# du . --human-readable --total | grep total 111M total
Here you can see that you have a minimal system which occupies just over 100Mb of space. Notice that your prompt still contains the name of your "real" system? Don't be tempted to change that - since the hostname change will take effect on your host system too.
Apart from the hostname issue the new installation is independent from your real system, it just shares the IP address. If you want to use this environment properly you'll need to make a couple of changes. (Exit from the chroot first):
root@desktop:~# mount proc /tmp/blah/proc/ -t proc root@desktop:~# cp /etc/resolv.conf /tmp/blah/etc root@desktop:~# cp /etc/hosts /tmp/blah/etc root@desktop:~# chroot /tmp/blah/ root@desktop:/#
Since the new system will not be using PPP, it will have access to the network via the host system, you can tidy it up a little:
root@desktop:~# chroot /tmp/blah /usr/bin/dpkg --purge ppp pppoe pppconfig pppoeconf root@desktop:~# chroot /tmp/blah /usr/bin/apt-get install deborphan less root@desktop:~# chroot /tmp/blah root@desktop:/# deborphan libpcap0.7 root@destkop:/# dpkg --purge libpcap0.7
Now we have a nice minimal installation which can be modified or purged whenever you're done with it. If you mounted /proc you should remove it first:
root@desktop:~# umount /tmp/blah/proc root@desktop:~# rm -rf /tmp/blah/Caching
If you're going to use this command often you'll most likely wish to use a local mirror, or even better a cache.
For example if you have the machine itchy running an installation of apt-proxy you can use that :
root@desktop:~# debootstrap sarge /tmp/blah/ http://itchy:9999/debianThis will download all the packages via your proxy, and the second time you execute it will give you a significant speedup (since the Debian packages will be coming from the cache).
So what can you use the combination of chroot and debootstrap for? Well the possibilities are endless! If you remember the previous article on testing packages with pbuilder you might see a connection. pbuilder allows you to setup pristine "environments" for testing Debian packages, and it does that by invoking debootstrap to create fresh installations of Debian into a directory.
So using debootstrap, as we've shown, you can test package installations and dependencies. You could even setup a new environment for each package build by hand if you were masochistic.
If you're using Xen or UML you'll find debootstrap is the way to install new instances of Debian. Make the installation then add in a /etc/fstab file and networking details in /etc/network/interfaces and you're done.
There are a lot of useful options which debootstrap has which we've not covered, to setup an arch, etc. See the manpage for details:
man debootstrap
True enough. The last time I installed my system I noticed this and installed approx instead.
(I chose that only because it is 99% compatible with apt-proxy and listens on the same port by default. This meant I didn't need to update any of my client machines.)
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
But debootstrap is useful for more than just the installer - and it might be new to people so I figure a summery is a good thing.
Besides this will be a nice introduction to the idea of installing into a directory for next weeks piece on rpmstrap ;)
[ Parent ]
A clear list of the steps that need to be taken instead would be a useful thing.
[ Parent ]
Write a script.
I have some Xen instances that are done that way around. Actually debconf supports preseeding the answers to questions asked by packages as they are installed, but I never got into setting that up as it seemed like too much work. Much easier to simply write a script that does:
These things are much easier to deal with through scripting in my oppinion. Especially if you have a thing like cfengine (as per Steve's excellent articles) to take care of everything after getting the basic system online.
[ Parent ]
[ Parent ]
I don't mind at all - there are 3 interesting scripts here:
xen_make_bash.sh
#!/bin/sh
#
echo '$Id: xen_make_base.sh 72 2006-04-11 08:00:27Z admin $'
cnf=/hoeg/share/xen.conf
if [ -e $cnf ] ; then
source $cnf
else
echo "Configuration $cnf not found. Aborting..."
exit 1
fi
source $(dirname $0)/$shared
echo "Setting up..."
lv="${lv}_base"
dev=/dev/$vg/$lv
mnt=$(mktemp -d)
if [ -e $dev ] ; then
echo "Removing old LV..."
oldmnt=$(mount | grep $lv | cut -f3 -d ' ')
if [ x"$oldmnt" != "x" ] ; then
umount $oldmnt
rmdir $oldmnt
fi
remove_LV $dev
fi
echo "Creating and mounting LV..."
/sbin/lvcreate -L${xen_root_size} -n $lv $vg
create_FS $dev $fs_type
mount -t $fs_type $dev $mnt
if [ ! -d $src/pool ] ; then
echo "Mounting $src..."
mount $src
fi
echo "Bootstrapping ${distribution}/${flavour}..."
cdebootstrap -q -f $flavour $distribution $mnt $aptsrc
echo "Copying packages and scripts for installation..."
cp $debsrc/stage? $mnt/tmp -R
cp $bin/$config_script $mnt/tmp/
lists=/var/lib/apt/lists
cp $lists/*Packages ${mnt}${lists}
cp $lists/*Release ${mnt}${lists}
kernel=$(uname -r | sed 's/xen0/xenU/')
mkdir $mnt/lib/modules
cp /lib/modules/*-xenUphh $mnt/lib/modules/ -R
touch $mnt/CONFIGURE_ME
echo "Calling configure script in chroot"
chroot $mnt /tmp/$config_script $fs_type $xen_domain $xen_net
echo "Making tarball..."
if [ -e $tarsrc ] ; then
rm $tarsrc
fi
cd $mnt
tar -czf $tarsrc *
cd /
echo "Umounting and removing ${mnt}..."
umount $mnt
rmdir $mnt
echo "Removing LV ${dev}..."
remove_LV $dev
echo "Done!"
exit 0
xen_config_base.sh
#!/bin/sh
#
echo '$Id: xen_config_base.sh 186 2006-06-08 20:40:52Z admin $'
pkglist="/tmp/packages.list"
fs_type=$1
domain=$2
net=$3
if [ ! -e /CONFIGURE_ME ] ; then
echo "Do not run me on anything but a fresh XEN host. Aborting."
exit 1
else
rm /CONFIGURE_ME
fi
function log()
{
echo "Fixing ${1}..."
}
unset LANGUAGE LC_ALL LANG
log "/lib/tls"
mv /lib/tls /lib/tls.disabled
log "/etc/fstab"
cat > /etc/fstab << _END_
#
proc /proc proc defaults 0 0
/dev/hda1 / $fs_type noatime,errors=remount-ro,data=writeback 0 1
/dev/hda2 none swap sw 0 0
/dev/hda3 /srv $fs_type noatime,data=writeback 0 0
_END_
log "shadow passwords"
/sbin/shadowconfig on
log "host name"
echo "BASE_SYSTEM_NOT_CONFIGURED" > /etc/hostname
log "network settings"
mkdir /etc/network
cat > /etc/network/interfaces << _END_
auto lo
iface lo inet loopback
_END_
cat > /etc/network/options << _END_
ip_forward=no
spoofprotect=yes
syncookies=no
_END_
cat > /etc/resolv.conf << _END_
search $domain
nameserver ${net}.2
nameserver 10.0.0.1
_END_
cat > /etc/hosts << _END_
127.0.0.1 localhost.localdomain localhost
10.0.0.1 doris.lan.tz.hoeg.com doris
${net}.1 doris.$domain doris
${net}.2 donna.$domain doris
${net}.3 debbie.$domain doris
${net}.4 diana.$domain doris
_END_
log "APT"
cat > /etc/apt/sources.list << _END_
deb http://security.debian.org/ stable/updates main contrib non-free
deb http://ftp.us.debian.org/debian/ stable main non-free contrib
deb http://www.backports.org/debian/ sarge-backports main
_END_
cat > /etc/apt/preferences << _END_
Package: *
Pin: release a=sarge-backports
Pin-Priority: 90
_END_
log "CFengine"
mkdir /etc/cfengine
cat > /etc/cfengine/update.conf << _END_
# Basic update.conf to get us started
control:
any::
actionsequence = ( copy )
domain = ( tz.hoeg.com )
SplayTime = ( 0 )
ignore:
.svn
copy:
/srv/cfengine.tz/cfengine
dest=/etc/cfengine
r=inf
mode=644
type=binary
exclude=*~
exclude=#*
server=doris.$domain
_END_
log "consoles"
inittemp=$(mktemp)
cat /etc/inittab | grep -v "^[2-6]:23:respawn" > $inittemp
mv $inittemp /etc/inittab
log "bootstrap script"
cat > /boot_strap.sh << _END_
#!/bin/sh
passwd
cfagent -q -v > ~/cf.log
_END_
chmod 755 /boot_strap.sh
log "MOTD"
cat > /etc/motd << _END_
Run /boot_strap.sh to:
- change the root password
- run cfagent -q -v > ~/cf.log
_END_
log "packages"
/usr/bin/dpkg -P cdebootstrap-helper-diverts
cd /tmp
for s in stage? ; do
cd $s
/usr/bin/dpkg -i *.deb
cd ..
done
/usr/bin/dpkg --configure -a
log "invalid keys"
rm /var/lib/cfengine2/ppkeys/*
rm /etc/ssh/ssh_host*key*
log "clean up"
rm /var/cache/apt/srcpkgcache.bin
rm /tmp/stage? -R
exit 0
xen_new_host.sh
#!/bin/bash
echo '$Id: xen_new_host.sh 205 2006-06-17 14:19:10Z admin $'
force=0
go_from="lv"
cnf="/hoeg/share/xen.conf"
if [ -e $cnf ] ; then
source $cnf
else
echo "Configuration $cnf not found. Aborting..."
exit 1
fi
source $(dirname $0)/$shared
# mandatory parameters
name=""
ip=""
# optional parms
data_size=""
make_data=0
# defaults
header='$Id: xen_new_host.sh 205 2006-06-17 14:19:10Z admin $'
function print_help() {
echo "Usage: $0 [options]"
cat << _END_
Required:
-i IP address (example: -i 20 for ${xen_net}.20)
-n name of VM (example: -n doris)
-m memory in MB (default: -m $xen_mem)
Optional:
-g start from section [lv|base|config|xen]
lv = create LVs
base = unpack base system
config = configure base system
xen = create XEN configuration file
-f force overwrite of existing LVs (DANGEROUS)
-r size of root LV in GB (default: -r $xen_root_size)
-s size of data LV in GB (example: -s 2)
-t file system type (default: -t $fs_type)
-v name of VG (default: -v $vg)
_END_
}
function check_LV() {
if [ -e $1 ] ; then
if [ $force -eq 1 ] ; then
remove_LV $1
else
echo "LV $1 already exists. Aborting..."
exit 3
fi
fi
}
if [ "$#" -lt 4 ] ; then
print_help
exit 1
fi
until [ -z "$1" ] ; do
case "$1" in
# mandatory
"-i")
shift
ip="${xen_net}.${1}"
shift
;;
"-n")
shift
name=$1
shift
;;
"-m")
shift
mem=$1
shift
;;
# optional
"-g")
shift
go_from=$1
shift
;;
"-f")
force=1
shift
;;
"-r")
shift
xen_root_size=$1
shift
;;
"-s")
shift
data_size=$1
make_data=1
shift
;;
"-t")
shift
fs_type=$1
shift
;;
"-v")
shift
vg=$1
shift
;;
*)
echo "Unknown parameter!"
echo "Error: $1"
exit 2
esac
done
lv_root_name=${lv}_${name}_root
lv_swap_name=${lv}_${name}_swap
lv_data_name=${lv}_${name}_data
lv_root_dev=/dev/${vg}/${lv_root_name}
lv_swap_dev=/dev/${vg}/${lv_swap_name}
lv_data_dev=/dev/${vg}/${lv_data_name}
function do_LV_and_FS() {
echo "Creating LVs..."
# root
check_LV $lv_root_dev
/sbin/lvcreate -L${xen_root_size} -n ${lv_root_name} ${vg}
create_FS ${lv_root_dev} ${fs_type}
# swap
check_LV $lv_swap_dev
/sbin/lvcreate -L${xen_swap_size} -n ${lv_swap_name} ${vg}
create_FS ${lv_swap_dev} "swap"
# data
check_LV $lv_data_dev
if [ $make_data == 1 ] ; then
/sbin/lvcreate -L${data_size}G -n ${lv_data_name} ${vg}
create_FS ${lv_data_dev} ${fs_type}
fi
}
function do_base_system() {
echo "Creating base file system..."
mntdir=$(mktemp -d)
/bin/mount -t ${fs_type} ${lv_root_dev} ${mntdir}
cd $mntdir
tar xfz $tarsrc
}
function do_base_config() {
echo "Configuring ${name}..."
echo $name > $mntdir/etc/hostname
echo "Setting XEN identifier..."
echo $(date) > $mntdir/etc/xenU
echo "Configuring network at ${ip}..."
cat > $mntdir/etc/network/interfaces << _END_
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address $ip
netmask 255.255.255.0
gateway ${xen_net}.1
_END_
echo "Configuring host file..."
cat > $mntdir/etc/hosts << _END_
127.0.0.1 localhost.localdomain localhost
10.0.0.1 doris.lan.tz.hoeg.com doris
${xen_net}.1 doris.$xen_domain doris
${ip} ${name}.${xen_domain} ${name}
_END_
echo "Creating SSH keys..."
auth=/root/.ssh/authorized_keys
/usr/bin/ssh-keygen -q -t dsa -f $mntdir/etc/ssh/ssh_host_dsa_key -N ''
if [ ! -d $mntdir/root/.ssh ] ; then
mkdir $mntdir/root/.ssh
fi
/usr/bin/ssh-keygen -q -t dsa -f $mntdir/root/.ssh/id_dsa -N ''
if [ "$(grep root@${name}$ $auth | wc -l)" -gt 0 ] ; then
tempfile=$(mktemp)
grep -v root@${name}$ $auth > $tempfile
mv $tempfile $auth
fi
cat $mntdir/root/.ssh/id_dsa.pub >> $auth
echo "Creating CFengine keys..."
cfdir=/var/lib/cfengine2/ppkeys
cd $mntdir/$cfdir
cp $cfdir/localhost.pub root-10.0.0.1.pub
ln -s root-10.0.0.1.pub root-10.0.1.1.pub
ln -s root-10.0.0.1.pub root-10.0.2.1.pub
# rm $mntdir/var/lib/cfengine2/ppkeys/localhost.*
cd / ; chroot $mntdir /bin/sh -c "cfkey ; exit"
cp $mntdir/$cfdir/localhost.pub $cfdir/root-$ip.pub
}
function do_xen_config() {
echo "Creating XEN configuration file for ${name}..."
disk="'phy:$vg/$lv_root_name,hda1,w', 'phy:$vg/$lv_swap_name,hda2,w'"
if [ $make_data == 1 ] ; then
disk="[ ${disk}, 'phy:$vg/$lv_data_name,hda3,w' ]"
else
disk="[ ${disk} ]"
fi
# restart values: always, onreboot, never
cat > $xen_cfgdir/config-$name << _END_
# Autogenerated by: $header
#
kernel = "/boot/vmlinuz-2.6-xenU"
ramdisk = "/boot/initrd-2.6-xenU"
memory = $xen_mem
name = "$name"
vif = [ 'ip=$ip' ]
disk = $disk
root = "/dev/hda1 ro"
extra = "2"
restart = 'onreboot'
_END_
}
case "$go_from" in
lv)
do_LV_and_FS
do_base_system
do_base_config
do_xen_config
;;
base)
do_base_system
do_base_config
do_xen_config
;;
config)
do_base_config
do_xen_config
;;
xen)
do_xen_config
;;
*)
echo "Unknown option. Aborting..."
exit 1
esac
/bin/umount -l $mntdir
echo "To fire up your new domain: xm create -c $xen_cfgdir/config-$name"
exit 0
[ Parent ]
Or you could just use xen-tools, a pre-packaged tool which will setup and configure an installation of Sarge/Etch/Sid/CentOS4/Gentoo easily.
This will also do the post-install configuration, etc, and also provide you with the simple facility to run post-install hooks of your own choosing. (Supplied hooks support installing X + VNC, or removing packages which aren't desired.)
[ Parent ]
Sure, but where is the fun in that? ;-)
It would definately be an option, but xen-tools is currently only in testing/unstable and I only run stable and backports. Normally I would run testing but due to the fact that I am on this p*ss poor connection (serves me right for chosing a job in a 3rd world country), there is simply no way I can keep up with the torrent of new packages in testing.
Furthermore, these scripts tie in with my other normal sysadm scripts which I needed to write anyway.
But thanks for the hint - will take a look at xen-tools when etch goes gold.
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
Init, etc, don't really come into play here. As you suggest a kernel isn't installed.
You can install a new system via debootstrap (which I did a day or two agao) but you'll need to install a kernel and bootloader yourself after debootstrap has run.
debootstrap is more designed to install jails, or chroot systems, which can be used locally via the chroot command - and in that situation there is no "booting" as such.
[ Parent ]
[ Parent ]
Sure here is a brief example. It assumes you've installed Sarge into a directory and chrooted into it.
Setup the sources list to the distribution you're installing, in this case sarge:
echo 'deb ftp://ftp.debian.org/debian stable contrib' >> /etc/apt/sources.list
Install a kernel:
apt-get update apt-get install kernel-image-2.6.8-2-k7
(Choose an appropriate kernel for your system type. ie. 686 vs. k7)
Now install a bootloader:
apt-get install grub grub-install /dev/hda update-grub
That should be enough. Although it is worth checking that /boot/grub/menu.lst looks sane afterwards.
For a more detailed step you'd probably be as well to try it and ask for specific help if/when you have problems, or consult a mailing list.
[ Parent ]
[ Parent ]
Generally people don't install a kernel package in a chroot environment.
Still if you do you'll want to make sure that /proc and /sys are mounted in the chrrot environment. Something like this:
mount --bind /proc /path/to/chroot/proc mount --bind /sys /path/to/chroot/sys chroot /path/to/chroot umount /path/to/chroot/sys umount /path/to/chroot/proc
With these two mounts the kernel script will be able to detect your root drive, etc, and install.
[ Parent ]
[ Parent ]
You appear to be confused about what is happening and what order to do things in.
1. Mount the partition.
2. Mount /dev + /sys inside it.
3. chroot.
4. Install the kernel.
5. Exit the chroot.
6. Unmount /dev + /sys.
Like so:
debian:~# mount /dev/hda3 /mnt/hda3 debian:~# mount --bind /proc /mnt/hda3/proc debian:~# mount --bind /sys /mnt/hda3/sys debian:~# chroot /mnt/hda3/ debian:~# ... install the kernel here ... debian:~# ... exit the chroot ... debian:~# umount /mnt/hda3/sys debian:~# umount /mnt/hda3/proc
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
No, I'm afraid not.
I usually make sure that they are not running to avoid problems before I exit though:
apt-get install apache2 /etc/init.d/apache2 stop
Tahts the best advice I have I'm afraid.
[ Parent ]
[ Parent ]
mv "$TARGET/sbin/start-stop-daemon" "$TARGET/sbin/start-stop-daemon.REAL" echo \ "#!/bin/sh echo echo \"Warning: Fake start-stop-daemon called, doing nothing\"" > "$TARGET/sbin/start-stop-daemon" chmod 755 "$TARGET/sbin/start-stop-daemon" ### ... mv "$TARGET/sbin/start-stop-daemon.REAL" "$TARGET/sbin/start-stop-daemon"
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]