Wednesday 14 July 2010

Fun with Foreign Debian Bootstrapping

Yesterday I found myself booting Linux on a device with no attached permanent storage - all I had was several gigabytes of RAM and the ability to netboot it through TFTP. I had been using a very minimal root filesystem inside the kernel image, but I began to wonder if it would be possible to have an entire Debian installation in the ramdisk instead - the box certainly had enough RAM to fit a minimal installation.

Ordinarily one could just use debootstrap to set up a minimal Debian installation inside a directory and make a ramdisk from that, but this was further complicated by the fact that this was a PowerPC device. Debootstrap does have a --foreign option to perform the first part of the installation on a different architecture, but the --second-stage still needs to be run as root on native hardware and assumes that it is being run from within an existing Linux installation with a bunch of standard tools available to it.

The only machines I had root on were all x86 (other than the device in question, but the ramdisk I had been using had some limitations that would have complicated matters) and some other test boxes (which I would have had to wait to requisition). So instead I decided to do a partial debootstrap on my local x86 box and complete the installation using only my local x86 box and that partial image on the PowerPC box.

If you are following this article as a guide I should note that it assumes you are able to compile and boot your own kernel and have a decent familiarity with Linux in general.

So first, begin the debootstrap process, but use --foreign to only perform the first part of the bootstrapping process (NOTE: almost everything here needs to be run as root, signified by the # at the start of each line):

# mkdir deb-ppc
# debootstrap --arch=powerpc --foreign squeeze deb-ppc http://<mirror>/debian

After this command completes you have an incomplete Debian installation in deb-ppc - some basic tools are installed (but not configured) and some packages have been downloaded but not installed. I did not select any additional packages into the initial root disk at this stage, though had I been thinking ahead it would have been useful to also include openssh-server and rsync, but that was not a major setback for me. You might want to include them, and if you don't like vi or nano you might also want to install your console editor of choice. At the moment the root disk is not bootable, so let's fix that:

# ln -s /bin/bash deb-ppc/init

This still won't boot into a full Debian installation - after the kernel finishes it's initialisation and tries to spawn the init userspace process to take over booting, it will instead spawn an interactive shell which can be used to complete the bootstrapping process. Since I'm bundling this inside the kernel image as an initramfs as opposed to an initrd loaded separately, I link an interactive shell into /init. If you were doing this with an initrd you would instead link it to /initrd.

Before we can make a ramdisk image from that directory we need to save this script as from Documentation/filesystems/ramfs-rootfs-initramfs.txt in the kernel sources:

# Copyright 2006 Rob Landley <> and TimeSys Corporation.
# Licensed under GPL version 2

if [ $# -ne 2 ]
  echo "usage: mkinitramfs directory imagename.cpio.gz"
  exit 1

if [ -d "$1" ]
  echo "creating $2 from $1"
  (cd "$1"; find . | cpio -o -H newc | gzip) > "$2"
  echo "First argument must be a directory"
  exit 1

NOTE: when using this script be sure you are calling this script and not a separate program also named mkinitramfs from your distribution.

Let's bundle the root disk into a cpio image:

# ./ deb-ppc ramdisk.cpio.gz

Now you need to compile the kernel and netboot it - I'll leave the details of how to actually do that out of this article - there's plenty of good resources for that around already and the netboot procedure may vary depending on your setup (if you are netbooting at all). If you are doing this with an initramfs like I am you will need to point CONFIG_INITRAMFS_SOURCE to that image - once you have configured the kernel edit the .config file and remove the 'CONFIG_INITRAMFS_SOURCE=""' line. Then run make oldconfig which will ask you to set that option as well as some UID and GUI mapping (which you can leave as 0 since the image already should already have the correct ownership). After that you can run make and wait for the kernel to build. I'll also assume you know which zImage is the correct one to boot on your hardware.

Once you have successfully booted the kernel you should find yourself at a bash prompt. You should be aware that the environment is extremely limited at this point - for one thing there is no job control so don't try to spawn a process that you need to ctrl+c out of (I made the mistake of pinging a host to check that the network was up).

The debootstrap --second-stage did not work for me, so instead I completed the installation manually:

# export PATH=/usr/sbin:/usr/bin:/sbin:/bin
# dpkg --force-depends --install /var/cache/apt/archives/*.deb

A few things may complain during that and you may need to tell apt to fix up any problems:

# apt-get -f install

Now you will have a much more complete userspace - including vi. There's a few more things we need to do to get the system usable. Firstly, let's edit /etc/fstab and add an entry for /proc since so much userspace depends on it:

# vi /etc/fstab

proc /proc proc defaults 0 0

# mount /proc

Now we should probably get networking set up (I'm assuming you are using DHCP and your interface is eth0):

# vi /etc/network/interfaces

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

# vi /etc/hostname
# ifup lo
# ifup eth0

Do not make the mistake I made of checking if the interface is up by pinging something. You can run ifconfig to make sure your IP address looks right.

And set up apt (note /debian postfix in sources.list which isn't in the template provided by debootstrap - I spent around 10 minutes contemplating the 403 I was getting before I noticed that):

# vi /etc/apt/sources.list

deb http://<mirror>/debian squeeze main

# vi /etc/apt/apt.conf.d/10local
APT::Install-Recommends "0";
APT::Install-Suggests "0";

# apt-get update

Now you can install any additional packages you may need (if you didn't do this in the initial debootstrap), so let's install what we need to be able to copy our changes out of the machine (interactive SSH won't work just yet, but file copying will):

# apt-get install openssh-server rsync
# passwd

Note that if you are interacting with the machine via serial it may be a bit awkward to interact with the configuration for some packages (such as localepurge) so just install the bare essentials for the moment. After installing some packages it's probably a good idea to clean the apt cache since we are likely pretty tight on RAM:

# apt-get clean

Speaking of serial, if you are logging into the machine via serial (as I was) you may want to spawn a console on the serial line:

# vi /etc/inittab

T0:2345:respawn:/sbin/getty -L ttyS0 57600 vt100

Back on the x86 box we can now copy all those changes back into the ramdisk and make it actually boot Debian:

# rsync -avx <host>:/ deb-ppc
(NOTE: the x is important, otherwise /proc will be copied as well)
# rm deb-ppc/init
# ln -s /sbin/init deb-ppc/init
# ./ deb-ppc ramdisk.cpio.gz
(again, the mkinitramfs from the kernel doc, not a distro)

Again, compile the kernel and boot it. You will need to do this last part every time you make a change in the ramdisk that you want to make persistent.

Once booted you will be able to interactively SSH into it and will find you now have a complete Debian installation you can do whatever you like with within the constraints of the available RAM. With full SSH, job control and proper TTY management you can now perform some changes that would have been a little tricky earlier, such as reconfiguring any packages you couldn't configure properly earlier (tzdata for me) and stripping out unneeded locales (this messed up a little for me since locales wasn't installed before localepurge. I haven't tested this and it's probably longer than it needs to be, but I think it will work):

# apt-get install locales
# locale-gen en_AU-UTF-8
# dpkg-reconfigure locales
# apt-get install localepurge
# localepurge
# apt-get clean

You might also want to strip out some unneeded packages, for example with:

# apt-get purge logrotate mac-fdisk rsyslog yaboot info install-info man-db manpages nano

Remember to follow the above instructions to make those changes persistent if you are happy with them. Later I'll probably play around with docpurge (from maemo) and look at other ways of reducing the size of the image (disabling logging is probably a good place to start).

If you're after some further reading on booting the kernel with initial ramdisks, check out Documentation/early-userspace/README and Documentation/filesystems/ramfs-rootfs-initramfs.txt in the kernel source.