Booting Ubuntu 10.10 Server from software RAID

I recently switched my home server from Windows Server 2003 to Ubuntu 10.10. The machine is an older Shuttle box I’ve had tucked away in various places over the past few years.

The plan was to disable the BIOS “fakeRAID”, install Ubuntu on one of the drives, copy my data over to it and then have the other disk join the new Linux software RAID array.  I figured out the procedure on a VirtualBox before attempting it on my actual machine.

Install the server with NODMRAID and partition the drive with no LVM, that is partition with “Guided – use entire disk”. We can’t have devicemapper getting in the way.

Install on first hard disk (/dev/sda).  If there’s any data you need off the other disk, now’s the time to mount it and copy it over.

To start, we need the mdadm package.

sudo apt-get install mdadm

You need two modules to be loaded at boot-time, raid1 and md.  In the kernel included with Ubuntu 10.10, md was compiled in, so I didn’t need to worry about loading it as a module.

I verified this by checking the kernel’s config:

fgrep CONFIG_MD /boot/config-$(uname -r)

Add the module raid1 to the /etc/raid1

sudo nano /etc/modules

Let’s load the raid1 module

sudo modprobe raid1

To ensure it is loaded:

lsmod | grep raid1

So, it seems we’re ready for prepare our other disk. A neat little trick is to use sfdisk to dump the partition table of one disk to configure another.

sudo sfdisk -d /dev/sda > sda.out
sudo sfdisk /dev/sdb < sda.out

but sfdisk responds with “I don’t like these partitions – nothing changed.”  Well, excuse me, sfdisk, but I don’t like your attitude. (Yes, I’m arguing with a disk utility now) So, I added the –force flag.  It grudgingly copied the partition table for me.  I verified it by comparing the output of sfdisk -l /dev/sda and sfdisk -l /dev/sdb.

Change the partition type of the Linux partition(s) to “Linux raid autodetect”

sudo sfdisk --change-id /dev/sdb 1 fd

We’re ready to create the array. We specify a RAID 1 array with 2 devices. The first is missing and the second is /dev/sdb1:

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1

Now, we need an mdadm.conf. It seems like the way to do this is with

sudo cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.oldconf
sudo cp /etc/mdadm/mdadm.conf mdadm.conf
mdadm --detail --scan >> mdadm.conf

It will put in a parameter metadata=00.90 that will cause warnings later. Apparently, this is a bug and it is safe to remove it.  After editing it, I copied back to where it belongs.

sudo cp mdadm.conf /etc/mdadm/mdadm.conf

Now, we format the raid volume

mkfs -t ext4 /dev/md0

Find out the UUID of the array

sudo blkid

Make a copy of /etc/fstab in your home directory and change root to mount to the UUID of the array.  Don’t change the real /etc/fstab quite yet.

Add a custom setup in GRUB2, I’m going to edit a copy of this file in my home directory too.

cp /etc/grub.d/40_custom 09_swraid1_setup
nano 09_swraid1_setup

Details here, but here’s what I added to my file:

menuentry 'Ubuntu, with Linux 2.6.35-22-server' --class ubuntu --class gnu-linux --class gnu --class os {
        recordfail
        insmod raid
        insmod mdraid
        insmod ext2
        set root='(md0)'
        linux   /boot/vmlinuz-2.6.35-22-server root=/dev/md0 ro   quiet
        initrd  /boot/initrd.img-2.6.35-22-server
}

This differs from the linked article in that ’2.6.35-22-server’ is my kernel release version (the one that ships with Ubuntu server) and I’m booting from md0.

Back to the physical console.

sudo telinit 1

Copy the above temp files (fstab and 09_swraid1_setup) to their actual locations

update-grub

Adjust ramdisk

update-initramfs -u

Ensure GRUB2 is on /dev/sdb and sda

grub-install /dev/sdb

grub-install /dev/sda

Mount the array on /tmparray

mount /dev/md0 /tmparray

rsync -aqxP / /tmparray

Restart and we have now booted off the array!

Change the partition type of sda now

sudo sfdisk --change-id /dev/sda 1 fd

Add it to the array:

mdadm --add /dev/md0 /dev/sda1

Welcome to the party, /dev/sda! Let’s watch it sync:

watch cat /proc/mdstat

Our special GRUB entry is no longer needed

rm -f /etc/grub.d/09_swraid1_setup
update-grub
update-initramfs -u

Done.

I worked out the kinks in the virtual machine and documented it in a draft of this post.  I used it as my guide when performing the install on the actual machine. It worked perfectly.

Update, Feb 25, 2011: Clarification in response to comments.