Building a USB stick to boot machines to Debian Linux Live desktops

neilschelly's picture

The Problem
I had a goal. It sounded simple. I wanted to make a classroom full of machines boot up to Live Debian Linux desktops, so they could maintain their Windows installations unchanged. I am aiming to utilize these machines in an upcoming DynEdu class I'm working on at Dyn. This is a classroom full of relatively simple machines that will use WiFi to connect to the network. What I learned can hopefully help just about anyone who wants to make custom Debian bootable USB sticks.

The end goal of my project today should have been simpler to accomplish. Debian releases live CD images after all, and those worked just like you'd expect. Unfortunately, since nearly every WiFi chipset (and many wired ethernet drivers too) requires binary firmware to be loaded in order to work in Linux, that wasn't enough. Those firmwares, typically qualify as non-free in Debian parlance, meaning they exist under licensing that makes distribution of them with Debian Linux a violation of the principles Debian is founded upon.

Googling to the Rescue
A bit of googling led me to find that there are unofficial Debian CD images built specifically to cater to my need. Debian's wiki includes a page specifically about Firmware solutions to work around this distribution problem. Unfortunately, I cannot figure out how those images are supposed to work. I booted up with one of them, but didn't see any additional firmware support.

More Googling led to many potential solutions, some of which had to do with making bootable CDs, custom Live CDs, adding the firmware files to the standard install mediums, etc. It took a long while to work through my options. I was able to use a tool called `isomaster` to add the firmware files to the CD images pretty easily, but found they wouldn't boot from USB anymore. They booted fine if I used them in VirtualBox, but not if I actually rebooted a real machine and tried booting from them.

And then, finally...
When all is said and done, this is what I did and it worked well. The keywords from a coworker were to look for the `live-build` tool. This is the tool that Debian uses for it's live image builds. The config and source files for these builds are available via a Git repository. This tool is documented, and I found that was able to get me very close to what I wanted. Here's the steps I followed

# sudo apt-get install live-build git
There may be a few other packages necessary for the building of CD images, things like genisoimage, debootstrap, syslinux, etc. I don't know what dependencies I already had installed, or what dependencies would be covered just by the packages above, but these are the ones I needed to get going.
# git clone git://live-systems.org/git/live-images.git
This Git repo contains all the configuration and scripts necessary to build the live CD images used by Debian. These will be the basis for the end goal.
# cd live-images/images/{gnome-desktop,kde-desktop,lxde-desktop,rescue,standard,xfce-desktop}
Start in one of these trees, according to which image you want to use as your jumping off base.
# sudo lb config --archive-areas "main contrib non-free" --parent-archive-areas "main contrib non-free"
In my case, I needed to expand the sources that would be used for bringing packages into the live CD. In particular, I needed the non-free repos to ensure I could get the non-free firmwares. This command is the key to my solution and your needs may differ. You can add more sources, change the architecture you're building for, etc. Take a look at `man lb_config` to see what else you've got available to you. I also found several how-tos that pointed at this command, but only included the adjustment to --archive-areas, but not --parent-archive-areas. I needed both. I'm honestly not too clear on the difference between them, but I know that both of them accomplished what I needed and only the first did not. I am curious, but I'm not sure if I'll come back and edit this if I get around to investigating deeper here since I already spent too long on this.
# $EDITOR config/package-lists/morepackages.chroot
Use your $EDITOR of choice and create a packages file of whatever name you choose. Add a list of packages you want to include in your custom build here as a list. You have some additional flexibility here for conditional statements and logic too, if you need it. I attached the file I used here to accomplish the installation of non-free firmware packages, if someone is looking for exactly that fix.
# sudo lb build
Run this and wait for a good while. I think it took me an hour or so. But in the end, I had a binary.hybrid.iso file I could write to a USB key that accomplished what I wanted.
# dd if=binary.hybrid.iso of=/dev/sdb bs=4M
This will write the ISO image file to a USB key. You may want to verify that /dev/sdb is where your USB key is located before running this. Otherwise, you will destroy the data on whatever block device you have at /dev/sdb. Please be careful.

I hope that helps someone out. I could have saved a lot of time today if any of the blog posts on this topic had some of these pointers. It's worth pointing out also that the Debian manuals for live-build are pretty awesome at describing this whole process. At best, this blog post is a tldr; of that manual for one specific use case.