Previously, for the 1st task in The Eudyptula Challenge, we built a Loadable Kernel Module (LKM) that adds "Hello World" to our kernel's message buffer any time the module is installed. You can read my notes all about that here if you want to learn more.
Or, if you wish to work on The Eudyptula Challenge yourself before you read my notes (recommended), you can use my git repository, which has all 20 tasks and the code I used to complete each one.
For this challenge, the 2nd task of The Eudyptula Challenge, we will use the Kbuild system once again. This time we will use the Kbuild system to compile the latest version of the Linux Kernel. Then after compiling it (this takes some time), we install and boot from it.
Now that you have written your first kernel module, it's time to take off the training wheels and move on to building a custom kernel. No more distro kernels for you, for this task you must run your own kernel. And use git! Exciting isn't it! No, oh, ok...
The tasks for this round is:
- download Linus's latest git tree from git.kernel.org (you have to figure out which one is his, it's not that hard, just remember what his last name is and you should be fine.)
- build it, install it, and boot it. You can use whatever kernel configuration options you wish to use, but you must enable CONFIG_LOCALVERSION_AUTO=y.
- show proof of booting this kernel. Bonus points for you if you do it on a "real" machine, and not a virtual machine (virtual machines are acceptable, but come on, real kernel developers don't mess around with virtual machines, they are too slow. Oh yeah, we aren't real kernel developers just yet. Well, I'm not anyway, I'm just a script...) Again, proof of running this kernel is up to you, I'm sure you can do well.
Hint, you should look into the 'make localmodconfig' option, and base your kernel configuration on a working distro kernel configuration. Don't sit there and answer all 1625 different kernel configuration options by hand, even I, a foolish script, know better than to do that!
After doing this, don't throw away that kernel and git tree and configuration file. You'll be using it for later tasks, a working kernel configuration file is a precious thing, all kernel developers have one they have grown and tended to over the years. This is the start of a long journey with yours, don't discard it like was a broken umbrella, it deserves better than that.
Just like mechanics, plumbers, and all the other skilled trades, having the right tools for the job makes a world of difference. This "tools maketh man" proverb rings true even for software developers, even though most of our tools are less tangible.
The tools we will need to properly build the kernel vary wildly depending on many factors, like your hardware, kernel version, how we plan to install the kernel, as well as the linux distribution on your computer. All the minimum requirements we will need to build the kernel are listed in the documentation here, chances are though you will need to use more.
The tools I needed to build version 5.8 of the kernel on my (charmingly retro) Lenovo T430, running Ubuntu's 18.04 LTS (Bionic Beaver) release, are easily installed with:
$ sudo apt install build-essential libncurses-dev bison flex libssl-dev libelf-dev
When you first navigate to the kernel's git repositories, you will be greeted with hundreds of projects all having something to do with archiving or improving the Linux Kernel in some way. To help simplify the process of finding Linus' repository, which has the latest patches, we can search through the projects listed (using Ctrl+F) for "torvalds/".
Once you have found the URL, we can use
git to download a copy of Linus' Linux repository using the typical
git clone command. If you are unsure about how to clone a repository, the git documentation has a great article about getting started with git.
$ git clone https://git.kernel.org/pub/scm/linux ...
With a copy of the kernel's source code in hand, our next task is to set the many thousands of configuration options needed for the Kbuild System to properly compile our kernel. Each configuration option, like the
CONFIG_LOCALVERSION_AUTO the Little Penguin mentioned in the challenge message above, informs the Kbuild System of the drivers and features needed to be installed to function properly with our computer's hardware.
However, instead of setting roughly 8700 config options by hand, one-at-a-time (a very error prone, time consuming process), we can copy the configuration file from the linux distribution currently running on our computer as a sort of starting point.
The first step in copying our distribution's configuration file is to find it in our
There are multiple configuration files inside our
/boot directory. The one our computer is using depends on the kernel version our computer is using. For example my computer, using kernel version
4.15.0-112, will have a configuration file named
config-4.15.0-112-generic in the
To copy your distribution's config file into the root directory of the kernel use the this command:
$ cp /boot/config-`uname -r` .config
yes to any new configuration options added between the release of our distribution's kernel (they usually lag behind a few versions) and the bleeding edge we've downloaded from Linus.
$ yes "" | make oldconfig
Configuring By Hand
While not technically needed to complete this eudyptula challenge, this is a good place to talk about how we can customize our newly copied configuration file for our kernels. While there are a few, largely similar, commands that help us edit the
.config file, the command I prefer is a ncurses based menu that can run on most systems or on remote servers if wanted.
$ make menuconfig
It produces a color menu, full of radio-lists and dialogues, that looks something like this:
Each menu option allows us to enable and disable certain features in our kernels, along with anything that depended on that option. For example, if we disabled Networking, all network-related options, like Amateur Radio, will also be disabled.
The full list of commands used to edit the kernel's configuration file is available to you in the kernel documentation, if you want to learn more. They largely use different GUIs to edit our
With all the preparations complete, we can start to work on compiling the kernel into a compressed image ready for our computers. Just like in userspace applications, a simple
make command is all that is needed to build our kernel:
$ make -j `getconf _NPROCESSORS_ONLN` CONFIG_LOCALVERSION_AUTO=y LOCALVERSION=-eudyptula
While a bare
make command, without the added arguments, will also work to build our kernel, however, the kernel is such a large project, using the
-j flag to compile multiple files at the same time will dramatically reduce the total time needed to compile.
We are also passing two extra configuration options with the
make command above:
Both of these configuration options will add information to our kernel's version string, helping us prove to The Little Penguin we installed the latest kernel published by Linus Torvalds.
CONFIG_LOCALVERSION_AUTOoption will append enough characters from the latest commit id to be unique in our kernel's version name. For me this was
LOCALVERSIONwill also add
-eudyptulato that same version string
producing a kernel named something like this:
Our last and final step to completing this challenge is to install the kernel along with its supporting modules into our
/lib/modules folders. This will allow our boot loader, GRUB, to properly initialize our freshly compiled linux kernel the next time we reboot.
Each distribution of linux installs the kernel in their way. For example Arch Linux's documentation tells us to manually copy (with
cp) and not to use
make install to install the kernel in our
/boot directory. All this to say "your mileage may vary".
For most linux distributions we can install our kernel with a simple
$ sudo make modules_install install
Along with installing all the kernel's modules and the kernel itself, this command will also update our
/boot/grub/grub.conf files, so we don't have to manually update GRUB ourselves.
If your linux distribution has trouble with the
make install commands above, an alternative is to bundle our linux kernel into a package. This method is slower (a real pain when developing drivers) however, packages usually have more success installing on certain systems.
For Debian and Ubuntu based distributions we can use
deb-pkg to generate the packages:
$ make deb-pkg
This command will generate up to 5 Debian packages:
linux-image-versionwhich contains the kernel image and the associated modules.
linux-headers-versionhas the header files required to build external modules.
linux-firmware-image-versioncontains the firmware files needed by some drivers (this package could be missing when you build from the kernel sources provided by Debian).
linux-image-version-dbgwhich contains the debugging symbols for the kernel image and its modules
linux-libc-devholds headers relevant to some user-space libraries.
All of these packages can easily be installed with a simple
sudo dpkg -i ../*.deb
And that's it! Reboot you computer, and run
uname -r and you should see something like:
$ uname -r linux-image-5.8.0-rc4-eudyptula-00381-g1df0d8960499
This output will prove to The Little Penguin you've successfully compiled and installed your first (of many I'm sure) custom linux kernel. Great Job!