As many people have already undertaken similar efforts and compiled them nicely into blog posts 1 2 3, I think being able to quickly create such an environment at hand is very meaningful. Here I’ll record the rough mechanism. I’ve compiled the results into scripts and uploaded them to Github.
Features
- Supports building kernels for major distributions like CentOS6, CentOS7, Ubuntu20.04, enabling practical applications
- Uses Busybox to deploy userland in memory, creating a pure and minimal environment on each boot
- SSH login and external network connectivity are possible, making it easy to verify operations involving integration with other systems
- Can reference internal kernel data through debugging with GDB
- Currently only supports x86/64
Building the Kernel
Building the kernel basically corresponds to writing the build configuration in a .config file and running the make command. Since .config is a text file, it can be edited with any suitable editor, but dedicated commands (make oldconfig, make defconfig, make menuconfig, etc.) are provided and are often used. There are upstream kernels and those modified by each distribution, but here I’ve set up an environment that can build all of the following kernels.
- upstream (kernel v2.6.39)
- centos6 (kernel v2.6.32-754.35.1.el6)
- centos7 (kernel v3.10.0-1160.13.1.el7)
- ubuntu20.04 (kernel v5.4.0-65.73)
Since compiling old kernels with recent GCC is difficult, I prepared dedicated Docker images for building for each kernel version. For example, I decided to build kernel v2.6.39 using the CentOS6 build environment.
FROM ghcr.io/buddying-inc/centos68:latest
RUN sed -i "s|#baseurl=|baseurl=|g" /etc/yum.repos.d/CentOS-Base.repo \
&& sed -i "s|mirrorlist=|#mirrorlist=|g" /etc/yum.repos.d/CentOS-Base.repo \
&& sed -i "s|http://mirror\.centos\.org/centos/\$releasever|https://vault\.centos\.org/6.10|g" /etc/yum.repos.d/CentOS-Base.repo
RUN yum install -y gcc perl glibc-static kernel kernel-devel \
autoconf zlib-devel zlib-static openssl-static openssl-devel
Create a Docker image named buildenv-v2.6.39 based on the above Dockerfile. This corresponds to the dedicated build environment. Then attach the Linux kernel source tree to the Docker container and run the make bzImage command.
sudo docker run --rm -v linux-v2.6.39:/tmp \
buildenv-v2.6.39 \
/bin/bash -c "cd tmp; make bzImage"
Building Userland
The userland is created based on Busybox to be self-contained on a RAM disk. Busybox is a single binary that includes basic CLI tools, and can behave as each command through symbolic links.
ls -la /bin/ls /bin/vi /bin/cat /bin/busybox
# -rwxr-xr-x 1 0 0 2332560 Jul 6 05:31 /bin/busybox
# lrwxrwxrwx 1 0 0 7 Jul 6 05:31 /bin/cat -> busybox
# lrwxrwxrwx 1 0 0 7 Jul 6 05:31 /bin/ls -> busybox
# lrwxrwxrwx 1 0 0 7 Jul 6 05:31 /bin/vi -> busybox
Also, since DHCP client and SSH server are configured, network connectivity from the virtual machine on QEMU to external networks and SSH login are possible. The build procedure is relatively long, so refer to https://github.com/bobuhiro11/understanding-the-linux-kernel/tree/main/busybox.
Running on QEMU
Boot the kernel image bzImage and RAM disk initrd on QEMU.
Kernel address space randomization is disabled by adding nokaslr to the boot parameters.
Port 22 of the virtual machine is used for SSH. This is mapped to port 10022 on the host side through port forwarding.
# Start the virtual machine
qemu-system-x86_64 -kernel linux-v2.6.39/arch/x86/boot/bzImage \
-m size=512 -initrd busybox/initrd --nographic \
--append "root=/dev/ram rw console=ttyS0 rdinit=/sbin/init init=/sbin/init nokaslr" \
-nic user,model=virtio-net-pci,hostfwd=tcp::10022-:22
# SSH login
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@localhost -p 10022
If you want to use the debugger GDB, start QEMU with -gdb tcp::10000 -S added to the startup options,
and attach from GDB as follows.
gdb --directory=./linux-v2.6.39 ./linux-v2.6.39/vmlinux \
-ex 'target remote localhost:10000'
That’s it. I think I’ve prepared an environment that can comprehensively build kernels from newer ones to the relatively old kernel explained in the book “Understanding the Linux Kernel, Third Edition”. I hope this is helpful to someone.