Published: July 6th, 2024
Last updated: August 14th, 2024
It took me about a week to get Gentoo running on a Raspberry Pi 4B with full disk encryption, but I’m happy with the result and I learned a couple of things. I wanted to share my findings so no one else has to spend that much time trying to figure this out. :)
Note that this is a rough outline, not a step-by-step guide on how to install Gentoo.
Here are the things that my setup involves:
I could have also used a monitor and keyboard to set this up instead of a serial connection. I think this would have been significantly easier overall; however, since I’ll be using the Raspberry Pi headlessly, I wanted to know how to set it up without a keyboard and monitor.
I mainly followed Gentoo’s official Raspberry Pi Install Guide and Raspberry Pi4 64 Bit Install to get an initial Gentoo installation on a microSD card. However, I’d like to talk about some pain points I faced during this process.
The hardest parts about installation for me were getting the wireless networking and serial console to function. I’ll cover each separately.
Functional wireless networking required getting a couple of things right:
Downloading the firmware, placing it in the appropriate locations,
and creating a couple of symbolic links. This is covered in the
Gentoo
wiki.
I found referencing the Raspberry Pi OS directory listing for
/usr/lib/firmware/brcm/ helped me to see what the symbolic link
naming scheme was.
Here’s what a working /usr/lib/firmware/brcm/ looks like for me
(though note that I haven’t tested bluetooth):
$ ls -lh /usr/lib/firmware/brcm/
total 704K
-rw-r--r-- 1 root root 63K Jun 24 12:07 BCM4345C0.hcd
lrwxrwxrwx 1 root root 13 Jun 24 12:07 BCM4345C0.raspberrypi,4-model-b.hcd -> BCM4345C0.hcd
-rw-r--r-- 1 root root 629K Jun 24 12:04 brcmfmac43455-sdio.bin
-rw-r--r-- 1 root root 2.7K Jun 24 12:05 brcmfmac43455-sdio.clm_blob
lrwxrwxrwx 1 root root 22 Jun 24 12:06 brcmfmac43455-sdio.raspberrypi,4-model-b.bin -> brcmfmac43455-sdio.bin
lrwxrwxrwx 1 root root 27 Jun 24 12:06 brcmfmac43455-sdio.raspberrypi,4-model-b.clm_blob -> brcmfmac43455-sdio.clm_blob
lrwxrwxrwx 1 root root 22 Jun 24 12:06 brcmfmac43455-sdio.raspberrypi,4-model-b.txt -> brcmfmac43455-sdio.txt
-rw-r--r-- 1 root root 2.1K Jun 24 12:05 brcmfmac43455-sdio.txt
Installing software for networking, such as net-misc/dhcpcd and
net-misc/wpa_supplicant, and configuring the software as needed.
These things are documented in the Gentoo
handbook.
Despite the second step appearing straightforward, I ran into an issue fairly quickly with this: how do I install networking tools, when I need those same tools to configure my networking?
For many people, the easiest solution would be to plug in an Ethernet
cable and install things that way. I also read that it’s possible to
download the distfiles and copy them over since in theory, emerge
won’t need to download anything.
I already knew that I probably wouldn’t be able to use Ethernet for reasons, so I tried the distfiles method and received this error:
root@nasberry /etc/portage # emerge --ask net-misc/dhcpcd net-wireless/wpa_supplicant
!!! /etc/portage/make.profile is not a symlink and will probably prevent most merges.
!!! It should point into a profile within /var/db/repos/gentoo/profiles/
!!! (You can safely ignore this message when syncing. It's harmless.)
!!! Your current profile is invalid. If you have just changed your profile
!!! configuration, you should revert back to the previous configuration.
!!! Allowed actions are limited to --help, --info, --search, --sync, and
!!! --version.
After looking into it more, I realized that /var/db/repos/gentoo
didn’t exist… I’d probably need to run emerge-webrsync to get it,
which requires networking.
What worked for me was to plug the microSD card into a USB adapter, mount the root filesystem on my laptop, create a QEMU chroot, and then compile the software I needed inside the chroot using my laptop’s networking.
To get the serial console working, I did these things:
I made sure that enable_uart=1 was present in /boot/config.txt. I
found this in the Raspberry Pi documentation for
config.txt
I added console=tty console=serial0,115200 to /boot/cmdline.txt.
cmdline.txt is documented
here.
Note that the order is quite important because whichever
console= entry is last is the one that /dev/console is sent to. I
found this out because mine were in the wrong order and the LUKS
decryption prompt, as well as OpenRC output, were never being sent to
my serial console.
More information is available at the kernel.org documentation for the serial console.
For early console
support,
I added earlycon=uart8250,mmio32,0xfe215040 to /boot/cmdline.txt.
To get a tty, I uncommented the line in /etc/inittab that points to
the /dev/ttyS0 device.
# SERIAL CONSOLES
s0:12345:respawn:/sbin/agetty -L 115200 ttyS0 vt100
#s1:12345:respawn:/sbin/agetty -L 115200 ttyS1 vt100
At this point, a fully functioning Gentoo installation with wireless networking and serial console access was present on the microSD card. The next challenge was getting the root filesystem transferred and working on an encrypted SSD.
The idea is that the Raspberry Pi 4B will read firmware files and
everything else needed to boot from the unencrypted EFI partition
present on the microSD card. /boot/config.txt and /boot/cmdline.txt
contain configuration entries and parameters relevant to the boot
process.
I securely wiped the disk by overwriting the SSD with random data to hinder cryptanalysis and performed a memory cell clearing (also known as “secure erase”).
After that, I created a LUKS volume with LVM inside of it on the SSD. Then I made filesystems on there as well.
I made sure that support for relevant tech was built into the kernel. In my case, this included LVM, dm-crypt, and XFS, as these are all needed during the boot process.
There’s a choice between sys-kernel/raspberrypi-sources and
sys-kernel/gentoo-sources. I can confirm that both work, although I
started with sys-kernel/raspberrypi-sources first.
sys-kernel/gentoo-sources does lack some functionality like device
tree overlay support, but it works for my use case.
It might be possible to configure other kernels to work, like the distribution kernel, but I haven’t tested that.
I followed instructions presented in the Linux kernel section of the Raspberry Pi docs, adjusting where necessary:
Create a backup of the previous kernel image if present.
# cp /boot/kernel8.img /boot/kernel8.img.bak
Change directory to kernel sources.
# cd /usr/src/linux
Build the kernel. The Raspberry Pi docs recommend setting make jobs to 1.5x the number of processors.
# make -j6 Image.gz modules dtbs
Install the kernel modules. Do this before generating the initramfs, since the initramfs uses the modules created from this step.
# make -j6 modules_install
Run make install to generate the initramfs since it’ll be needed to
decrypt and mount the root filesystem. -j6 doesn’t appear to provide
any benefit here.
# make install
Copy necessary files to the correct locations.
# cp arch/arm64/boot/Image.gz /boot/kernel8.img
# cp arch/arm64/boot/dts/broadcom/*.dtb /boot/
# cp arch/arm64/boot/dts/overlays/*.dtb* /boot/overlays/
# cp arch/arm64/boot/dts/overlays/README /boot/overlays/
/etc/fstab needs to be adjusted to include the NVMe disk instead of the
SD card.
#/dev/mmcblk0p3 / ext4 defaults 0 0
/dev/gentoolvm/root / xfs defaults 0 1
/boot/config.txt needs to be adjusted. kernel8.img will be
automatically selected as the default kernel, but we need to add the
initramfs in.
initramfs initramfs-6.6.31_p20240529-raspberrypi-v8.img followkernel
/boot/cmdline.txt needs to be changed so it provides information on
where the root filesystem is located (root=UUID=...), what LUKS volume
to decrypt (rd.luks.uuid=...), and what LVM volume group to activate
(rd.lvm.vg=lvmgroupname). UUIDs can be found with blkid.
root=UUID=... rd.luks.uuid=... rd.lvm.vg=gentoolvm
I made a full system backup of the Raspberry Pi’s microSD card with bsdtar.
# bsdtar \
--acls \
--xattrs \
--zstd \
--exclude='/dev/*' \
--exclude='/proc/*' \
--exclude='/sys/*' \
--exclude='/tmp/*' \
--exclude='/run/*' \
--exclude='/mnt/*' \
--exclude='/media/*' \
--exclude='/lost+found/' \
-cvaf /home/user/backup.tar.zst \
/
Then I unpacked it onto the SSD. The LUKS partition was decrypted and
the filesystems were mounted at /mnt.
# cd /mnt
# bsdtar --acls --xattrs -xvpf /home/user/backup.tar.zst
After rebooting, I was presented with the decryption prompt and was able to boot the rest of the system.
The kernel provided by the Raspberry Pi foundation is unsupported
security-wise by Gentoo. So I wanted to switch to
sys-kernel/gentoo-sources. This is a relatively straightforward
process, fortunately.
Change directory to the new kernel sources.
$ cd /usr/src/linux-6.6.30-gentoo
Copy the working config over.
# cp /usr/src/linux-6.6.31_p20240529-raspberrypi/.config .
Update the current kernel configuration so it works with this kernel.
# make oldconfig
Follow the process from before to build the kernel, except ignore
copying the overlay files since sys-kernel/gentoo-sources doesn’t
create them. Once that’s done, /boot/config.txt will need to be
updated so that it points to the correct initramfs file.
Another thing I had to do is update /boot/cmdline.txt so that it
contains 8250.nr_uarts=1, otherwise the serial console fails to
initialize
properly.
This setup won’t prevent a determined adversary from removing the microSD card and tampering with anything on the boot partition (the kernel; the initramfs; the firmware), nor does it prevent hardware level attacks. An attacker could theoretically leverage these to obtain the encryption password entered on boot. But the encryption does protect the data itself when the Raspberry Pi is powered off. Besides, a targeted and sophisticated attack like that isn’t currently in my threat model. This is more about raising the level of difficulty to recover the data.
If those types of attacks ever did become part of my threat model, perhaps I’d look into Secure Boot on the Raspberry Pi.