[tracking] Raspberry Pi 3 boot status

[vielmetti]

This issue tracks efforts to boot linuxkit on the Raspberry Pi 3, using a 64-bit armv8 kernel.

It's a follow on from #1377 (ARM boot), except for a bit of added effort, there's the need to adapt to the Pi 3, and also some challenges in the development effort because of limited memory on that system.

Some related topics to knit this together:

• https://github.com/DieterReuter/rpi64-kernel/ - 64-bit Raspberry Pi 3 kernel from @DieterReuter
• https://github.com/WorksOnArm/cluster/issues/12 - proposal from @hairyhenderson and @eyz for support from @WorksOnArm to build on Packet.

Some of the discussion for this is happening in the Docker Community #linuxkit Slack channel, this tracking item will capture anything that emerges.

[rn]

I have some patches to the kernel config to enable RPi3 specific options. but it's very much WIP (only started today). I would prefer to use the stock upstream kernel instead of building one from https://github.com/raspberrypi/linux as it is pain to maintain separate kernel sources/builds

[eyz]

I heavily second @rn on this. Links that come to mind are:

• https://elinux.org/RPi_Upstream_Kernel_Compilation (walkthrough on building a kernel from mainline)
• https://github.com/raspberrypi/tools (kernel support for rPI)

I have successfully built a kernel using the first link, but have not tested it on hardware. There are modules in the .config for rPI 3 that are required, and not included in the default .config

[DieterReuter]

@rn absolutely true, using mainline LK is also my preferred way! But for the first shot it was easier to use an already working kernel.

[eyz]

Current efforts are likely:

• testing wireless and 100baseT
• moving to a rpi3 kernel built from the mainline kernel -- likely with rpi-specific kernel source added in from github.com/raspberrypi/tools.git

Challenges currently are:

• continuing to boot after migrating a working pkg/getty image (currently running in the init section) back into services and having working pkg/dhcpcd in the root image, as in the standard LinuxKit examples/minimal and examples/getty @ https://github.com/eyz/linuxkit/tree/rpi3 (will merge in as this progresses)
• wifi stability
• low RAM on the device, which currently prevents compiling on the rPI platform itself

[DieterReuter]

Here is a gist with the first bootlog of LinuxKit on RPi3 with some more details: https://gist.github.com/DieterReuter/15b00a4293e829644c52c86b2b5bcd06.

[rn]

I have it booting with a LinuxKit kernel and u-boot. man this is tedious to do...the iteration of building, copy to flash etc. Still don't have serial.

Still a lot of manual steps which I'll write up before integrating it with moby build.

[justincormack]

You can do network boot, uboot supports pxe...

[rn]

I know...but I only just got the whole shebang to work and don't have a PXE boot server setup at home :)

[rn]

Instructions for linuxkit/u-boot: https://gist.github.com/rn/340c185d826132122975bc9d7df76921
Also bits of status.

[eyz]

Here's a diff from a md5 manifest of mainline 4.9.53 vs md5 manifest of rpi 4.9.53 @ eyz/linux-rpi-diff@49e6ddf.

Given the ~ 546 different kernel tree files between mainline 4.9.53 and rpi 4.9.53 I am currently leaning towards a specific LinuxKit kernel package, but still recommend pursuing building from mainline in the fututre, perhaps more mid-term than short-term

Next up: I'm going to run the rpi kernel cross-compile build through strace and find only the files which are used in the build, and filter the above diff with this lost to hopefully find the minimal changes from mainline