Feedback: The bootloader is capable of booting NixOS 23.05 with mainline kernel 6.3.X+ #11
Description
I have been successfully running NixOS on my R5S board using the 1.0 release of the bootloader and the corresponding DTB in UEFI mode for a couple of weeks already (I have tested both GRUB and systemd-boot boot managers - both work just fine). I have /boot on an SD card used as EFI system partition (ESP), as having it on the eMMC did not work for some reason. The bootloader itself is installed on the eMMC, and the dtb file is also loaded from a FAT32 partition on eMMC (the partition is marked for boot, too). The OS and most files are on an NVMe drive (SK Hynix P31 Gold, which I recommend for this board due to high power efficiency. Do use thermal pads anyway, though).
See here for my adventures regarding that.
I should note that the kernel from the NixOS repo does not support PCI, HDMI, and LEDs on this chip with its default configuration, so the kernel needs to be recompiled with some kernel options changed. It is easy to do on NixOS, but it takes a lot of time (12-14 hours) on the board itself - so it is wise to "pin" the kernel packages version and update it from time to time for now. Still, it runs the mainline kernel which is a big win for these boards.
Thus, the main takeaway snippet (I think I should try to upstream some of these kernel configuration options to avoid kernel recompilations):
{ config, pkgs, lib, ... }:
{
config = lib.mkIf pkgs.stdenv.isAarch64 {
boot = {
# partial Rockchip related changes from Debian 12 kernel version 6.1
# Also, see here:
# https://discourse.nixos.org/t/how-to-provide-missing-headers-to-a-kernel-build/11422/3
kernelPatches = [
{
name = "rockchip-config.patch";
patch = null;
extraConfig = ''
PCIE_ROCKCHIP_EP y
PCIE_ROCKCHIP_DW_HOST y
ROCKCHIP_VOP2 y
'';
}
{
name = "status-leds.patch";
patch = null;
# old:
# LEDS_TRIGGER_NETDEV y
extraConfig = ''
LED_TRIGGER_PHY y
USB_LED_TRIG y
LEDS_BRIGHTNESS_HW_CHANGED y
LEDS_TRIGGER_MTD y
'';
}
];
initrd.availableKernelModules = [
## Rockhip
## Storage
"sdhci_of_dwcmshc"
"dw_mmc_rockchip"
"analogix_dp"
"io-domain"
"rockchip_saradc"
"rockchip_thermal"
"rockchipdrm"
"rockchip-rga"
"pcie_rockchip_host"
"phy-rockchip-pcie"
"phy_rockchip_snps_pcie3"
"phy_rockchip_naneng_combphy"
"phy_rockchip_inno_usb2"
"dwmac_rk"
"dw_wdt"
"dw_hdmi"
"dw_hdmi_cec"
"dw_hdmi_i2s_audio"
"dw_mipi_dsi"
];
};
# Most Rockchip CPUs (especcially with hybrid cores) work best with "schedutil"
powerManagement.cpuFreqGovernor = "schedutil";
};
}
Bootstrapping the installation required building a custom UEFI install image for me using nixos-generators project (it is easier to do that when u-boot and the DTB file are already on the board's eMMC). You will need the snippet above to build a usable installation image, too.
Another takeaway is to run irqbalance --oneshot on startup to spread interrupts processing between cores, otherwise it is too easy to overwhelm the first core with interrupts, especially on high network interfaces activity (use irqtop for monitoring).
On NixOS this can be achieved using this snippet:
{ config, pkgs, lib, ... }:
{
config = {
systemd.services."irqbalance-oneshot" = {
enable = true;
description = "Distribute interrupts after boot using \"irqbalance --oneshot\"";
documentation = [ "man:irqbalance" ];
wantedBy = [ "sysinit.target" ];
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
ExecStart = "${pkgs.irqbalance.out}/bin/irqbalance --foreground --oneshot";
};
};
};
}
Additionally to the above, I have the following in my configuration (one may, or may not need that depending on the needs):
{ config, pkgs, lib, ... }:
## NanoPi R5S-board specific configuration
# This board can be used with NixOS thanks only to this project:
# https://github.com/inindev/nanopi-r5/
# A lot of information is taken from there
{
config = lib.mkIf pkgs.stdenv.isAarch64 {
boot.kernelParams = [
"console=tty1"
"console=ttyS2,1500000"
"earlycon=uart8250,mmio32,0xfe660000"
];
# Let's blacklist the Rockchips RTC module so that the
# battery-powered HYM8563 (rtc_hym8563 kernel module) will be used
# by default
boot.blacklistedKernelModules = [ "rtc_rk808" ];
};
}
I have been running the device as LXC/libvirt/Docker host for weeks now without any issues. Thank you @inindev for your hard work! You made it possible!