Running bootc install to-filesystem broke the host's UEFI boot entry

[bcl]

While doing some experiments with bootc, and the docs here - https://bootc-dev.github.io/bootc//bootc-install.html#more-advanced-installation clearly say Note that while --privileged is used, this command will not perform any destructive action on the host system. Which I made the mistake of trusting.
It ran efibootmgr and removed the boot entry, and then appeared to recreate it. But now the system won't boot.

So, at the least the docs need to have a warning.

And it should not be running efibootmgr, ever. The filesystem or block device being installed to may have no association with the host hardware as was the case here.

[bcl]

The problem appears that it overwrote grub.cfg, I think, with a very minimal version.
Luckily I had a grub.cfg.rpmnew saved and was able to recover the system without resorting to backups.

[cgwalters]

Thanks, indeed this is an important bug.

And it should not be running efibootmgr, ever.

We do want to do so in the to-existing-root flow actually, which is why the code is there.

In practice also, while we definitely want to support e.g. loopback mounting and running this as root...in my experience the vast majority of time one wants to build an operating system one wants to be able to run it conveniently (locally) - and that means virt. And if you have virt, you might as well isolate components with it. On that topic, https://github.com/bootc-dev/bcvk/ is one convenient tool that is designed for bootc - it helps isolate the to-disk flow which is just sugar for spawning an ephemeral run, where one can run arbitrary code including ones that do to-filesystem.

---

On a different angle here, a general best practice with --privileged container flows is using -v /sys:/sys:ro, we have that documented at https://docs.fedoraproject.org/en-US/bootc/provisioning-container/#_privileged_user_namespaces_and_sys

Are you running bootc install to-filesystem in a container or outside of one?

I think while we definitely want to fix the to-filesystem flow by telling bootupd not to touch the firmware, it also makes sense for us to force the readonly /sys when we're not operating on the actual target root.

[bcl]

I was running it using podman, my startup script looks like this, which has /sys as ro.

#!/usr/bin/bash
sudo podman run --rm --privileged --pid=host -it \
-v /var/lib/containers:/var/lib/containers \
-v /dev:/dev \
-v /sys:/sys:ro \
-v $PWD/rootfs:/rootfs \
--security-opt label=type:unconfined_t $1 bash

The container it was running is super simple:

FROM quay.io/fedora/fedora-bootc:42
RUN dnf install -y tmux vim-enhanced && dnf clean all
COPY my-user.conf /usr/lib/sysusers.d
COPY ssh.pub /root/.ssh/authorized_keys
# Make sure it all looks ok
RUN bootc container lint

IIRC I had to mkdir -p /rootfs/boot/efi in my empty /rootfs directory and then ran bootc install to-filesystem /rootfs (without making the empty /boot/efi it complained there was no boot and quit. I didn't capture the output but I saw it run efibootmg, it deleted the boot entry and recreated it. I don't remember seeing anything about grub, but it seems like it mounted my ESP and rewrote grub.cfg in /boot/efi/EFI/fedora/

If I'd been using install to-disk I'd have been more cautious and done this inside a VM, but the combination of assuming install to-filesystem was just writing files, and the docs saying it won't touch the host lulled me into a sense of complacency :)

[HuijingHei]

Test with -v /sys:/sys:ro, it still runs efibootmgr to create the boot entry. Check in container /sys is mounted as ro, but efivarfs is mounted as rw.

# podman run --rm -it --privileged -v /sys:/sys:ro --pid=host --security-opt label=disable quay.io/fedora/fedora-bootc:42 bash
bash-5.2# mount | grep /sys 
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)

The workaround is with -v /sys/firmware/efi/efivars:/sys/firmware/efi/efivars:ro, check in container efivars is mounted as ro, so will skip running efibootmgr.

# podman run --rm -it --privileged -v /sys/firmware/efi/efivars:/sys/firmware/efi/efivars:ro --pid=host --security-opt label=disable quay.io/fedora/fedora-bootc:42 bash
bash-5.2# mount | grep /sys
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (ro,nosuid,nodev,noexec,relatime)

[cgwalters]

So today adding --generic-image should suppress this behavior.

It'd be a somewhat breaking change, but we could probably detect the case where the target filesystem is not the root filesystem and default to doing that.