binfmt_misc/binfmt-support not working as expected

[cinderblock]

Windows Build Number

Microsoft Windows [Version 10.0.19043.1110]

WSL Version

• WSL 2
• WSL 1

Kernel Version

Linux version 5.4.72-microsoft-standard-WSL2 (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Wed Oct 28 23:40:43 UTC 2020

Distro Version

Ubuntu 20.04

Other Software

qemu
qemu-user-static
binfmt-support

Repro Steps

I don't know what I did to get my system into this state. It was working fine. I think I rebooted with a Windows Update, but not sure.

sudo apt-get reinstall binfmt-support

Expected Behavior

/proc/sys/fs/binfmt_misc/ to be filled with qemu-* files. Like this:

cameron@uberfall:~$ ls -l /proc/sys/fs/binfmt_misc/
total 0
-rw-r--r-- 1 root root 0 Jun 24 09:01 python2.7
-rw-r--r-- 1 root root 0 Jul  1 06:07 python3.8
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-aarch64
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-alpha
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-arm
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-armeb
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-cris
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-hppa
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-m68k
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-microblaze
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-mips
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-mips64
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-mips64el
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-mipsel
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-mipsn32
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-mipsn32el
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-ppc
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-ppc64
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-ppc64abi32
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-ppc64le
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-riscv32
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-riscv64
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-s390x
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-sh4
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-sh4eb
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-sparc
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-sparc32plus
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-sparc64
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-xtensa
-rw-r--r-- 1 root root 0 Jun 24 09:01 qemu-xtensaeb
--w------- 1 root root 0 Jul  1 06:07 register
-rw-r--r-- 1 root root 0 Jul  1 06:07 status

Actual Behavior

$ ls -l /proc/sys/fs/binfmt_misc/
total 0

Diagnostic Logs

Fresh boot of wsl (after wsl --shutdown)

cameron@Nook:~$ uptime -p
up 0 minutes

Some binfmt is loaded, missing qemu.

cameron@Nook:~$ ls -l /proc/sys/fs/binfmt_misc/
total 0
-rw-r--r-- 1 root root 0 Jul 14 15:09 WSLInterop
--w------- 1 root root 0 Jul 14 15:09 register
-rw-r--r-- 1 root root 0 Jul 14 15:09 status

It is mounted as expected

cameron@Nook:~$ mount | grep binfmt
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)

Cleaning installed version (with RUNLEVEL=1 fix for runlevel detection)

cameron@Nook:~$ sudo RUNLEVEL=1 apt -y remove --purge binfmt-support
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  binfmt-support*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 246 kB disk space will be freed.
(Reading database ... 32683 files and directories currently installed.)
Removing binfmt-support (2.2.0-2) ...
 * Disabling additional executable binary formats binfmt-support                                                                                                                                                                [ OK ] 
Processing triggers for man-db (2.9.1-1) ...
(Reading database ... 32672 files and directories currently installed.)
Purging configuration files for binfmt-support (2.2.0-2) ...
Processing triggers for systemd (245.4-4ubuntu3.7) ...

No binfmt (as expected)

cameron@Nook:~$ ls -l /proc/sys/fs/binfmt_misc/
total 0

Not mounted either (as expected)

cameron@Nook:~$ mount | grep binfmt

Reinstall (with RUNLEVEL=1 fix for runlevel detection)

cameron@Nook:~$ sudo RUNLEVEL=1 apt install binfmt-support
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  binfmt-support
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/58.2 kB of archives.
After this operation, 246 kB of additional disk space will be used.
Selecting previously unselected package binfmt-support.
(Reading database ... 32670 files and directories currently installed.)
Preparing to unpack .../binfmt-support_2.2.0-2_amd64.deb ...
Unpacking binfmt-support (2.2.0-2) ...
Setting up binfmt-support (2.2.0-2) ...
Created symlink /etc/systemd/system/multi-user.target.wants/binfmt-support.service → /lib/systemd/system/binfmt-support.service.
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.7) ...

Still empty 😕

cameron@Nook:~$ ls -l /proc/sys/fs/binfmt_misc/
total 0

Not mounted 😕

cameron@Nook:~$ mount | grep binfmt

umount fails... now during apt reinstall ....

cameron@Nook:~$ sudo RUNLEVEL=1 apt reinstall binfmt-support
Reading package lists... Done
Building dependency tree       
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 0 B/58.2 kB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 32683 files and directories currently installed.)
Preparing to unpack .../binfmt-support_2.2.0-2_amd64.deb ...
 * Disabling additional executable binary formats binfmt-support                                                                                                                                                                       umount: /proc/sys/fs/binfmt_misc: not mounted.
update-binfmts: warning: Couldn't unmount the binfmt_misc filesystem from /proc/sys/fs/binfmt_misc.
                                                                                                                                                                                                                                [ OK ]
Unpacking binfmt-support (2.2.0-2) over (2.2.0-2) ...
Setting up binfmt-support (2.2.0-2) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.7) ...

I have also tried reinstalling qemu and qemu-user-static.

Related

• Cannot install qemu-user-static #2103
• binfmt_misc flags: 'OC' not working #2620 Says "binfmt flags support is complete in WSL2" so I assume it should all just work. It also did for a while...

[benhillis]

Since WSL doesn't run systemd / init scripts the qemu binfmt interpreter will have to be re-registered when you restart your VM. I believe there is probably a script you can run to accomplish this, but I'm not familiar.

[cinderblock]

@benhillis Good catch. I'd missed the Systemd reference.

Looking in /etc/systemd/system/multi-user.target.wants/binfmt-support.service I see the start script.

Running:

sudo update-binfmts --enable

Seems to have fixed it! Thanks!

---

This leads to the bigger question I guess. How to make it so I don't need to run update-binfmts --enable... How do packages that expect to have SystemD setup and working in WSL2?

[craigloewen-msft]

@cinderblock in general you can use commands like sudo service docker start to interact with services in WSL, for example the docker daemon and ssh daemon can be started this way instead of using systemd.

I'll close this issue out for our bookkeeping since it looks like you're resolved and this can be accrued to the feature of adding systemd support to WSL! Thanks for filing!

[trallnag]

Related to #994

[NelsonMinar]

I realize this bug is closed but I wanted to document a related failure mode: after installing qemu-user-static for the first time I get this error when running wslview

grep: /proc/sys/fs/binfmt_misc/WSLInterop: No such file or directory
WSL Interopability is disabled. Please enable it before using WSL.

I'm guessing something in the install process of these packages removes the WSLInterop entry. Rebooting fixes WSL but you're still left with the QEMU problem that update-binfmts fixes.

Did anyone file a bug with the upstream QEMU packages? I'm not even sure it's appropriate; they rely on systemd to work and systemd just isn't present here. It seems like a general limitation of WSL.

[Dragon1573]

I'm suffering the same problem. There's no proc/sys/fs/binfmt_misc/WSLInterop in my WSL2 instance.

Configurations of /etc/wsl.conf

10:51:50 ~/Code 101ms $ cat /etc/wsl.conf
[user]
# This instance is exported from official Ubuntu instance.
# I export it to "D:\\" and re-import into WSL2 else where.
default = dragon1573

[boot]
systemd = true

[interop]
enabled = true
appendWindowsPath = false

Launch Windows browser

10:51:41 ~/Code 2ms $ wslview https://cn.bing.com/
grep: /proc/sys/fs/binfmt_misc/WSLInterop: No such file or directory
WSL Interopability is disabled. Please enable it before using WSL.
grep: /proc/sys/fs/binfmt_misc/WSLInterop: No such file or directory
[error] WSL Interoperability is disabled. Please enable it before using WSL.
/usr/bin/wslview: line 216: /mnt/c/Windows/System32/reg.exe: cannot execute binary file: Exec format error
/usr/bin/wslview: line 149: /mnt/c/Windows/System32/chcp.com: cannot execute binary file: Exec format error
/usr/bin/wslview: line 156: /mnt/c/Windows/System32/WindowsPowerShell/v1.0/powershell.exe: cannot execute binary file: Exec format error
/usr/bin/wslview: line 149: /mnt/c/Windows/System32/chcp.com: cannot execute binary file: Exec format error

What's inside /proc/sys/fs/binfmt_misc/

10:51:59 ~/Code 1ms $ tree /proc/sys/fs/binfmt_misc/
/proc/sys/fs/binfmt_misc/
├── jar
├── llvm-15-runtime.binfmt
├── python3.10
├── register
└── status

0 directories, 5 files

---

Is my WSL2 instance (the *.vhdx) broken? Shall I unregister and reinstall a brand new offical Ubuntu instance?

[tianon]

Edit: see #8843 (comment) (which appears to be the issue this problem is getting redirected to more consistently)

---

I think this is due to https://github.com/systemd/systemd/blob/b9b0f338be2f63af96d295e4eaf9e8a8889190bb/src/binfmt/binfmt.c#L246-L251 -- effectively, systemd-binfmt assumes it must be managing all of the registrations for the system, which seems like a problem even for systemd-nspawn, for example. 😅

I used sudo systemctl mask systemd-binfmt.service to fix this (telling systemd explicitly to no longer assume it manages this). The consequence of this is that packages like qemu-user-static that also will try to register might not do so, but I'm also running Docker Desktop so I don't need that (what helped me figure this out is that its QEMU rules also get wiped out by this).

[liudonghua123]

I use sudo update-binfmts --enable to add llvm-17-runtime.binfmt python3.11 python3.12 to /proc/sys/fs/binfmt_misc/.

Details

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ lsmod | grep binfmt

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ ls /proc/sys/fs/binfmt_misc/
register  status  WSLInterop

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ ll /etc/systemd/system/multi-user.target.wants/binfmt-support.service
ls: cannot access '/etc/systemd/system/multi-user.target.wants/binfmt-support.service': No such file or directory

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ update-binfmts --enable
Command 'update-binfmts' not found, but can be installed with:
sudo apt install binfmt-support

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ sudo apt install binfmt-support
Installing:
  binfmt-support

Summary:
  Upgrading: 0, Installing: 1, Removing: 0, Not Upgrading: 69
  Download size: 64.3 kB
  Space needed: 214 kB / 1,018 GB available

Get:1 https://mirrors.tuna.tsinghua.edu.cn/kali kali-last-snapshot/main amd64 binfmt-support amd64 2.2.2-7 [64.3 kB]
Fetched 64.3 kB in 1s (121 kB/s)
Selecting previously unselected package binfmt-support.
(Reading database ... 203549 files and directories currently installed.)
Preparing to unpack .../binfmt-support_2.2.2-7_amd64.deb ...
Unpacking binfmt-support (2.2.2-7) ...
Setting up binfmt-support (2.2.2-7) ...
invoke-rc.d: could not determine current runlevel
Created symlink /etc/systemd/system/multi-user.target.wants/binfmt-support.service → /usr/lib/systemd/system/binfmt-support.service.
Processing triggers for man-db (2.12.1-2) ...
Processing triggers for kali-menu (2024.3.1) ...

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ ll /etc/systemd/system/multi-user.target.wants/binfmt-support.service
lrwxrwxrwx 1 root root 46 Oct 29 09:39 /etc/systemd/system/multi-user.target.wants/binfmt-support.service -> /usr/lib/systemd/system/binfmt-support.service

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ sudo update-binfmts --enable

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ ll /proc/sys/fs/binfmt_misc/status
-rw-r--r-- 1 root root 0 Oct 29 09:39 /proc/sys/fs/binfmt_misc/status

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ cat /proc/sys/fs/binfmt_misc/status
enabled

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ modprobe binfmt_misc

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ ls /proc/sys/fs/binfmt_misc/
llvm-17-runtime.binfmt  python3.11  python3.12  register  status  WSLInterop

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ qemu-i386 stack_overflow_32
Welcome to the CTF challenge!
Enter your input: aaa
You entered: aaa

Goodbye!

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ qemu-arm64 stack_overflow_arm64
qemu-arm64: Could not open '/lib/ld-linux-aarch64.so.1': No such file or directory

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ qemu-arm64 -L /usr/aarch64-linux-gnu/ stack_overflow_arm64
Welcome to the CTF challenge!
Enter your input: aaa
You entered: aaa

Goodbye!

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ qemu-arm -L /usr/arm-linux-gnueabi stack_overflow_arm7
Welcome to the CTF challenge!
Enter your input: aaa
You entered: aaa

Goodbye!

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$ qemu-armhf -L /usr/arm-linux-gnueabihf stack_overflow_arm7hf
Welcome to the CTF challenge!
Enter your input: aaa
You entered: aaa

Goodbye!

┌──(liudonghua㉿LDH-DESKTOP)-[~/ctf]
└─$

Steps to Enable Systemd in WSL2

1. Upgrade to WSL from the Microsoft Store (if you haven’t already):
   Ensure you’re using the latest version of WSL. You can install it from the Microsoft Store or upgrade with:

   wsl --update

2. Enable Systemd via wsl.conf File:
   Instead of using .wslconfig, you can enable systemd in the wsl.conf file inside your WSL2 distribution.

   1. Open or create /etc/wsl.conf in your WSL2 distribution:

      sudo nano /etc/wsl.conf

   2. Add the following configuration:

      [boot]
      systemd=true

   3. Save and close the file.

3. Restart WSL:
   After making changes to /etc/wsl.conf, restart WSL for the changes to take effect.

   wsl --shutdown

4. Reopen WSL2:
   Launch WSL2 again, and systemd should now be active. You can verify by running:

   systemctl list-units

Additional Note

The .wslconfig file (in your Windows user directory) is primarily for configuring WSL2 settings like memory limits and CPU allocation, while wsl.conf (inside the WSL instance) is now used for enabling systemd.