Container is not notified by file operations on shared volumes

[rjeffman]

When running a container with a mounted volume, events in a file/directory (file change, creation, deletion) is not triggered in the container.

This, for example, prevents actions (e.g. incremental rebuilds) based on file changes.

The behavior is the same as in Podman, and the issue is similar to containers/podman#22343

[egernst]

Hey @rjeffman - yes, this is a limitation on filesystem events being propagated with virtio-fs. We have some ideas of workarounds to facilitate tools like "inotify" working in the guest, but are still working on it.

[BlackHole1]

We have some ideas of workarounds

@egernst Cool!

If possible, could you share your thoughts? I spent several weeks on this issue before but got nowhere. Currently, most open-source solutions are based on watching file changes on the host and then touching the file in the VM to receive notifications, but this solution performs very poorly. Or are you planning to use a kernel patch solution instead?

[egernst]

@BlackHole1 - workarounds include watching the path in question on the host, and manually updating the particular files using chown inside the guest, that way the end user can get a notification once that touch happens.

[BlackHole1]

workarounds include watching the path in question on the host

It seems to align with the mainstream solutions in the community, but this approach currently has the following issues: unable to simulate delete events and poor performance.🤔

[egernst]

I’m curious about delete not working - will follow up.

Can you clarify what you mean specifically around performance? Latency?

[BlackHole1]

I’m curious about delete not working - will follow up.

Because when you execute chmod on a deleted file, it does not trigger the kernel’s inotify event. See: lima-vm/lima#1913 (comment)

Can you clarify what you mean specifically around performance? Latency?

Imagine the impact of executing a watch on a node_modules directory, especially when it has just been created (thousands of small files created in a short period, and then chmod is executed on each file). In Colima, the watch on the node_modules directory is ignored by default. abiosoft/colima@ad6ef96

---

Currently, this type of solution is predominant in the open-source community.

1. vm: add experimental long-polling based filesystem inotify events abiosoft/colima#655
2. Support for inotify in mounted directories lima-vm/lima#1913

In the past, Docker Desktop for macOS also used a similar solution, so when the host deletes a certain file, you would find: level=info msg="injected MODIFY event (ts: 1677314215061760000) on /host_mnt/private/ in the Docker logs.

However, currently both Docker Desktop and orbstack are using another solution that we are not aware of (most likely patching the kernel). They can now notify of Delete events, and orbstack can detect changes in node_modules, indicating that orbstack does not ignore node_modules. I have not tested Docker Desktop.