Skip to content

fix: handle recursive watcher ignore paths correctly#13756

Open
ManManavadaria wants to merge 5 commits into
docker:mainfrom
ManManavadaria:13750-watch-ignore-fix
Open

fix: handle recursive watcher ignore paths correctly#13756
ManManavadaria wants to merge 5 commits into
docker:mainfrom
ManManavadaria:13750-watch-ignore-fix

Conversation

@ManManavadaria
Copy link
Copy Markdown

What I did
Applied ignore rules at the watcher layer and updated recursive traversal to skip ignored files/directories, ensuring permission errors from paths in the ignore list do not fail watch.

Related issue
fixes #13750

(not mandatory) A picture of a cute animal, if possible in relation to what you did

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to fix Docker Compose watch startup failures when an ignored path is unreadable by pushing ignore handling down into the watcher layer. In the codebase, that primarily affects how pkg/compose constructs file watchers and how pkg/watch traverses directories on different platforms.

Changes:

  • Added an ignore matcher parameter to the watcher API and threaded it through Compose watch setup.
  • Updated the naive watcher to skip ignored directories during recursive traversal and to suppress permission errors for ignored paths during startup walks.
  • Adjusted Darwin watcher and test fixture construction to use the new watcher signature.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
pkg/watch/watcher_naive.go Adds ignore-aware traversal and permission handling to the non-fsevents watcher implementation.
pkg/watch/watcher_darwin.go Updates the Darwin watcher constructor signature to match the new watcher API.
pkg/watch/notify_test.go Updates test fixture setup to call NewWatcher with an ignore matcher.
pkg/watch/notify.go Extends the exported watcher constructor to accept an ignore matcher.
pkg/compose/watch.go Builds ignore matchers from watch triggers and passes them into watcher creation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/watch/watcher_naive.go Outdated
Comment thread pkg/watch/watcher_naive.go Outdated
Comment thread pkg/compose/watch.go Outdated
Comment thread pkg/watch/watcher_darwin.go Outdated
Comment thread pkg/watch/watcher_naive.go Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented May 4, 2026

Codecov Report

❌ Patch coverage is 58.06452% with 13 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/watch/watcher_naive.go 47.61% 8 Missing and 3 partials ⚠️
pkg/compose/watch.go 75.00% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@ManManavadaria ManManavadaria force-pushed the 13750-watch-ignore-fix branch from 1e22d66 to bba54a8 Compare May 6, 2026 19:15
@ManManavadaria
Copy link
Copy Markdown
Author

Hi @glours @ndeloof ,
I’ve addressed all the review feedbacks in the latest commit. please review it, appreciate your time and feedback.

@ManManavadaria ManManavadaria force-pushed the 13750-watch-ignore-fix branch from bba54a8 to 4ab735b Compare May 7, 2026 13:18
@glours
Copy link
Copy Markdown
Contributor

glours commented May 7, 2026

/review

Copy link
Copy Markdown

@docker-agent docker-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

This PR correctly applies ignore rules to the watcher layer for Linux/Windows (naive watcher) and handles permission errors from ignored paths during recursive directory traversal. However, there are two issues worth addressing:

  1. macOS (Darwin) — the ignore parameter is silently discarded, so the fix does not apply on macOS.
  2. Multi-trigger ignore OR-logic — merging all trigger ignore matchers with OR logic at the watcher layer can cause directories needed by one trigger to be skipped because another trigger ignores them.

Reviewed 6 files: pkg/compose/watch.go, pkg/watch/notify.go, pkg/watch/notify_test.go, pkg/watch/watcher_darwin.go, pkg/watch/watcher_naive.go, pkg/watch/watcher_naive_test.go

Comment thread pkg/watch/watcher_darwin.go Outdated
Comment thread pkg/compose/watch.go Outdated
Comment thread pkg/watch/watcher_naive.go Outdated
Comment thread pkg/watch/watcher_naive.go Outdated
Comment thread pkg/watch/watcher_naive.go Outdated
Comment thread pkg/watch/watcher_naive.go Outdated
Comment thread pkg/watch/watcher_darwin.go Outdated
@ManManavadaria ManManavadaria force-pushed the 13750-watch-ignore-fix branch 2 times, most recently from 43cc3f0 to cf46c2c Compare May 11, 2026 13:34
@ManManavadaria
Copy link
Copy Markdown
Author

ManManavadaria commented May 11, 2026

Hi @glours, Thanks for the valuable feedback. Based on the review comments, I’ve refactored the per-path trigger watchers and intersectPathMatcher and implemented multiNotify methods for grouped per-path watches and implemented PathMatcher ignores in Darwin watcher.

Please review the changes and share your feedback.

Thanks!

@ManManavadaria ManManavadaria requested a review from glours May 12, 2026 06:29
Copy link
Copy Markdown
Contributor

@glours glours left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ManManavadaria

Coming back with a fuller pass and I need to flag something I missed in earlier rounds.

What I missed

The previous watch.NewWatcher(paths) ran pathutil.EncompassingPaths internally, so overlapping trigger roots collapsed into a single inotify watch on the common ancestor. With the new per-path loop, each path gets its own watcher and
EncompassingPaths becomes a no-op. The result: overlapping subtrees fire every event twice, get forwarded twice through multiNotify, and are then masked by the QuietPeriod debouncer. Functionally tolerated, but we double our consumption of fs.inotify.max_user_watches, a bounded kernel resource users already hit on large repos.

I should have spotted this the first time, sorry for the late catch.

Proposed direction

Keep one shared watcher, push the ignore handling into it via a single matcher. Specifically:

  • Keep: the ignore PathMatcher parameter on NewWatcher, the shouldIgnore / shouldIgnoreEntireDir / reshaped shouldSkipDir logic in the naive watcher, the Darwin shouldNotify parity, and intersectPathMatcher. These are all good.
  • Drop: multiNotify / NewMultiWatcher and the per-path watcher loop in pkg/compose/watch.go.
  • Add: a root-aware ignore matcher implementing PathMatcher, backed by map[absRoot]PathMatcher. On Matches / MatchesEntireDir for a file, dispatch to the roots that contain it and intersect their results, i.e. a file is ignored only if every trigger that could care about it agrees to ignore it. This preserves per-trigger isolation without losing EncompassingPaths consolidation.

Tests-wise: keep the intersect / shouldSkipDir / Darwin coverage you already have, and replace the (currently absent) multiNotify tests with a handful for the new root-aware matcher.

One ask on the PR body

When you push the rework, please describe the architectural change accurately, "applied ignore rules at the watcher layer" undersells what's actually being touched, which is partly how I under-scoped my first read.

Happy to discuss alternatives (e.g. keeping per-path watchers but running EncompassingPaths over the keys of watchRootIgnores first) if the direction above doesn't feel right. The underlying fix is correct, just want to land the smallest version of it.

@ManManavadaria
Copy link
Copy Markdown
Author

and I need to flag something I missed in earlier rounds

Thanks for the detailed review. You're right, this isn't an optimal solution and unnecessarily doubles the fs events consumption, I should have explored alternative approaches and discussed them first instead of directly following the multiWatcher suggestion.

Your suggested direction makes sense, I'll rework this with a better solution shortly. Also, thanks for pointing out about the PR body discription, I'll keep that in mind for future PRs.

Signed-off-by: ManManavadaria <manmanavadaria@gmail.com>
…nd apply review fixes

Signed-off-by: ManManavadaria <manmanavadaria@gmail.com>
- Add multiNotify and watcher methods for grouped per-path watches

- Centralize per-path multi-watcher behavior so the naive watcher flow stays the same

- Add intersectPathMatcher implementing PathMatcher for shared roots

- Remove unnecessary conditions in the naive watcher as per feedback

Signed-off-by: ManManavadaria <manmanavadaria@gmail.com>
- Store PathMatcher on fseventNotify and filter events with shouldIgnore and shouldNotify, matching the naive watcher

Signed-off-by: ManManavadaria <manmanavadaria@gmail.com>
Use one watcher with per-root ignore maps so overlapping triggers do not apply another root's patterns when pruning or filtering events. Intersect matchers when multiple triggers share the same path.

- Start a single watcher from compose watch and drop NewMultiWatcher/multiNotify

- Pass map[path]PathMatcher into NewWatcher and normalize ignores as roots change

- Merge matchers when paths are promoted to existing ancestors or nested roots

- Apply subtree pruning and path filtering with the matcher for the matching root

- Align Darwin watcher startup and ignore handling with the naive watcher

- Add tests for per-root isolation, intersected ignores

Signed-off-by: ManManavadaria <manmanavadaria@gmail.com>
@ManManavadaria ManManavadaria force-pushed the 13750-watch-ignore-fix branch from cf46c2c to eee815e Compare May 14, 2026 18:18
@ManManavadaria ManManavadaria requested a review from glours May 14, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] watch tries to read ignored folders leading to permission denied

3 participants