Skip to content

Improve resiliency of git worktree commands#1943

Merged
tyrielv merged 1 commit into
microsoft:masterfrom
tyrielv:tyrielv/fix-concurrent-worktree-mount
Apr 15, 2026
Merged

Improve resiliency of git worktree commands#1943
tyrielv merged 1 commit into
microsoft:masterfrom
tyrielv:tyrielv/fix-concurrent-worktree-mount

Conversation

@tyrielv
Copy link
Copy Markdown
Contributor

@tyrielv tyrielv commented Apr 13, 2026

Summary

MountNewWorktree() in the post-command hook silently discarded the exit code of gvfs mount, leaving worktrees unprojected when concurrent adds caused mount contention.

Review Guide

3 files changed — read in this order:

1. GVFS.Hooks/Program.cs (smallest change)

  • Added GetHookExitCode() — parses --exit_code=N from hook args. Generic infrastructure any post-hook handler can use. Placed alongside GetHookType and GetGitCommand.

2. GVFS.Hooks/Program.Worktree.cs (product fix)

  • RunWorktreePostCommand — now checks exit code. Skips add/move post-processing if git itself failed; always runs remove cleanup.
  • TryMountWithRetry (new) — retries gvfs mount 2× with 100ms/250ms delays. First attempt shows console output; retries are quiet. Returns bool.
  • MountNewWorktree — captures git checkout exit code (was discarded). On checkout failure, emits warning with recovery command and skips mount. On mount failure, emits warning.
  • MountMovedWorktree — same TryMountWithRetry treatment.

3. WorktreeTests.cs (test improvements)

  • Adaptive scaling — starts at Environment.ProcessorCount worktrees (min 4). If concurrent adds overwhelm the primary mount ("does not appear to be mounted"), halves count and retries.
  • Pipe-based mount assertion — probes worktree-specific named pipe directly instead of File.Exists. Definitive signal that ProjFS is active.
  • Barrier synchronizationCountdownEvent ensures all threads launch simultaneously.
  • Diagnostics on failure — captures directory listing, .git contents on assertion failure.

Verification

Tested locally with 16 concurrent worktrees (Environment.ProcessorCount on dev machine):

Scenario Runs Failures Rate
Without product fix 5 5 100%
With product fix 5 0 0%

Note: On CI runners with fewer cores (4), the lower concurrency may not trigger the race. The test still validates correctness of concurrent worktree operations and the adaptive scaling logic — the product fix is independently justified by code inspection (silent exit code discard).

CI: all 45 checks passed.

@tyrielv tyrielv force-pushed the tyrielv/fix-concurrent-worktree-mount branch 3 times, most recently from 3cb3366 to 2d9b375 Compare April 13, 2026 19:08
@tyrielv tyrielv changed the title GVFS: fix silent mount failure on concurrent worktree add Improve resiliency of git worktree commands Apr 13, 2026
@tyrielv tyrielv marked this pull request as ready for review April 13, 2026 21:51
@tyrielv tyrielv requested a review from KeithIsSleeping April 13, 2026 21:51
Comment thread GVFS/GVFS.FunctionalTests/Tests/EnlistmentPerFixture/WorktreeTests.cs Outdated
Comment thread GVFS/GVFS.Hooks/Program.Worktree.cs Outdated
Comment thread GVFS/GVFS.Hooks/Program.cs
Comment thread GVFS/GVFS.Hooks/Program.Worktree.cs Outdated
MountNewWorktree() in the post-command hook silently discarded the exit
code of 'gvfs mount', leaving worktrees unprojected when concurrent
adds caused mount contention. Also discarded the git checkout exit code.

Product fix:
- Check git checkout exit code; skip mount on failure
- Check gvfs mount exit code with retry (2 retries, 100ms/250ms)
- Extract and check --exit_code from hook args to skip post-processing
  when the git command itself failed
- Return int? from GetHookExitCode so callers decide null semantics
- Apply same TryMountWithRetry to MountMovedWorktree
- Remount old worktree when 'git worktree move' fails (pre-hook unmounts)
- Remove maxRetries parameter; retry count driven by delay array length
- Emit actionable warning to stderr on mount failure with recovery command

Test fix:
- Increase concurrent worktrees from 2 to ProcessorCount for reliable
  mount contention
- Use CountdownEvent barrier for tight synchronization of launches
- Add pipe-based mount verification as primary assertion (probes the
  worktree-specific named pipe directly instead of relying on File.Exists)
- Add diagnostic capture on failure (directory listing, .git contents)
- Use dynamic branch names with GUID suffixes to avoid collisions
- Use numeric indices instead of char arithmetic for labels/paths to
  support >26 concurrent worktrees
- Only retry at lower concurrency when ALL failures are overload-related;
  mixed failure sets (overload + real regression) are reported immediately

Bug: https://dev.azure.com/microsoft/OS/_workitems/edit/61784115

Assisted-by: Claude Opus 4.6
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
@tyrielv tyrielv force-pushed the tyrielv/fix-concurrent-worktree-mount branch from 2d9b375 to e876cd7 Compare April 15, 2026 02:17
@tyrielv tyrielv merged commit 7669d6c into microsoft:master Apr 15, 2026
88 of 89 checks passed
@mjcheetham mjcheetham mentioned this pull request May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants