F-020: perf(services): semaphore-bound concurrent git show spawning#23
Open
Sephyi wants to merge 1 commit intodevelopmentfrom
Open
F-020: perf(services): semaphore-bound concurrent git show spawning#23Sephyi wants to merge 1 commit intodevelopmentfrom
Sephyi wants to merge 1 commit intodevelopmentfrom
Conversation
`fetch_file_contents` previously spawned one `git show` task per staged file with no upper bound on concurrency. Each file spawns two processes (staged blob + HEAD blob), so a 50-file stage fanned out to ~100 subprocesses at once — enough to cause fork/FD pressure on machines with a modest core count. Wrap the `JoinSet` spawns in a `tokio::sync::Semaphore` whose permit count is `available_parallelism() * 2`, clamped to `16..=32`. Permits are acquired inside each spawned task and held for the task's lifetime, so parallelism scales with the host but never exceeds a safe ceiling. The public signature of `fetch_file_contents` is unchanged; callers see identical behavior for small stages and smoother subprocess spawning on large ones. Uses `std::thread::available_parallelism()` to avoid adding a `num_cpus` dependency. Closes audit entry F-020 from #3.
There was a problem hiding this comment.
Pull request overview
This PR addresses audit finding F-020 by adding a concurrency bound around git show subprocess spawning in GitService::fetch_file_contents, preventing large staged sets from triggering unbounded process creation.
Changes:
- Introduces a computed concurrency limit for
git showsubprocesses. - Wraps per-file content fetching in a
tokio::sync::Semaphorepermit to bound concurrentgit showexecution.
Comments suppressed due to low confidence (1)
src/services/git.rs:265
- The semaphore is acquired inside each spawned task, so
fetch_file_contentsstill creates one task per path immediately. On very large stages this can allocate a large number of pending tasks even though subprocess concurrency is bounded. If the goal is to bound overall work, consider acquiring the permit before spawning and moving it into the task (so only up to N tasks exist at once), or switching to a bounded-concurrency pattern that doesn't enqueue all tasks at once.
let mut set = tokio::task::JoinSet::new();
let work_dir: Arc<PathBuf> = Arc::new(self.work_dir.clone());
let semaphore = Arc::new(tokio::sync::Semaphore::new(
Self::git_show_concurrency_limit(),
));
for path in paths {
let work_dir = Arc::clone(&work_dir);
let semaphore = Arc::clone(&semaphore);
let path = path.clone();
set.spawn(async move {
// Semaphore is never closed, so acquire cannot fail.
let _permit = semaphore
.acquire_owned()
.await
.expect("git-show semaphore closed unexpectedly");
let staged =
Self::fetch_git_show(&work_dir, &format!(":0:{}", path.display())).await;
let head =
Self::fetch_git_show(&work_dir, &format!("HEAD:{}", path.display())).await;
(path, staged, head)
});
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+227
to
+234
| /// (staged + HEAD); capping at `cores * 2` (clamped to 16..=32) keeps | ||
| /// parallelism high on beefy machines without causing fork/FD pressure on | ||
| /// large stages. | ||
| fn git_show_concurrency_limit() -> usize { | ||
| let cores = std::thread::available_parallelism() | ||
| .map(std::num::NonZeroUsize::get) | ||
| .unwrap_or(4); | ||
| (cores * 2).clamp(16, 32) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
perf(services): semaphore-bound concurrent git show spawning.
Audit context
Closes audit entry F-020 from #3.
Verification
cargo fmt --checkcargo clippy --all-targets --all-features -- -D warningscargo test --all-targets