Skip to content

feat: limit in-flight operations in map/parallel#533

Draft
yaythomas wants to merge 1 commit into
mainfrom
feat/in-flight-max-concurrency
Draft

feat: limit in-flight operations in map/parallel#533
yaythomas wants to merge 1 commit into
mainfrom
feat/in-flight-max-concurrency

Conversation

@yaythomas

Copy link
Copy Markdown
Contributor

Description of changes:
max_concurrency previously sized the thread pool only, so branches that suspended (invoke, wait, callbacks) released their thread and the next item started immediately. All items were initiated at once regardless of the configured limit.

Rework the concurrency executor around a single coordinator loop:

  • The calling thread owns all branch state and schedules work; worker threads only run branches and report outcomes on a queue. No locks remain in the module.
  • max_concurrency now bounds in-flight branches. A suspended branch keeps its slot until it reaches a terminal state, matching the JS and Java SDKs. When every in-flight branch is suspended, the parent suspends with the earliest resume timestamp, or indefinitely.
  • Branches start in index order and operation ids derive from the branch index, so scheduling stays deterministic across invocations.
  • Timed suspends resume in-process via the event queue timeout, replacing the TimerScheduler background thread.
  • Consolidate completion logic (thresholds and reason inference) into CompletionPolicy, replacing ExecutionCounters and the duplicated logic in BatchResult.
  • Omit never-started branches from BatchResult, matching the JS SDK.
  • Stop scheduling when a branch is orphaned: an ancestor completed, so all further checkpoints under it would be rejected.
  • Validate config bounds at construction, matching the Java SDK: max_concurrency and min_successful must be at least 1, tolerated_failure_count non-negative, tolerated_failure_percentage within 0-100. Previously max_concurrency=0 silently meant unlimited and min_successful=0 silently meant all items.

Issue #, if available:
Closes #279

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@yaythomas yaythomas added this to the v2 milestone Jul 14, 2026
@yaythomas yaythomas marked this pull request as draft July 14, 2026 18:28
@yaythomas yaythomas force-pushed the feat/in-flight-max-concurrency branch 2 times, most recently from b5fe55d to 8325c37 Compare July 14, 2026 20:18
max_concurrency previously sized the thread pool only, so branches
that suspended (invoke, wait, callbacks) released their thread and
the next item started immediately. All items were initiated at once
regardless of the configured limit.

Rework the concurrency executor around a single coordinator loop:

- The calling thread owns all branch state and schedules work; worker
  threads only run branches and report outcomes on a queue. No locks
  remain in the module.
- max_concurrency now bounds in-flight branches. A suspended branch
  keeps its slot until it reaches a terminal state, matching the JS
  and Java SDKs. When every in-flight branch is suspended, the parent
  suspends with the earliest resume timestamp, or indefinitely.
- Branches start in index order and operation ids derive from the
  branch index, so scheduling stays deterministic across invocations.
- Timed suspends resume in-process via the event queue timeout,
  replacing the TimerScheduler background thread.
- Consolidate completion logic (thresholds and reason inference) into
  CompletionPolicy, replacing ExecutionCounters and the duplicated
  logic in BatchResult.
- Omit never-started branches from BatchResult, matching the JS SDK.
- Stop scheduling when a branch is orphaned: an ancestor completed,
  so all further checkpoints under it would be rejected.
- Record the completion decision in map/parallel summaries and obey
  it on replay: summaries carry the started-branch index set (or its
  complement, whichever is smaller) and replay reconstructs the exact
  live result instead of re-deriving statuses from child checkpoints.
  Summaries without the record fall back to checkpoint derivation.
  This also reconstructs FLAT results on replay, which checkpoint
  derivation could not discriminate.
- Validate config bounds at construction, matching the Java SDK:
  max_concurrency and min_successful must be at least 1,
  tolerated_failure_count non-negative, tolerated_failure_percentage
  within 0-100. Previously max_concurrency=0 silently meant
  unlimited and min_successful=0 silently meant all items.

Closes #279
@yaythomas yaythomas force-pushed the feat/in-flight-max-concurrency branch from 8325c37 to 9f04c8d Compare July 14, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: max_concurrency to limit operations rather than threads

1 participant