Skip to content

Parallelize Python and Windows .NET CI tests - #2251

Merged
SteveSandersonMS merged 2 commits into
mainfrom
stevesa/ci-parallelism
Aug 4, 2026
Merged

Parallelize Python and Windows .NET CI tests#2251
SteveSandersonMS merged 2 commits into
mainfrom
stevesa/ci-parallelism

Conversation

@SteveSandersonMS

@SteveSandersonMS SteveSandersonMS commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • run Python tests in two pytest-xdist worker processes
  • use file-level scheduling so each module-scoped E2E client and replay proxy remains on one worker
  • split the slow Windows/default .NET suite across two isolated CI jobs
  • keep xUnit execution serial within each process, avoiding the process-global environment and working-directory races that in-process class parallelism would introduce

Results

Timings from the PR check run:

Leg Recent baseline PR check
Python, Ubuntu, default 10.7 min 5.8 min
Python, Windows, default 21.6 min 12.5 min
Python, Windows, in-process 27.4 min 15.8 min
.NET, Windows test step 18.1 min 10.7 min critical shard
.NET, Windows whole job 21.9 min 17.8 min critical shard

The Windows .NET critical job was 17.8 minutes in the PR run because one shard had an anomalous 8.9-minute build; its test step was 7.5 minutes. In the separate all-SDK dispatch, builds were normal and the critical Windows shard completed in 15.4 minutes.

The overall PR workflow took 26.8 minutes because two unchanged Ubuntu .NET backend cells had anomalous 12.4- and 19.9-minute build steps. The same final commit completed the all-SDK dispatch in 16.3 minutes. This change does not parallelize or otherwise modify those Ubuntu cells.

The .NET shard filters cover all 710 discovered tests exactly once, split 355/355. All six SDK workflows passed on the final commit in workflow run 30923106592.

Relationship to #2186

#2186 splits Windows .NET execution by target framework for isolation. This PR splits it by test class for latency. The approaches are complementary; if #2186 merges first, the matrix should compose both dimensions (two frameworks by two test shards).

Use two pytest-xdist workers with file-level scheduling so module-scoped E2E clients and proxies remain together while independent modules run concurrently.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 4, 2026 14:46
@SteveSandersonMS
SteveSandersonMS requested a review from a team as a code owner August 4, 2026 14:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Runs Python CI tests concurrently while preserving module-scoped E2E fixtures and proxies.

Changes:

  • Adds pytest-xdist as a development dependency.
  • Runs pytest with two workers using file-level scheduling.
Show a summary per file
File Description
python/pyproject.toml Adds the parallel-test dependency.
.github/workflows/python-sdk-tests.yml Enables two-worker, file-scheduled test execution.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Balanced

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

SDK Consistency Review ✅

This PR only modifies Python CI test infrastructure:

  • Adds pytest-xdist dev dependency for parallel test execution
  • Updates the CI workflow to run tests with -n 2 --dist=loadfile

This is Python/pytest-specific tooling with no equivalent needed in other SDKs (each SDK uses its own test runner). No SDK API surface changes were made, so there are no cross-language consistency concerns.

Generated by SDK Consistency Review Agent for #2251 · sonnet46 15.4 AIC · ⌖ 5.35 AIC · ⊞ 6.6K ·

Run the slow Windows default-transport suite in two isolated jobs while preserving serial xUnit execution within each test host.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 15:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (1)

.github/workflows/dotnet-sdk-tests.yml:15

  • The PR title, summary, and reported results are scoped entirely to Python/pytest-xdist, but this file also introduces a separate .NET Windows sharding strategy. This makes the .NET behavior an unreported scope change with no shard results to validate it. Please split this change into a dedicated PR, or update the PR description and provide the corresponding .NET validation results.
    name: ".NET SDK Tests (${{ matrix.os }}, ${{ matrix.transport }}, ${{ matrix.backend }}, ${{ matrix.shard }})"
  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@github-actions github-actions Bot mentioned this pull request Aug 4, 2026
@SteveSandersonMS SteveSandersonMS changed the title Run Python tests in parallel Parallelize Python and Windows .NET CI tests Aug 4, 2026
@SteveSandersonMS
SteveSandersonMS merged commit dba8f83 into main Aug 4, 2026
113 checks passed
@SteveSandersonMS
SteveSandersonMS deleted the stevesa/ci-parallelism branch August 4, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants