Apply dotnet test zero-tests verdict at the whole-run level - #55362
Merged
Conversation
The whole-run zero-tests normalization (microsoft/testfx#7457) means an empty module's exit code 8 (ZeroTests) is normalized to success before aggregation, so it no longer collides with the failing module's exit code 2 to produce GenericFailure (1). The run's verdict is now AtLeastOneTestFailed (2). Update the expectation and rename the test accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e0b86e41-a49a-4a68-bea8-3cf5df5694d8
…dule E2E test - Skip the whole-run zero-tests verdict when invoked in help mode so the help command keeps returning exit code 0 (Success) instead of ZeroTests. - Add an E2E test (plus MultiTestProjectSolutionWithZeroTestsAndPassingTests asset) covering a run where one module matches no tests (exit code 8) but another passes, asserting the overall exit code is Success. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8abef3da-3035-4f48-85ab-485d5f704fc2
Treat an empty module as successful when computing the terminal run summary, matching the whole-run exit-code aggregation. Add focused coverage for an empty module alongside a passing module. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 182327bb-5826-4966-a2cc-1b92338b706f
|
Azure Pipelines: Successfully started running 1 pipeline(s). 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adjusts dotnet test (Microsoft.Testing.Platform path) so that a single module reporting ZeroTests (exit code 8) does not fail an otherwise successful multi-module run, and so the “zero tests ran” verdict is decided once at the whole-run level based on the total executed test count.
Changes:
- Normalize per-module
ZeroTestsresults before aggregate exit-code calculation, and decide whole-runZeroTestsonly from the overallTotalTestscount. - Update terminal reporting and exit-code behavior for mixed runs (empty module + passing/failing module).
- Add new test assets and tests to cover the mixed “zero tests + passing” scenario.
Show a summary per file
| File | Description |
|---|---|
| src/Cli/dotnet/Commands/Test/MTP/TestApplicationActionQueue.cs | Normalizes per-module ZeroTests before aggregating exit codes. |
| src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs | Applies whole-run ZeroTests verdict based on aggregate TotalTests == 0. |
| src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs | Treats ZeroTests as success for per-assembly success tracking. |
| test/dotnet.Tests/CommandTests/Test/TerminalTestReporterTests.cs | Adds focused reporter test for “zero tests + passing” summary behavior. |
| test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTests.cs | Updates/extends end-to-end tests for mixed-module exit code + summary behavior. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithZeroTestsAndPassingTests/MultiTestProjectSolutionWithZeroTestsAndPassingTests.sln | New multi-project solution test asset for “zero tests + passing tests”. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithZeroTestsAndPassingTests/global.json | Configures test asset to use Microsoft.Testing.Platform runner. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithZeroTestsAndPassingTests/TestProject/TestProject.csproj | New empty-module MTP test project (intentionally produces zero tests). |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithZeroTestsAndPassingTests/TestProject/Program.cs | Implements an MTP adapter that publishes no tests (zero-tests module). |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithZeroTestsAndPassingTests/AnotherTestProject/AnotherTestProject.csproj | New passing-module MTP test project. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithZeroTestsAndPassingTests/AnotherTestProject/Program.cs | Implements an MTP adapter that publishes a passing test node. |
Copilot's findings
- Files reviewed: 11/11 changed files
- Comments generated: 2
Preserve dispose failures after normalizing zero-test module results and retain the per-assembly zero-tests label in terminal output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 182327bb-5826-4966-a2cc-1b92338b706f
Evangelink
enabled auto-merge
July 20, 2026 10:50
YuliiaKovalova
approved these changes
Jul 20, 2026
This was referenced Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A test run should not fail just because one module matched no tests when other modules executed successfully. The current per-module exit code aggregation also leaves the terminal summary inconsistent: the command exits successfully, but reports the run as failed.
This change:
ZeroTestsresults before aggregate exit-code calculationExit code: 8diagnosticThis implements the SDK side of microsoft/testfx#7457.