Implement global command-line options for dotnet test (MTP): --timeout and --maximum-failed-tests

[Evangelink]

Implements the remaining "global" (run-level) command-line options for dotnet test under Microsoft.Testing.Platform, as requested in #49709. Options placed before -- are treated as global and apply to the whole run; options after -- continue to be forwarded per test application.

New global options

• --timeout <duration> — aborts the run if total test-application execution time exceeds the given duration (e.g. 90s, 5m, 100ms). Timing starts when the first test app begins and only accrues while at least one app is running. On timeout the run returns TestSessionAborted (exit code 3), matching native MTP behavior.
• --maximum-failed-tests <n> — stops the run once the number of failed/errored/timed-out/cancelled test results reaches n, returning exit code 13 (TestExecutionStoppedForMaxFailedTests). Counting is independent of retry bookkeeping.

How cancellation works

The SDK now advertises a ServerControlPipeName capability in the handshake reply and opens a reverse control pipe. When a run-level policy trips, the SDK sends a cooperative CancelSession message to each test app over that pipe (the TestFx side landed in microsoft/testfx#9549). This requires the MTP 2.4 line, so MicrosoftTestingPlatformPackageVersion is bumped from 2.3.0 to 2.4.0-preview.

Key changes

• New run-level coordinator TestRunPolicy (thread-safe timeout + max-failure state machine).
• New IPC contract: ServerControlMessage / WaitForServerControlRequest models + serializers, field ids, and registration.
• TestApplication / TestApplicationActionQueue / TestApplicationHandler wire the policy, reverse control pipe, and linked cancellation.
• Option definitions, validation, help text, and localization (.resx + .xlf).

Tests

• TestRunPolicyTests (8, incl. a concurrency regression test for the timeout race), ServerControlMessageSerializerTests (3), TestApplicationHandlerTests (19), TestCommandDefinitionTests incl. new global-option parsing/validation cases (66).
• End-to-end in GivenDotnetTestBuildsAndRunsTests: RunMTPSolutionWithMaximumFailedTestsReturnsPolicyExitCode (exit 13) and RunMTPProjectWithGlobalTimeoutReturnsTestSessionAborted (exit 3).

Fixes #49709.

[Evangelink]

Code review summary

Three review passes plus one independent code-review subagent pass over the run-level logic (TestRunPolicy, TestApplication reverse control pipe, TestApplicationActionQueue, TestApplicationHandler, MicrosoftTestingPlatformTestCommand exit-code override, IPC models/serializers + field ids, option definitions/parsing/routing, resx/.xlf).

Pass 1 — self, core files (TestRunPolicy, option definitions, IPC serializers, command routing): no actionable findings. Verified ParseTimeout suffix precedence (ms/mil before s/m), ValidatePositiveInteger, exit-code precedence (policy override unconditional after aggregate; handshake remap gated on Success), and Discovery/Help correctly disabling both policies (null).

Pass 2 — self, remaining logic (TestApplication reverse pipe, TestApplicationActionQueue, TestApplicationHandler, TerminalTestReporter, WaitForServerControlRequestSerializer): no actionable findings. Confirmed the long-poll control request/cancellation coordination has no lost-wakeup/hang, the WhenAny(processExit, policy.Cancellation) + grace + Process.Kill fallback is sound, serializer round-trips (Kind=1, serializer ids 13/14), and passthrough routing forwards only post--- args. No unused usings introduced.

Pass 3 — code-review subagent (independent): found one real Medium bug — negative Timer due-time race in TestRunPolicy. With test-app parallelism >1, OnTestApplicationExited drives the remaining timeout non-positive a moment before it flips the reason to Timeout; a test app starting in that window constructed a Timer with a negative due time → ArgumentOutOfRangeException, spurious module (GenericFailure) failure, and a leaked active-application count. It also noted a latent secondary observation (the Reason getter maps completed state → None, weakening the start guard) that is not reachable in the current orchestration and is neutralized by the same fix.

Findings & disposition

Finding	Severity	Status	Fix
Negative Timer due-time race in TestRunPolicy.OnTestApplicationStarted	Medium	Fixed	c2a164dd0b — clamp due time to zero (fires immediately → OnTimeout cancels); added concurrency regression test ConcurrentStartAndExitAtTimeoutBoundaryDoesNotThrow
Latent: Reason getter maps completed→None (start guard)	Low / not reachable	No change	Neutralized by the clamp; not reachable in current orchestration
Exit-code precedence (Timeout→3, MaxFailed→13, handshake remap)	—	Clean	—
IPC serializer/field-id correctness (ids 13/14, ObjectFieldIds)	—	Clean	—
Option parsing/routing (global before --, passthrough after)	—	Clean	—
ParseTimeout suffix precedence, ValidatePositiveInteger	—	Clean	—
.xlf consistency (5 new + 1 needs-review-translation uniformly, consistent with /t:UpdateXlf)	—	Clean	—
Unused usings	—	Clean	—

Tests re-run green after the fix (incremental build)

TestRunPolicyTests 8/8 (incl. new regression test), ServerControlMessageSerializerTests 3/3, TestCommandDefinitionTests 66/66, TestApplicationHandlerTests 19/19, e2e RunMTPSolutionWithMaximumFailedTestsReturnsPolicyExitCode (exit 13) + RunMTPProjectWithGlobalTimeoutReturnsTestSessionAborted (exit 3) 2/2 — 98 total.

Branch commits

• 9b51583a37 Implement global MTP test cancellation options
• c2a164dd0b Fix negative Timer due-time race in TestRunPolicy

Coverage caveat

The two e2e tests validate SDK-side policy + exit codes but would also pass on the Process.Kill fallback, so they don't fully prove the reverse-pipe wire round-trip against TestFx 2.4 — a coverage gap (not a known bug) worth a follow-up integration test.

[Copilot]

Pull request overview

Implements remaining run-level (“global”) dotnet test options for Microsoft.Testing.Platform (MTP) by adding a thread-safe run policy (timeout + max failed tests), wiring cooperative session cancellation via a reverse control pipe, and updating CLI parsing/help/localization and tests.

Changes:

• Add --timeout and --maximum-failed-tests as global MTP dotnet test options with validation and localized help.
• Introduce a run-level coordinator (TestRunPolicy) and a new IPC reverse-control channel to cooperatively cancel test sessions.
• Add unit + end-to-end tests and update MTP dependency version to 2.4 preview line.

Show a summary per file

File	Description
test/Microsoft.NET.TestFramework/Constants.cs	Add new exit codes for timeout abort and max-failures policy.
test/Microsoft.DotNet.Cli.Utils.Tests/TransientSdkResolutionErrorDetectorTests.cs	Convert tests to MSTest attributes to match test project SDK.
test/dotnet.Tests/CommandTests/Test/TestRunPolicyTests.cs	New unit tests for TestRunPolicy behavior and concurrency regression.
test/dotnet.Tests/CommandTests/Test/TestCommandParserTests.cs	Add parsing/validation tests for new global options and -- forwarding.
test/dotnet.Tests/CommandTests/Test/ServerControlMessageSerializerTests.cs	New serializer contract tests for reverse-control IPC messages.
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTests.cs	Add E2E coverage for exit codes from max-failures and timeout.
src/Cli/Microsoft.DotNet.Cli.Utils/DangerousFileDetector.cs	Make COM factory initialization thread-safe to avoid race-induced false negatives.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hant.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hans.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.tr.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ru.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pt-BR.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pl.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ko.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ja.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.it.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.fr.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.es.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.de.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.cs.xlf	Localization updates for new/updated dotnet test option strings.
src/Cli/Microsoft.DotNet.Cli.Definitions/Commands/Test/TestCommandDefinition.MicrosoftTestingPlatform.cs	Define new options, add validators, and implement --timeout parsing.
src/Cli/Microsoft.DotNet.Cli.Definitions/CommandDefinitionStrings.resx	Add/adjust option descriptions and validation error strings.
src/Cli/dotnet/Commands/Test/MTP/TestRunPolicy.cs	New run-level policy state machine for timeout + maximum failed tests.
src/Cli/dotnet/Commands/Test/MTP/TestApplicationHandler.cs	Feed failure counts into the run policy from test result messages.
src/Cli/dotnet/Commands/Test/MTP/TestApplicationActionQueue.cs	Stop scheduling new test apps when run-level cancellation is requested.
src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs	Add reverse control pipe, advertise capability in handshake, and request cooperative cancellation.
src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs	Track cancellation state for policy-triggered cancellation reporting.
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs	Create and wire TestRunPolicy, link cancellation, and map reasons to exit codes.
src/Cli/dotnet/Commands/Test/MTP/IPC/Serializers/WaitForServerControlRequestSerializer.cs	New serializer for control-channel request.
src/Cli/dotnet/Commands/Test/MTP/IPC/Serializers/ServerControlMessageSerializer.cs	New serializer for control-channel cancellation message.
src/Cli/dotnet/Commands/Test/MTP/IPC/Serializers/RegisterSerializers.cs	Register new IPC serializers for control channel.
src/Cli/dotnet/Commands/Test/MTP/IPC/ObjectFieldIds.cs	Add message/field ids for new control-channel contract.
src/Cli/dotnet/Commands/Test/MTP/IPC/Models/WaitForServerControlRequest.cs	New request model for waiting on server control messages.
src/Cli/dotnet/Commands/Test/MTP/IPC/Models/ServerControlMessage.cs	New response model for server control messages.
src/Cli/dotnet/Commands/Test/MTP/ExitCode.cs	Add max-failed-tests policy exit code constant.
src/Cli/dotnet/Commands/Test/CliConstants.cs	Add handshake property id and server-control kind constant.
eng/Version.Details.xml	Update dependency version/SHA for Microsoft.Testing.Platform.
eng/Version.Details.props	Bump MicrosoftTestingPlatformPackageVersion to 2.4 preview.

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 3

[Evangelink]

Rebased onto latest main (merge commit e0d1b1ca) and resolved conflicts:

• Artifact post-processing feature (new on main): TestApplication/TestApplicationActionQueue/TestApplicationHandler constructors were reshaped to take artifactPostProcessingManager/artifactPostProcessingInvocation. Merged those with the run-level TestRunPolicy. testRunPolicy is now optional/nullable on all three, and every _testRunPolicy use is null-conditional, so post-processing tool invocations (which don't pass a policy) don't NPE or leak a CancellationTokenSource. The process-wait path keeps the policy-driven timeout/cancel + grace-Kill logic for normal test apps and main's ArtifactPostProcessingTimeout path for the post-processing tool.
• IPC serializer / field ids: no collisions — main took ids 11/12, this change uses 13/14; handshake property ids 12 (feature) vs 13/14/15 (main). Kept both.
• Version bump: took main's Microsoft.Testing.Platform 2.4.0-preview.26373.11 (newer than the branch's .6), resolving the earlier 2.3/1.1 vs 2.4/1.4 protocol concern.
• DangerousFileDetector race: both branches fixed the same race; took main's version.
• xUnit → MSTest migration: main migrated dotnet.Tests to MSTest. Converted the new TestRunPolicyTests, ServerControlMessageSerializerTests, TestCommandParserTests, and the two e2e tests in GivenDotnetTestBuildsAndRunsTests from [Fact]/[Theory]/[InlineData] to [TestClass]/[TestMethod]/[DataRow] and TestContext.CancellationToken.
• Regenerated the 14 .xlf files via /t:UpdateXlf for the clarified CmdTestInvalidTimeout string.

The 3 review comments above are addressed in 6abcae7abe.

Validation after merge: dotnet.csproj + redist layout build clean; targeted tests green — TestRunPolicyTests (8), ServerControlMessageSerializerTests (3), TestCommandDefinitionTests/parser + TestApplicationHandlerTests (113 combined), and both e2e tests (RunMTPProjectWithGlobalTimeoutReturnsTestSessionAborted → exit 3 / TestSessionAborted; RunMTPSolutionWithMaximumFailedTestsReturnsPolicyExitCode → exit 13). PR is now MERGEABLE.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Evangelink]

/backport to release/11.0.1xx-preview7

[github-actions]

Started backporting to release/11.0.1xx-preview7 (link to workflow run)

[github-actions]

@Evangelink backporting to release/11.0.1xx-preview7 was not run because the source pull request has not been merged. Please merge this pull request before requesting a backport.

[Evangelink]

/backport to release/11.0.1xx-preview7

[github-actions]

Started backporting to release/11.0.1xx-preview7 (link to workflow run)

[github-actions]

@Evangelink backporting to release/11.0.1xx-preview7 failed, the patch most likely resulted in conflicts. Please backport manually!

git am output

$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: Fix thread-safety race in DangerousFileDetector and unblock test build
Using index info to reconstruct a base tree...
M	src/Cli/Microsoft.DotNet.Cli.Utils/DangerousFileDetector.cs
M	test/Microsoft.DotNet.Cli.Utils.Tests/TransientSdkResolutionErrorDetectorTests.cs
Falling back to patching base and 3-way merge...
Auto-merging src/Cli/Microsoft.DotNet.Cli.Utils/DangerousFileDetector.cs
CONFLICT (content): Merge conflict in src/Cli/Microsoft.DotNet.Cli.Utils/DangerousFileDetector.cs
Auto-merging test/Microsoft.DotNet.Cli.Utils.Tests/TransientSdkResolutionErrorDetectorTests.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 Fix thread-safety race in DangerousFileDetector and unblock test build
Error: The process '/usr/bin/git' failed with exit code 128

Link to workflow output

[Evangelink]

/backport to release/11.0.1xx-preview7

[github-actions]

Started backporting to release/11.0.1xx-preview7 (link to workflow run)

[github-actions]

@Evangelink backporting to release/11.0.1xx-preview7 failed, the patch most likely resulted in conflicts. Please backport manually!

git am output

$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: Fix thread-safety race in DangerousFileDetector and unblock test build
Using index info to reconstruct a base tree...
M	src/Cli/Microsoft.DotNet.Cli.Utils/DangerousFileDetector.cs
M	test/Microsoft.DotNet.Cli.Utils.Tests/TransientSdkResolutionErrorDetectorTests.cs
Falling back to patching base and 3-way merge...
Auto-merging src/Cli/Microsoft.DotNet.Cli.Utils/DangerousFileDetector.cs
CONFLICT (content): Merge conflict in src/Cli/Microsoft.DotNet.Cli.Utils/DangerousFileDetector.cs
Auto-merging test/Microsoft.DotNet.Cli.Utils.Tests/TransientSdkResolutionErrorDetectorTests.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 Fix thread-safety race in DangerousFileDetector and unblock test build
Error: The process '/usr/bin/git' failed with exit code 128

Link to workflow output

[Evangelink]

Manual backport to
elease/11.0.1xx-preview7\ opened as #55476. The automated bot couldn't apply it because its per-commit \git am\ re-applied the \DangerousFileDetector\ race fix that already exists on the release branch; a \git cherry-pick -m 1\ of the merge commit resolved that overlap cleanly.