Add artifact post-processing orchestration

[Evangelink]

Summary

Implements the remaining testfx-side pieces of RFC 018 so dotnet/sdk can orchestrate artifact consolidation after multi-module MTP runs.

• adds the experimental IArtifactPostProcessor contract, DTOs, and registration surface
• promotes tool registration APIs for extension-owned merge tools
• advertises post-processor kinds and legacy extensions during the dotnet-test handshake
• adds the hidden internal-merge-artifacts dispatcher with manifest routing and merged artifact return over the existing pipe
• adds the user-facing merge-trx tool and TRX post-processor over the existing merge engine
• adds localized diagnostics, API baselines, unit coverage, and help/info expectations

Review follow-ups

Two independent review passes were completed, including the MSTest/MTP expert reviewer. Follow-up fixes include:

• localizing platform-side dispatcher and manifest diagnostics
• restricting artifact-post-processor pipe setup and handshake identity to the internal dispatcher
• warning when processor capabilities overlap
• using deterministic unique merged TRX filenames to avoid cross-run overwrites
• adding malformed-manifest and successful TRX processing coverage
• scoping tool option validation to the selected tool

Validation

• Microsoft.Testing.Platform.UnitTests — 1399 passed
• Microsoft.Testing.Extensions.UnitTests — 775 passed, 5 expected skips
• merge-trx end-to-end produced a valid merged TRX
• package build and NuGet validation succeeded
• all-extensions --help and --info acceptance tests passed across net462, net8.0, and net10.0

Related to dotnet/sdk#47613 and RFC 018 (#9187).

[Copilot]

Pull request overview

Implements RFC 018 artifact post-processing orchestration for MTP, including extension APIs, SDK handshake support, internal dispatching, and TRX merging.

Changes:

• Adds artifact post-processor contracts, registration, manifest routing, and pipe reporting.
• Promotes tool registration and adds the merge-trx tool.
• Adds API baselines, localized resources, and tests.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Extensions/ArtifactPostProcessing/ArtifactPostProcessingTests.cs	Tests capabilities, registration, and manifests.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/TrxArtifactPostProcessorTests.cs	Tests TRX post-processing.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Adds merge-tool info expectations.
src/Platform/Microsoft.Testing.Platform/Tools/IToolsManager.cs	Makes tool registration public.
src/Platform/Microsoft.Testing.Platform/Tools/ITool.cs	Promotes and documents tool contracts.
src/Platform/Microsoft.Testing.Platform/ServerMode/IPushOnlyProtocol.cs	Adds identity and message sending.
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Constants.cs	Adds handshake capabilities and modes.
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/DotnetTestConnection.cs	Advertises tool execution mode.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlf	Syncs Traditional Chinese resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlf	Syncs Simplified Chinese resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlf	Syncs Turkish resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlf	Syncs Russian resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlf	Syncs Portuguese resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlf	Syncs Polish resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlf	Syncs Korean resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlf	Syncs Japanese resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlf	Syncs Italian resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlf	Syncs French resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlf	Syncs Spanish resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlf	Syncs German resources.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlf	Syncs Czech resources.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Adds dispatcher diagnostics.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.cs	Exposes a resource to tests.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt	Tracks new public APIs.
src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt	Tracks new internal APIs.
src/Platform/Microsoft.Testing.Platform/Hosts/ToolsTestHost.cs	Adds pipe-aware tool execution.
src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.Modes.cs	Builds processors and dispatcher connections.
src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.cs	Registers built-in dispatcher components.
src/Platform/Microsoft.Testing.Platform/Hosts/ITestHostBuilder.cs	Adds processor registration internally.
src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs	Advertises processor capabilities.
src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/IArtifactPostProcessor.cs	Defines processor contracts and DTOs.
src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/IArtifactPostProcessingManager.cs	Defines processor registration.
src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingManifest.cs	Parses orchestration manifests.
src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingManager.cs	Builds registered processors.
src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingHandshakeProperties.cs	Serializes processor capabilities.
src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingDispatcherTool.cs	Routes and reports processed artifacts.
src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingDispatcherTool.CommandLine.cs	Defines dispatcher CLI options.
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineHandler.cs	Hides internal tools from info.
src/Platform/Microsoft.Testing.Platform/Builder/TestApplicationBuilder.cs	Exposes registration managers.
src/Platform/Microsoft.Testing.Platform/Builder/ITestApplicationBuilder.cs	Adds public registration surfaces.
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxReportExtensions.cs	Registers TRX processor and tool.
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxReportEngine.Utilities.cs	Generates deterministic merge IDs.
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxMergeTool.cs	Implements merge-trx.
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxMergeTool.CommandLine.cs	Defines merge-tool options.
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxCompareTool.cs	Marks the comparer visible.
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxArtifactPostProcessor.cs	Implements automatic TRX merging.
src/Platform/Microsoft.Testing.Extensions.TrxReport/ToolTrxMergeFactory.cs	Creates merge-tool components.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Syncs Traditional Chinese TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Syncs Simplified Chinese TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.tr.xlf	Syncs Turkish TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.ru.xlf	Syncs Russian TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Syncs Portuguese TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.pl.xlf	Syncs Polish TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.ko.xlf	Syncs Korean TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.ja.xlf	Syncs Japanese TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.it.xlf	Syncs Italian TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.fr.xlf	Syncs French TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.es.xlf	Syncs Spanish TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.de.xlf	Syncs German TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.cs.xlf	Syncs Czech TRX resources.
src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/ExtensionResources.resx	Adds TRX merge diagnostics.
src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt	Tracks TRX internal APIs.

Review details

• Files reviewed: 62/62 changed files
• Comments generated: 4
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 63/63 changed files
• Comments generated: 3
• Review effort level: Medium

[Copilot]

Review details

Comments suppressed due to low confidence (2)

src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingManifest.cs:32

• This normalization is still incomplete on non-NETCOREAPP targets. JsonConfigurationFileParser.netstandard.cs:34 casts the deserialized root directly to JsonObject, so a valid non-object JSON root such as [] throws InvalidCastException, not JsonException. That exception bypasses both this wrapper and the dispatcher's IOException/UnauthorizedAccessException/FormatException filter, faulting the tool instead of returning the localized invalid-manifest result. Normalize InvalidCastException here as well.

        catch (JsonException ex)
        {
            throw new FormatException(ex.Message, ex);
        }

src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingManifest.cs:48

• An empty or scalar input entry is silently discarded instead of being rejected. The JSON parser represents {} in the first array slot as the key inputs:0, but this parts.Length >= 3 filter ignores that key, so inputs: [{}] loads as an empty manifest and never reaches the required-path check below. Include two-part index keys so every declared input is validated.

        {
            throw new FormatException(PlatformResources.ArtifactPostProcessingManifestInvalid);
        }

        int[] inputIndices = [.. values.Keys
            .Where(key => key.StartsWith("inputs:", StringComparison.Ordinal))
            .Select(key => key.Split(':'))

• Files reviewed: 63/63 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

Comments suppressed due to low confidence (2)

src/Platform/Microsoft.Testing.Platform/Tools/IToolsManager.cs:16

• The promoted registration surface is not usable for third-party tools that need arguments. Tool option scoping now requires IToolCommandLineOptionsProvider (ToolsTestHost.cs:207-210), but that interface is internal (Tools/IToolCommandLineOptionsProvider.cs:8), so an external extension can register an ITool here but cannot associate any command-line options with it; ordinary public ICommandLineOptionsProvider implementations are rejected as unknown options. Expose a public way to bind option providers to a tool (for example, make the scoped provider contract public or add an options-aware registration overload).
  src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Constants.cs:126
• The published dotnet-test pipe specification is now stale: docs/mstest-runner-protocol/004-protocol-dotnet-test-pipe.md:337-345 lists only the prior host types, execution modes, and properties through ID 13. Document ArtifactPostProcessor, tool, and capability properties 14/15 so SDK and host implementers have an accurate wire contract.

• Files reviewed: 63/63 changed files
• Comments generated: 2
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 70/70 changed files
• Comments generated: 4
• Review effort level: Medium

[Copilot]

Review details

Comments suppressed due to low confidence (1)

src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingManifest.cs:101

• Malformed optional metadata is silently accepted. For example, "kind": { "value": "trx" } has no scalar at the kind key, so this first branch returns null before inspecting the raw JSON; empty objects/arrays follow the same path. This contradicts the manifest's string-or-null schema and the strict handling already applied to numeric metadata. Validate the raw JSON type whenever the property exists, then read its scalar value.

• Files reviewed: 75/75 changed files
• Comments generated: 0 new
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 75/75 changed files
• Comments generated: 2
• Review effort level: Medium

[Copilot]

Review details

Comments suppressed due to low confidence (1)

src/Platform/Microsoft.Testing.Platform/Extensions/ArtifactPostProcessing/ArtifactPostProcessingDispatcherTool.cs:138

• This path comparison is incorrect on case-insensitive non-Windows filesystems (notably default macOS volumes): Ordinal treats Input.trx and input.trx as different even when they resolve to the same file. It also compares lexical paths without resolving symlinks. Consequently, a processor output that aliases an input can pass this guard, violating the read-only-input contract, and a casing variant of the output directory can be rejected despite being valid. Please canonicalize paths (including links where supported) and determine case sensitivity for the output filesystem, as SharedExtensionHelpers/MergeOutputFileHelper.cs:17-42 does.

        StringComparison pathComparison = RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
            ? StringComparison.OrdinalIgnoreCase
            : StringComparison.Ordinal;

        return outputPath.StartsWith(outputDirectoryPrefix, pathComparison)
            && File.Exists(outputPath)
            && !inputs.Any(input => string.Equals(Path.GetFullPath(input.Path), outputPath, pathComparison))

• Files reviewed: 75/75 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

🔍 Build Failure Analysis

⚠️ Partial analysis — the binlog-mcp MCP server was unreachable from this sandbox (network access blocked by security policy). The binary logs could not be queried programmatically.

What we know

• Azure DevOps build: [#1518663]((dev.azure.com/redacted) — failed
• PR head SHA: 09cf260d2e72473f905d0c5fc6eaf9c1cd83453c
• Binlogs staged (6 legs): Build_Linux_Release, Build_Windows_NT_Release, Build_Linux_Debug, Build_Darwin_Release, Build_Windows_NT_Debug, Build_Darwin_Debug

What to do

Please inspect the originating [Azure DevOps build]((dev.azure.com/redacted) directly for the compiler errors, then check:

1. Build errors (CS####, SA####, RS####, CA####) in the build log
2. Public API surface — if new public members were added, ensure PublicAPI.Unshipped.txt is updated
3. Nullable annotations — recent nullability changes cascade across call sites

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 47.6 AIC · ⌖ 5.06 AIC · ⊞ 6.2K · [◷]( · ◷)

[Copilot]

Review details

• Files reviewed: 74/74 changed files
• Comments generated: 0 new
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 74/74 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #10086

32 tests graded across 7 files. 28 × A, 4 × C. The most common issue is a single IsTrue(result.IsValid) assertion that confirms success but does not capture which option caused it, and two tests that bundle multiple distinct behaviors (valid + several invalid scenarios) in one method body. Consider splitting the multi-scenario tests and enriching the boolean-only assertions with an ErrorMessage or similar check.

Grade	Test	Notes
C (70–79)	mod ArtifactPostProcessingTests. ValidateProcessedArtifact_ RejectsInvalidPaths	Tests valid path, missing file, outside-dir, and reused-input in one body — consider splitting into four focused tests.
C (70–79)	mod CommandLineHandlerTests. ParseAndValidateAsync_ ToolOptionDoesNotConflictWith NormalExtension	Only Assert.IsTrue(result.IsValid) — consider also asserting the absence of an ErrorMessage to anchor the contract.
C (70–79)	mod CommandLineHandlerTests. ParseAndValidateAsync_ RepeatedToolOption_ ValidatesAggregatedArguments	Only Assert.IsTrue(result.IsValid); the mock is set up to return failure for wrong counts, but the test never checks the error path — consider adding a negative DataRow.
C (70–79)	new TrxArtifactPostProcessorTests. ProcessAsync_ WithTwoInputs_ WritesUniquelyNamedMergedReport	Two behaviors (merge produces output; reordered inputs produce same file) in ~50 lines — consider splitting idempotency into its own test.
A (90–100)	new ArtifactPostProcessingTests. FindProcessorConflicts_ DuplicateCapabilitiesWithinOneProcessor_ ReturnsEmpty	No issues found.
A (90–100)	new ArtifactPostProcessingTests. FindProcessorConflicts_ KindMatchingAnotherProcessorExtension_ ReturnsEmpty	No issues found.
A (90–100)	new ArtifactPostProcessingTests. FindProcessorConflicts_ SameCapabilityAcrossProcessors_ ReturnsFirstProcessor	No issues found.
A (90–100)	new ArtifactPostProcessingTests. HandshakeProperties_ AreSortedAndDeduplicated	No issues found.
A (90–100)	new ArtifactPostProcessingTests. HandshakeProperties_ WithNoCapabilities_ ReturnsNull	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manager_ BuildsOnlyEnabledProcessors	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manager_ CapabilityContainingSeparator_ ThrowsInvalidOperationException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manager_ InvalidCapability_ ThrowsInvalidOperationException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ LoadsVersionedAttributedInputs	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ WithEmptyInputEntry_ ThrowsFormatException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ WithInputMissingPath_ ThrowsFormatException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ WithInputsThatIsNotArray_ ThrowsFormatException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ WithInvalidValueType_ ThrowsFormatException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ WithMalformedJson_ ThrowsFormatException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ WithTopLevelArray_ ThrowsFormatException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. Manifest_ WithUnsupportedVersion_ ThrowsFormatException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. ProcessedArtifact_ BlankKind_ ThrowsArgumentException	No issues found.
A (90–100)	new ArtifactPostProcessingTests. SupportsArtifactPostProcessing_ ReturnsExpectedValue	No issues found.
A (90–100)	mod CommandLineHandlerTests. ParseAndValidateAsync_ UnknownJsonOptionForSelectedTool_ ReturnsInvalid	No issues found.
A (90–100)	new DotnetTestPipeArtifactPostProcessingTests. Dispatcher_ AdvertisesCapabilityAndReturnsMergedArtifact	No issues found.
A (90–100)	new DotnetTestPipeArtifactPostProcessingTests. Dispatcher_ InvalidOutputDirectory_ ReturnsInvalidCommandLine	No issues found.
A (90–100)	new DotnetTestPipeArtifactPostProcessingTests. MergeTool_ OptionsFromTestConfig_ WritesOutputTrx	No issues found.
A (90–100)	new DotnetTestPipeArtifactPostProcessingTests. MergeTool_ RepeatedInputs_ WritesOutputTrx	No issues found.
A (90–100)	new DotnetTestPipeArtifactPostProcessingTests. MergeTool_ UnexpectedPositionalArgument_ ReturnsInvalidCommandLine	No issues found.
A (90–100)	mod DotnetTestProtocolContractTests. HandshakeMessageExecutionModes_ AreStable	No issues found.
A (90–100)	mod DotnetTestProtocolContractTests. HandshakeMessagePropertyNames_ AreStable	No issues found.
A (90–100)	mod ProtocolTests. HandshakeMessageExecutionModes_ ValuesAreStable	No issues found.
A (90–100)	mod ProtocolTests. HandshakeMessagePropertyNames_ ValuesAreStable	No issues found.
A (90–100)	new TrxArtifactPostProcessorTests. Capabilities_ DescribeTrxArtifacts	No issues found.
A (90–100)	new TrxArtifactPostProcessorTests. CreateMergeRunId_ IsIndependentOfInputOrder	No issues found.
A (90–100)	new TrxArtifactPostProcessorTests. CreateMergeRunId_ WithDifferentExecutionId_ ProducesDifferentId	No issues found.
A (90–100)	new TrxArtifactPostProcessorTests. ProcessAsync_ WithFewerThanTwoInputs_ ReturnsNull	No issues found.
A (90–100)	new ToolsHostTests. AggregateArguments_ CombinesRepeatedOptionOccurrences	No issues found.
A (90–100)	new ToolsHostTests. ValidateCommandLineOptionsAsync_ ReturnsProviderFailure	No issues found.
A (90–100)	new ToolsHostTests. ValidateToolProviders_ DuplicateToolOption_ ReturnsInvalid	No issues found.
A (90–100)	new ToolsHostTests. ValidateToolProviders_ SystemOptionCollision_ ReturnsInvalid	No issues found.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 136.7 AIC · ⌖ 5.44 AIC · ⊞ 8.9K · [◷]( · ◷)