Skip to content

Add Microsoft.Testing.Extensions.JUnitReport extension (#4268)#8850

Merged
Evangelink merged 13 commits into
microsoft:mainfrom
Evangelink:dev/amauryleve/junit-report
Jun 8, 2026
Merged

Add Microsoft.Testing.Extensions.JUnitReport extension (#4268)#8850
Evangelink merged 13 commits into
microsoft:mainfrom
Evangelink:dev/amauryleve/junit-report

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes #4268

CLI

  • --report-junit — enable the report
  • --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

  • Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
  • MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
  • Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
  • Atomic write: temp file + rename to avoid partial reports.
  • Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
  • Outcome mapping: passed → empty body, skipped<skipped>, failed<failure>, errored/timedOut<error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
  • XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

  • 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
  • MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
  • Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

  • 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
    • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
    • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
    • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
    • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
    • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
  • HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

Adds a new MTP extension that emits JUnit-format XML reports compatible
with Jenkins, GitLab, Azure DevOps and other CI parsers.

CLI options:
  --report-junit
  --report-junit-filename <name>

Design highlights (see docs/RFCs/016-JUnit-Report.md):
- Flat <testsuites>/<testsuite> for broad parser compatibility.
- MTP tree-of-tests preserved per testcase via <property name="testpath"/>
  rather than nested <testsuite>, which Jenkins/Surefire handle inconsistently.
- Strict <testcase> child ordering (properties -> skipped/error/failure ->
  system-out/-err) required by Surefire/Jenkins.
- Atomic write via temp file + rename.
- Defensive caps (testpath length, parent-chain depth with cycle detection)
  and stream/message/identity/trait memory limits.
- Outcome mapping: passed -> empty, skipped -> <skipped>, failed -> <failure>,
  errored/timedOut -> <error>. Discovered/InProgress nodes are not emitted as
  testcases but participate in the parent chain.

Implementation:
- 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport (net8.0,
  net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
- MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired
  into the platform csproj and TestFx.slnx.
- Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf.

Tests (all green):
- 5 new acceptance tests x 3 TFMs (JUnitReportTests).
- HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated
  with the new option/registration assertions.

Closes microsoft#4268

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 5, 2026 07:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

  • Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
  • Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
  • Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.
Show a summary per file
File Description
TestFx.slnx Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md Adds the RFC describing schema/design/testing strategy.

Copilot's findings

  • Files reviewed: 35/35 changed files
  • Comments generated: 8

Comment thread src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs Outdated
Comment thread src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs Outdated
Comment thread src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs Outdated
Comment thread src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs Outdated
Verifies that {tfm}, {pname} and {asm} placeholders in the JUnit report file name template are resolved end-to-end.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@Youssef1313 Youssef1313 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

- Rename root/per-suite `disabled` attribute to `skipped` (Jenkins/Surefire correct name)
- Map cancelled outcome to <error>/errors++ (was incorrectly <failure>/failures++)
- Emit <skipped> element for skipped outcome (was missing)
- Fix BuildTestPath to always append the leaf DisplayName even when no parent
- Rename property name=`test-uid` to `uid` for consistency
- Prefix trait property keys with `trait.`
- Classify CancelledTestNodeStateProperty as `cancelled` in TestResultCapture (with CS0618/MTP0001 pragma)
- Add acceptance test JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes covering passed/failed/skipped/errored mix with parent container

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 5, 2026 12:05
@Evangelink

Copy link
Copy Markdown
Member Author

@Youssef1313 great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 35/35 changed files
  • Comments generated: 2

Comment thread src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs Outdated
Evangelink and others added 2 commits June 5, 2026 17:34
- Use unique per-run temp file name (Path.GetRandomFileName) to avoid
  collisions between concurrent runs that share the same default report
  name. Replaces the fixed '<final>.tmp' path.
- Add XDocument-based <testcase> child ordering assertion in the
  JUnit_WhenTestsFailOrSkip acceptance test to enforce the RFC 016
  normative element order beyond substring checks.
- Register Microsoft.Testing.Extensions.JUnitReport in
  Microsoft.Testing.Platform.slnf and NonWindowsTests.slnf so the
  package is produced for Linux/MacOS Debug Test jobs.
- Fix markdown lint MD038 (spaces inside code spans) in RFC 016 by
  moving the leading space out of '\[attempt 2]\' / '\[attempt 3]\'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…STest acceptance

- RFC 016: add JUnit flavors comparison table; expand retry-handling section covering both MSTest [Retry] and MTP --retry-failed-tests; clarify duplicate-name suffix wording.
- MTP acceptance: introduce AssertWellFormedJUnitReport helper using XDocument; assert <testcase> child ordering; add DummyTestFramework mixed-outcomes test.
- MSTest acceptance: add JUnitReportTests covering [Retry(N)] attribute and the MTP --retry-failed-tests extension.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 5, 2026 16:43
@Evangelink

Copy link
Copy Markdown
Member Author

@Evangelink follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

  • MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
  • MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
  • When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

  • JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
  • JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 38/38 changed files
  • Comments generated: 2

Comment thread docs/RFCs/016-JUnit-Report.md Outdated
Comment thread docs/RFCs/016-JUnit-Report.md Outdated
Evangelink and others added 2 commits June 5, 2026 19:05
- Use --list-tests (the actual CLI option) instead of --discover-tests.
- Document that reserved Windows filenames are sanitized (prefixed with _), not rejected, matching ReplaceInvalidFileNameChars and the HtmlReport behavior.
- Spell out the full list of reserved names and the invalid character set.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 7, 2026 07:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 38/38 changed files
  • Comments generated: 1

…e / RFC 016

Address review feedback on PR microsoft#8850.

The previous marker was inconsistent with the repo's standard truncation
format (TestResultCapture.Truncate) and with RFC 016, which both specify
the marker as `\n…[truncated, original length: N]` where N is the real
untruncated length.

Two issues are fixed:

1. The marker now has the leading `\n` prefix so XML consumers and log
   readers can split on it cleanly.
2. The reported "original length" is now the full untruncated testpath
   length (segments + separators), computed up front, rather than the
   partially-built StringBuilder length at the moment we exceeded the
   cap (which omits remaining segments and is therefore strictly
   smaller).

Also adds the surrogate-pair safety check from TestResultCapture so the
cut at MaxTestPathLength never splits a UTF-16 high surrogate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
Member Author

Follow-ups:

  • Experimental marker: the public surface is already gated on TPEXP:

    • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
    • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
    • > **⚠️ Experimental:** callout in PACKAGE.md

    This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

  • ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

Match the versioning scheme used by other experimental Microsoft.Testing.Platform extensions (Logging, OpenTelemetry, AI). Packages will now ship as 1.0.0-alpha.* instead of the repo-default *-preview.*.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 8, 2026 08:45
@Evangelink

Copy link
Copy Markdown
Member Author

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 38/38 changed files
  • Comments generated: 2

Replace XDocument-based structural checks with verbatim XML snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable attributes (time, timestamp, hostname) are normalized via attribute-scoped regexes; everything else — attribute order, suite ids, counts, exit-code, properties block contents, self-closing failure/skipped/error elements — must match byte-for-byte.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
After the JUnitReport package was moved to the experimental versioning scheme (1.0.0-alpha) it no longer matches the platform version requested by the acceptance test asset csproj. Introduce a dedicated MicrosoftTestingExtensionsJUnitReportVersion property on AcceptanceTestBase and a \\\$\ placeholder so the dummy app references the package at the version actually produced by the local pack.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
Member Author

Follow-up commits on this branch:

  • 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
  • f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
  • 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

- Truncate RawUid, ParentRawUid, and the _parentChain dictionary key to MaxIdentityFieldLength, matching the existing truncation of Uid/DisplayName. Without this cap, an attacker-controlled or pathological test framework could exhaust memory by emitting a very long TestNode.Uid.

- Switch the HelpInfoAllExtensionsTests asset csproj to the dedicated \\$ placeholder (the JUnit extension now follows the experimental versioning scheme and no longer matches \\$).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 8, 2026 11:45
@Evangelink

Copy link
Copy Markdown
Member Author

Follow-up commit on this branch:

  • 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 39/39 changed files
  • Comments generated: 4

The JUnitReport extension ships under the experimental 1.0.0-alpha versioning scheme, so the acceptance test assets that reference it must use MicrosoftTestingExtensionsJUnitReportVersion instead of MicrosoftTestingPlatformVersion. HelpInfoAllExtensionsTests was already updated; this fixes the remaining two assets (MSBuild known-extension registration and MSTest JUnit retry tests) where NuGet restore was failing with NU1102.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink
Evangelink enabled auto-merge (squash) June 8, 2026 14:39
@Evangelink
Evangelink merged commit 233eec9 into microsoft:main Jun 8, 2026
33 checks passed
@Evangelink
Evangelink deleted the dev/amauryleve/junit-report branch June 8, 2026 14:46
Evangelink added a commit to Evangelink/testfx that referenced this pull request Jun 9, 2026
Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest
report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing
pipeline end-to-end as part of our own CI test runs.

JUnit:
- test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport
  as a ProjectReference for every test project that opts into MTP and
  wires --report-junit / --report-junit-filename into the MSBuild path.
- azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass
  --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release
  dotnet test --test-modules path (which bypasses MSBuild evaluation).
- Each test project's Program.cs explicitly calls AddJUnitReportProvider()
  (TestingPlatformBuilderHook auto-registration only fires for NuGet
  consumers, not source ProjectReferences).

OpenTelemetry:
- test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry
  as a ProjectReference.
- Each test project's Program.cs registers AddOpenTelemetryProvider with
  AddTestingPlatformInstrumentation on both the tracer and meter builders.
  No exporter is wired: the SDK still creates real Activities and Counters
  because the sources/meters now have listeners, so data flows through the
  full OpenTelemetryResultHandler pipeline and is dropped at the export
  stage. Sufficient to exercise the handler in CI without polluting logs.
- VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match
  its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for junit test result

5 participants