Make ReflectionFree the default source-gen mode + harden AOT acceptance tests#9777
Conversation
MSTestSourceGenMode now defaults to ReflectionFree (was Rooting) in MSTest.TestAdapter.targets. The source generator is only active for Native AOT / trim consumers (PublishAot or a manual MSTest.SourceGeneration reference), and ReflectionFree is a superset of Rooting (materialized attributes + delegate invokers), so it is the safer default for that audience. The SourceGeneration acceptance metadata mode now passes -p:MSTestSourceGenMode=Rooting explicitly so the Rooting path stays covered now that it is no longer the default. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Set MSTestSourceGenMode=ReflectionFree explicitly in the Native AOT acceptance asset (it is also the shipped default now) and correct the stale comments: the asset genuinely runs reflection-free, and warnAsError stays off because MSTest.TestAdapter's native MTP path uses the vstest Microsoft.TestPlatform.ObjectModel TestCase/TestProperty types regardless of source-gen mode, emitting IL20xx/IL30xx outside this repo's control. Enabling warnAsError is blocked on removing that ObjectModel reachability (tracked by #9769). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates MSTest’s AOT/trim source-generation defaults by making ReflectionFree the shipped default MSTestSourceGenMode, while keeping the Rooting path explicitly covered in acceptance/unit tests and clarifying related documentation/comments across the repo.
Changes:
- Switch default
MSTestSourceGenModeinMSTest.TestAdapter.targetsfromRootingtoReflectionFree. - Harden acceptance coverage by explicitly passing/pinning
MSTestSourceGenModefor both source-gen variants and Native AOT. - Refresh analyzer/generator/test comments to reflect the new default and mode-selection behavior.
Show a summary per file
| File | Description |
|---|---|
| test/Utilities/Microsoft.Testing.TestInfrastructure/MetadataMode.cs | Updates enum doc comment to reflect ReflectionFree is now the shipped default for AOT source-gen mode. |
| test/Utilities/Microsoft.Testing.TestInfrastructure/AcceptanceSourceGen.cs | Ensures both acceptance build variants pass MSTestSourceGenMode explicitly (including Rooting) to preserve coverage independent of product defaults. |
| test/UnitTests/MSTest.SourceGeneration.UnitTests/MSTestReflectionMetadataGeneratorTests.cs | Clarifies unit-test comments around generator fallback behavior when the property is unset. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/NativeAotTests.cs | Pins MSTestSourceGenMode=ReflectionFree in the AOT asset and expands rationale for not enabling warnAsError. |
| src/Analyzers/Shared/SourceGenModeHelper.cs | Updates helper documentation to reflect ReflectionFree is the shipped default (from adapter targets) and that “unset” falls back to rooting behavior. |
| src/Analyzers/MSTest.SourceGeneration/Generators/ReflectionMetadataGenerator.cs | Updates comment explaining which generator emits under which mode (and should account for the “unset” fallback). |
| src/Adapter/MSTest.TestAdapter/buildTransitive/common/MSTest.TestAdapter.targets | Changes default MSTestSourceGenMode to ReflectionFree and updates the explanatory comment accordingly. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 6
- Review effort level: Low
🔴 Build Failure Analysis
Root Cause: Missing Internal API declarations (RS0051)Two new
Each error is repeated 3× (once per target framework:
|
- Re-add the UTF-8 BOM (utf-8-bom per .editorconfig) to the five .cs files whose BOM was accidentally stripped while editing their leading lines. - Clarify the rooting generator comment: it emits whenever the mode is not ReflectionFree, i.e. both when Rooting is selected and when the property is left unset (IsReflectionFree returns false in both cases). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The CI build (Windows/Linux legs) was already red on main before this PR due to two undeclared internal API surfaces flagged by RS0051: - BaseSerializer.ReadFields / WriteListPayload<T> (added by #9774) — the file is linked into Microsoft.Testing.Platform and the HangDump, Retry, TrxReport and Extensions.MSBuild extensions, so each project's InternalAPI.Unshipped.txt needs the declarations. - PlatformServicesConfigurationAdapter and its members (from #9755) in MSTest.TestAdapter, which only surface on the Windows/net462 TFMs the Linux analysis build never compiled. These are unrelated to the source-gen default change but block this PR's pipeline, so they are fixed here in a separate commit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Complete the #9755 BridgedConfiguration -> PlatformServicesConfigurationAdapter rename: the old type name still lingered in InternalAPI.Shipped.txt and referenced a type that no longer exists. The new name was already added to InternalAPI.Unshipped.txt; drop the three stale shipped entries so the tracked internal-API surface is accurate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
/backport to rel/4.3 |
|
Started backporting to rel/4.3: https://github.com/microsoft/testfx/actions/runs/29034521432 |
|
@Evangelink backporting to rel/4.3 failed, the patch most likely resulted in conflicts: $ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch
Applying: Make ReflectionFree the default MSTest source-gen mode
Applying: Harden NativeAotTests to explicitly exercise reflection-free source gen
Applying: Address PR review: restore UTF-8 BOM and clarify rooting-mode comment
Applying: Fix pre-existing RS0051 InternalAPI-tracking breakage on main
Using index info to reconstruct a base tree...
A src/Adapter/MSTest.TestAdapter/InternalAPI/InternalAPI.Unshipped.txt
A src/Platform/Microsoft.Testing.Extensions.HangDump/InternalAPI/InternalAPI.Unshipped.txt
A src/Platform/Microsoft.Testing.Extensions.MSBuild/InternalAPI.Unshipped.txt
A src/Platform/Microsoft.Testing.Extensions.Retry/InternalAPI/InternalAPI.Unshipped.txt
A src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt
A src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt
Falling back to patching base and 3-way merge...
Applying: Remove stale BridgedConfiguration entries from InternalAPI.Shipped.txt
Using index info to reconstruct a base tree...
A src/Adapter/MSTest.TestAdapter/InternalAPI/InternalAPI.Shipped.txt
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): src/Adapter/MSTest.TestAdapter/InternalAPI/InternalAPI.Shipped.txt deleted in HEAD and modified in Remove stale BridgedConfiguration entries from InternalAPI.Shipped.txt. Version Remove stale BridgedConfiguration entries from InternalAPI.Shipped.txt of src/Adapter/MSTest.TestAdapter/InternalAPI/InternalAPI.Shipped.txt left in tree.
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0005 Remove stale BridgedConfiguration entries from InternalAPI.Shipped.txt
Error: The process '/usr/bin/git' failed with exit code 128NOTE: A PR will be created, but needs to be revised manually! |
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
Verdict Table
| # | Dimension | Verdict |
|---|---|---|
| 1 | Algorithmic Correctness | ✅ Clean |
| 2 | Threading & Concurrency | N/A |
| 3 | Security & IPC Contract Safety | N/A |
| 4 | Public API & Binary Compatibility | ✅ Clean |
| 5 | Performance & Allocations | N/A |
| 6 | Cross-TFM Compatibility | ✅ Clean |
| 7 | Resource & IDisposable Management | N/A |
| 8 | Defensive Coding at Boundaries | N/A |
| 9 | Error Handling & Diagnostics | N/A |
| 10 | Test Coverage & Quality | ✅ Clean |
| 11 | Naming, Readability & Documentation | ✅ Clean |
| 12 | Code Style & Conventions | ✅ Clean |
| 13 | File Encoding & Formatting | ✅ Clean |
| 14 | MSBuild / Project File Hygiene | ✅ Clean |
| 15 | Localization | N/A |
| 16 | Public API Declaration | N/A |
| 17 | Init Accessor Rule | N/A |
| 18 | TODO / FIXME Comment Policy | ✅ Clean |
| 19 | CLI Options & Help Text Consistency | N/A |
| 20 | Scope Discipline | ✅ Clean |
| 21 | Test Infrastructure Correctness | ✅ Clean |
| 22 | Miscellaneous / Code Health | ✅ Clean |
Overall Assessment
All clear — no issues found. This is a well-structured PR with correct, self-consistent changes.
Key observations
-
Default change is safe:
MSTestSourceGenModeswitching fromRooting→ReflectionFreeinMSTest.TestAdapter.targetsonly affects the AOT/trim audience that opts into the source generator. TheCondition="'$(MSTestSourceGenMode)' == ''"guard preserves explicit consumer overrides. ReflectionFree is a superset of Rooting, making it the safer default. -
Generator fallback is correctly documented:
SourceGenModeHelper.IsReflectionFree()logic is unchanged — it returnsfalsewhen the property is unset (no MSBuild targets imported), falling back to rooting. The updated comments acrossReflectionMetadataGenerator.cs,SourceGenModeHelper.cs, and the unit test accurately describe this two-layer default (MSBuild default vs. generator fallback). -
Critical test infrastructure fix in
AcceptanceSourceGen.cs: Changing theSourceGenerationvariant fromstring.Empty→" -p:MSTestSourceGenMode=Rooting"is essential — without it, the Rooting test path would silently run ReflectionFree (the new product default) and lose coverage of the Rooting generator. TheGetOutputSubFolderswitch expression guard preventsMetadataMode.Reflectionfrom reaching this path. -
NativeAotTestspinning is good practice: Explicitly setting<MSTestSourceGenMode>ReflectionFree</MSTestSourceGenMode>in the test asset makes the test resilient to future default changes while clearly documenting intent. -
BOM restorations are correct: The five
// Copyrightline-1 diffs are UTF-8 BOM restorations matching the.editorconfigutf-8-bomrule.
Merge recommendation
✅ No blocking or major issues. The PR is ready to merge once CI completes green.
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (1)
src/Adapter/MSTest.TestAdapter/InternalAPI/InternalAPI.Shipped.txt:249
InternalAPI.Shipped.txtstill declaresMicrosoft.VisualStudio.TestTools.UnitTesting.MSTestBridgedTestFramework, but there are no source references/definitions for this type anywhere in the repo (a full search only finds it in this API file). Leaving stale entries here makes the declared internal surface inaccurate and can trigger RS0052 (“declared API … could not be found”) on analysis builds. Remove theMSTestBridgedTestFrameworkentries here (and also remove any otherMSTestBridgedTestFrameworkentries elsewhere in the file, e.g. overrides) so the tracked surface matches the current code.
Microsoft.VisualStudio.TestTools.UnitTesting.MSTestBannerCapability
Microsoft.VisualStudio.TestTools.UnitTesting.MSTestBannerCapability.GetBannerMessageAsync() -> System.Threading.Tasks.Task<string?>!
Microsoft.VisualStudio.TestTools.UnitTesting.MSTestBannerCapability.MSTestBannerCapability(Microsoft.Testing.Platform.Services.IPlatformInformation! platformInformation) -> void
Microsoft.VisualStudio.TestTools.UnitTesting.MSTestBridgedTestFramework
Microsoft.VisualStudio.TestTools.UnitTesting.MSTestBridgedTestFramework.MSTestBridgedTestFramework(Microsoft.VisualStudio.TestTools.UnitTesting.MSTestExtension! mstestExtension, System.Func<System.Collections.Generic.IEnumerable<System.Reflection.Assembly!>!>! getTestAssemblies, System.IServiceProvider! serviceProvider, Microsoft.Testing.Platform.Capabilities.TestFramework.ITestFrameworkCapabilities! capabilities) -> void
- Files reviewed: 8/8 changed files
- Comments generated: 0 new
- Review effort level: Low
This comment has been minimized.
This comment has been minimized.
These entries referenced symbols not present as public API in these assemblies (BaseSerializer.ReadFields/WriteListPayload and PlatformServicesConfigurationAdapter), causing RS0017 build failures. The original PR microsoft#9777 did not touch any PublicAPI.Unshipped.txt files. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Flipping the default MSTestSourceGenMode to ReflectionFree changed which generator emits, and two acceptance assertions hard-coded the rooting generator's output filename: - SourceGenerationNonAotTests asserted '<AssemblyName>.MSTestReflectionMetadata.g.cs' (the rooting generator's file). Under the ReflectionFree default the package instead emits 'MSTestReflectionMetadata.Registry.g.cs' (+ SupportTypes/Registration), so the assertion failed. Match '*MSTestReflectionMetadata*.g.cs' to prove the generator ran regardless of the selected mode. - TrimAndAotAssertions.MSTestOwnedSourceFiles guarded only 'MSTestReflectionMetadata.g.cs'; that substring no longer matches the reflection-free registry files, silently defeating the guard. Add the reflection-free file names (and the mode-agnostic '.MSTestReflectionMetadata.g.cs' suffix) so the guard stays effective in both modes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
…Type) Flipping the default to ReflectionFree surfaced (via the strengthened TrimAndAotAssertions guard) that the reflection-free generator's own emitted registry was not trim-clean: 'MSTestReflectionMetadata.Registration.g.cs' raised IL2072 because the emitted 'TestClassReflectionInfo.Type' property has no DynamicallyAccessedMembers annotation, yet it flows into the generated ResolveMethod / ResolveProperty helpers (which require Public/NonPublic Methods and Properties). Annotate the emitted Type property with the union of those requirements. The registry assigns 'typeof(<concrete test class>)', which the trimmer treats as satisfying any DynamicallyAccessedMembers requirement, so no warning is produced at the assignment site. Verified locally: TrimTests now publishes with no MSTest-owned generated-file trim warnings, and all 102 source-generation unit tests pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The backport of microsoft#9777 dropped the change to SourceGenerationNonAotTests.cs. Once ReflectionFree becomes the default source-gen mode, the rooting generator no longer emits '<AssemblyName>.MSTestReflectionMetadata.g.cs'; the reflection-free generator emits 'MSTestReflectionMetadata.Registry.g.cs' (plus SupportTypes/Registration) instead. Match either output with a '*MSTestReflectionMetadata*.g.cs' glob so the smoke test stays independent of the default mode. Mirrors the original PR microsoft#9777. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
🧪 Test quality grade — PR #9777
This advisory comment was generated automatically. Grades are heuristic
|
…ce tests by @Evangelink in #9777 (backport to rel/4.3) (#9791) Co-authored-by: Amaury Levé <amauryleve@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
What
Follow-up to the #9769 investigation (source-gen dogfooding gap).
ReflectionFreethe defaultMSTestSourceGenMode(wasRooting) inMSTest.TestAdapter.targets.PublishAot, or a manualMSTest.SourceGenerationreference) — the ClassicEngine path never references it — so the blast radius is exactly the AOT/trim audience that benefits most.ReflectionFreeis a superset ofRooting(materialized attributes + delegate invokers on top of type/method rooting), so it is the safer default for that audience.SourceGenerationacceptance metadata mode now passes-p:MSTestSourceGenMode=Rootingexplicitly so theRootingpath stays covered now that it is no longer the default.NativeAotTeststo explicitly setMSTestSourceGenMode=ReflectionFreeand corrected the stale comments (the asset previously claimed reflection-free but ranRooting).Why not enable
warnAsErrorfor reflection-free modeInvestigated per the discussion. The IL2026/IL2072 trim errors in #9769 come from
Microsoft.TestPlatform.ObjectModel.TestObject.ConvertPropertyFrom/To<T>, dragged in byMSTest.TestAdapterand reached through the native MTP path (TestingPlatformAdapter/MSTestTestNodeConverter,MSTestFilterContext) which uses the VSTestTestCase/TestPropertytypes directly. That reachability is independent ofMSTestSourceGenMode— source-gen mode only changes user-test metadata acquisition, not the platform↔VSTest plumbing. So a blanketwarnAsErrorcannot pass in either mode until that ObjectModel reachability is removed. That fix is intentionally out of scope here (owned by @Evangelink per the issue) and is the real unblock for the reported strict build.Pipeline fix (pre-existing, unrelated to the source-gen change)
CI was already red on
mainbefore this PR due to two undeclared internal-API surfaces flagged by RS0051. Fixed here in a separate commit so this PR can go green:BaseSerializer.ReadFields/WriteListPayload<T>(added by Deduplicate DotnetTest IPC serializer 3-phase boilerplate #9774) — the file is linked intoMicrosoft.Testing.Platformand the HangDump / Retry / TrxReport / Extensions.MSBuild extensions; each project'sInternalAPI.Unshipped.txtneeded the declarations.PlatformServicesConfigurationAdapterand its members (from Drop MSTest's VSTest bridge dependency on the MTP path #9755) inMSTest.TestAdapter, which only surface on the Windows / net462 TFMs the Linux analysis build never compiled.Review feedback addressed
utf-8-bomper.editorconfig) on the five.csfiles whose BOM was accidentally stripped while editing their leading lines.Notes
SourceGenModeHelper,ReflectionMetadataGenerator,MetadataMode,AcceptanceSourceGen, and a source-gen unit test to reflect the new default.Refs #9769