Native AOT trim-warning reduction: agnostic MSTestEngine, element-sourced MTP filter, and source-gen DynamicData by Evangelink · Pull Request #9861 · microsoft/testfx

Evangelink · 2026-07-11T12:57:58Z

Summary

Reduces trim/Native-AOT warnings on the native Microsoft.Testing.Platform (MTP) path for MSTest, working toward #9769. Each commit is an independent, self-contained step; the stack builds on the recently-merged reflection-free source generator and coexists with the [DynamicallyAccessedMembers] DynamicData fix from #9832 (this branch was rebased on top of it and integrates both).

What changed (commit by commit)

Route the MTP path through an agnostic MSTestEngine — the native MTP framework (MSTestTestFramework) previously reused the VSTest-shaped MSTestDiscoverer/MSTestExecutor, so the MTP layer called the VSTest layer to reach the platform-services engine. The neutral orchestration (telemetry, STA/apartment wrapper, engine calls) is extracted into a new adapter-level MSTestEngine. Both hosts now reach the engine with neutral inputs; the VSTest classes keep their public surface. No behavior change (the run-selected-test-cases path still materializes elements lazily after settings init).
Evaluate the native MTP filter from UnitTestElement — the shared TestMethodFilter matched by materializing a vstest TestCase (ToTestCase → TestObject.SetPropertyValue/GetPropertyValue), which pulls in the trim/AOT-unsafe TestObject property converters (IL2026/IL2072) and the AdapterTestProperties custom TypeConverters. A dedicated MTP-only filter (MtpTestElementFilterProvider/MtpTestElementFilter) evaluates the filter straight from the neutral element model using hard-coded vstest property labels. Because it's a distinct type the MTP graph references exclusively, the trimmer drops the TestCase-materializing branch. This also cascaded to remove the System.Runtime.Serialization (DataContract) reachability.
DynamicDataSourceResolver seam — DynamicDataOperations.GetData / DynamicDataAttribute.GetDisplayName resolved the source member / display-name method by reflection. New public-infrastructure registry (EditorBrowsable-Never, mirroring ReflectionMetadataHook) lets the source generator supply compile-time accessors; reflection becomes the fallback for reflection mode. Integrated with Make DynamicData trimming/NativeAOT-safe via DynamicallyAccessedMembers #9832: the reflection fallback keeps its [DynamicallyAccessedMembers(All)] annotations (flowed into the extracted helpers) so it stays trim-safe too — the source-gen path avoids reflection entirely, the fallback is DAM-safe.
Emit DynamicData accessors from the reflection-free generator — the generator registers a reflection-free accessor only when a generated direct member access provably behaves exactly like the runtime reflection lookup in DynamicDataOperations; otherwise it emits no registration and the (DAM-safe) reflection fallback runs. Concretely it emits accessors for static property, field, and parameterless method sources (resolving the declaring type from the test method's containing type, honoring an explicit typeof(...) and DynamicDataSourceType, and a custom display-name method resolved declared-only). It deliberately falls back to reflection for: parameterized method sources (whose arguments need the reflection binder's widening/conversion semantics that an exact unbox cast can't replicate), void/by-ref/pointer-returning methods, ambiguous overloads, names that map to more than one member kind, and members that aren't accessible from the consuming assembly. The requested DynamicDataSourceType is part of the registration key, so an accessor generated for one attribute cannot satisfy a different attribute that names the same member with an incompatible source kind. This removes the DynamicDataOperations/DynamicDataAttribute IL2070/IL2075 warnings from AOT publish for the supported shapes.
Runtime coverage — adds a [DynamicData] property-source test to the source-gen non-AOT acceptance asset so the generated accessor is exercised end-to-end.

Verification

Full build across all TFMs (incl. UWP): 0 warnings/errors.
Unit: TestFramework DynamicData (41), source generator (112), adapter, platform-services incl. filter — all pass.
Native MTP acceptance (packed, net11.0): filter/discovery/runsettings/test-run-parameters — all pass.
Native AOT trim analysis: TestObject.ConvertProperty*, AdapterTestProperties converters, System.Runtime.Serialization, TestMethodFilter, and DynamicDataOperations/DynamicDataAttribute warnings are all gone from the MTP path. Remaining MSTest-owned reflection warnings are TypeCache.InstantiateTestFilter / ProviderDiscovery (unrelated; next candidate before warnAsError can be enabled).

Notes

Adds public infrastructure API DynamicDataSourceResolver (tracked in PublicAPI.Unshipped.txt) — intended for the source generator's module initializer only, consistent with the existing ReflectionMetadataHook precedent.
Related to [4.3.0] Native AoT test project fails to compile due to trim analysis errors #9769.

… MSTestDiscoverer/MSTestExecutor The native Microsoft.Testing.Platform framework (MSTestTestFramework) previously reused the VSTest-shaped MSTestDiscoverer/MSTestExecutor classes, so the MTP layer called the VSTest layer which then called the platform-services engine. Extract the neutral orchestration (telemetry lifetime, STA/apartment run wrapper, and the calls into MSTestDiscovererHelpers/UnitTestDiscoverer/TestExecutionManager) into a new platform-agnostic MSTestEngine at the adapter level. Every input is neutral (IAdapterMessageLogger, ITestElementFilterProvider, IUnitTestElementSink, ITestResultRecorder, run settings XML, test-run directory), so: - MSTestDiscoverer/MSTestExecutor unwrap their IDiscoveryContext/IRunContext/ IFrameworkHandle into neutral inputs and delegate to MSTestEngine (keeping their VSTest ITestDiscoverer/ITestExecutor surface + Cancel()). - MSTestTestFramework builds the neutral inputs directly and drives MSTestEngine, no longer referencing the VSTest MSTestDiscoverer/MSTestExecutor classes. Behavior is preserved: the run-selected-test-cases path materializes elements lazily (after settings initialization) so a settings error still bails out before any conversion. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

… vstest TestCase On the native Microsoft.Testing.Platform path the shared TestMethodFilter matched by materializing a vstest TestCase (UnitTestElement.GetOrCreateHostTestCase -> ToTestCase -> TestObject.SetPropertyValue) and reading it back (TestObject.GetPropertyValue). Those TestObject property converters use TypeDescriptor.GetConverter, which is trim/AOT-unsafe (IL2026/IL2072, see #9769), and referencing AdapterTestProperties rooted the custom string[]/KeyValuePair TypeConverters (DataContract serialization). Add a dedicated, MTP-only filter path: - MSTestFilterExpression now implements IUnitTestElementFilterExpression so the underlying vstest expression (which already matches purely from a Func<string,object?> provider) can be evaluated without a TestCase. - New MtpTestElementFilterProvider / MtpTestElementFilter evaluate the filter straight from the neutral UnitTestElement using hard-coded vstest property labels (FullyQualifiedName, Name, Id, ClassName, TestCategory, Priority) plus traits, so the path references neither ToTestCase nor AdapterTestProperties. - MSTestTestFramework wires the MTP discovery/run through this provider; the VSTest TestElementFilterProvider/TestMethodFilter path is unchanged. Because the two filters are distinct types and the MTP graph references only the element-sourced one, the trimmer drops the TestCase-materializing branch. A native-AOT trim analysis now shows zero TestObject.ConvertProperty*, zero custom TypeConverter, zero TestMethodFilter and zero System.Runtime.Serialization warnings on the MTP path (remaining IL20xx come from unrelated DynamicData/TypeCache reflection). Note: the vstest label for a test's display name is 'Name' (TestCaseProperties. DisplayName.Label), which is what users write as 'Name=...'. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…thout reflection DynamicDataOperations.GetData and DynamicDataAttribute.GetDisplayName resolved the data-source member / display-name method by reflecting over the declaring type (Type.GetProperty/GetMethod/GetField, TypeInfo.GetDeclaredMethod), which the trimmer cannot prove safe (IL2070/IL2075). Introduce DynamicDataSourceResolver, a public infrastructure registry (EditorBrowsable Never, mirroring ReflectionMetadataHook) that the MSTest source generator populates from a module initializer with compile-time accessors: - RegisterDataProvider(type, sourceName, Func<object?[],object?>) - RegisterDisplayNameProvider(type, methodName, Func<MethodInfo,object?[]?,string?>) DynamicDataOperations / GetDisplayName now consult the resolver first and only fall back to reflection when no accessor is registered (reflection mode). The reflection fallback is isolated into helpers annotated with justified UnconditionalSuppressMessage, since under trimming / Native AOT the source-generated registration provides the data instead. Adds TestFramework unit tests covering both the registered (resolver) and unregistered (reflection fallback) paths for data and display names. This is the framework-side seam; wiring the source generator to emit the registrations is the following change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…erator Completes the DynamicData source-gen story: the MSTest reflection-free generator now resolves each [DynamicData] source at compile time and registers a delegate with DynamicDataSourceResolver, so under trimming / Native AOT the data (and custom display name) are read without reflecting over the declaring type. This backs the framework-side suppressions added previously and removes the DynamicDataOperations / DynamicDataAttribute IL2070/IL2075 trim warnings from AOT publish. - TestClassModelBuilder resolves the [DynamicData] source member (property/method/field, including an explicit declaring type and method-source arguments) and the optional custom display-name method, degrading to no registration (runtime reflection fallback) when the member cannot be resolved / is inaccessible / has the wrong shape. - MetadataRegistryEmitter emits a DynamicDataSourceReflectionInfo per source with the accessor delegate (property/field read, or method call casting the source arguments) and the display-name accessor. - RuntimeRegistrationEmitter's module initializer registers them via DynamicDataSourceResolver.RegisterDataProvider / RegisterDisplayNameProvider. Also broadens the display-name reflection fallback suppression to IL2070. Adds source-generator unit tests for property/method-with-args/field/cross-type/custom display-name/unresolvable sources and a compile check of the emitted registration. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…OT acceptance test Adds a [DynamicData] property-source test method to the source-generated non-AOT asset so the emitted DynamicDataSourceResolver accessor is exercised end-to-end: the two data rows must discover, run, and pass through the source-generated data provider (not reflection). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Reduces Native AOT trimming warnings by separating native MTP orchestration/filtering from VSTest and generating reflection-free DynamicData accessors.

Changes:

Adds platform-agnostic MSTest discovery/execution orchestration and MTP-native filtering.
Adds a DynamicData accessor registry and source-generator support.
Expands unit and acceptance coverage.

Show a summary per file

File	Description
`test/UnitTests/TestFramework.UnitTests/Attributes/DynamicDataSourceResolverTests.cs`	Tests resolver registration and fallback.
`test/UnitTests/MSTest.SourceGeneration.UnitTests/MSTestReflectionMetadataGeneratorTests.cs`	Tests generated DynamicData accessors.
`test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SourceGenerationNonAotTests.cs`	Exercises generated DynamicData at runtime.
`src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt`	Tracks the public resolver API.
`src/TestFramework/TestFramework/InternalAPI/InternalAPI.Unshipped.txt`	Tracks internal resolver methods.
`src/TestFramework/TestFramework/Attributes/DataSource/DynamicDataSourceResolver.cs`	Implements the accessor registry.
`src/TestFramework/TestFramework/Attributes/DataSource/DynamicDataOperations.cs`	Prefers generated data accessors.
`src/TestFramework/TestFramework/Attributes/DataSource/DynamicDataAttribute.cs`	Uses generated display-name accessors.
`src/Analyzers/MSTest.SourceGeneration/Models/TestClassModel.cs`	Models resolved DynamicData sources.
`src/Analyzers/MSTest.SourceGeneration/Helpers/Constants.cs`	Adds the resolver type name.
`src/Analyzers/MSTest.SourceGeneration/Generators/TestClassModelBuilder.cs`	Resolves DynamicData source symbols.
`src/Analyzers/MSTest.SourceGeneration/Generators/RuntimeRegistrationEmitter.cs`	Registers generated accessors.
`src/Analyzers/MSTest.SourceGeneration/Generators/MetadataRegistryEmitter.cs`	Emits accessor metadata and delegates.
`src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestExecutor.cs`	Delegates execution to `MSTestEngine`.
`src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestDiscoverer.cs`	Delegates discovery to `MSTestEngine`.
`src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MtpTestElementFilter.cs`	Filters neutral MTP test elements directly.
`src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MSTestTestFramework.cs`	Routes native MTP through the neutral engine.
`src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MSTestFilterContext.cs`	Exposes element-based filter matching.
`src/Adapter/MSTest.TestAdapter/MSTestEngine.cs`	Centralizes neutral discovery and execution.
`src/Adapter/MSTest.TestAdapter/InternalAPI/uwp/InternalAPI.Unshipped.txt`	Tracks UWP engine APIs.
`src/Adapter/MSTest.TestAdapter/InternalAPI/InternalAPI.Unshipped.txt`	Tracks new adapter internals.

Review details

Files reviewed: 21/21 changed files
Comments generated: 12
Review effort level: Medium

@Class

…, BOMs, doc Tighten the source generator's [DynamicData] resolution so a reflection-free accessor is only emitted when it provably behaves like the runtime reflection lookup in DynamicDataOperations; otherwise fall back to the (DAM-safe) reflection path: - Default the source/display-name declaring type to the test method's containing type (matches methodInfo.DeclaringType), so inherited [DynamicData] resolves under the base type instead of the leaf. - Honor DynamicDataSourceType (explicit Property/Method/Field) and skip when the name maps to more than one member kind across the hierarchy (AutoDetect kind selection vs C# binding can diverge). - Validate the property getter itself (static + accessible from the consuming assembly), skip params/param-collection and by-ref source methods, and skip ambiguous method overloads. - Resolve display-name methods declared-only (GetDeclaredMethod), require RefKind.None params, and reject ambiguous overloads. - Guard declaring types to be closed + referenceable. - Escape emitted member/method identifiers so reserved-keyword names (e.g. @Class) compile. Also: add UTF-8 BOM to the four new .cs files, and correct the DynamicDataSourceResolver doc to describe the DAM-safe reflection fallback (#9832) rather than claiming it is unsafe. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f5797f94-192e-479e-a45f-2b2f8094c93b

Copilot

Review details

Files reviewed: 22/22 changed files
Comments generated: 1
Review effort level: Medium

Copilot

Review details

Files reviewed: 22/22 changed files
Comments generated: 0 new
Review effort level: Low

Copilot

Review details

Files reviewed: 22/22 changed files
Comments generated: 0 new
Review effort level: Low

…Data methods MethodInfo.Invoke applies the default binder's primitive widening / conversions (e.g. a boxed int 3 passed to a long parameter), which a generated direct call with an exact unbox cast ((long)args[0]) cannot replicate and would throw InvalidCastException. Only register reflection-free accessors for parameterless static source methods; parameterized source methods keep the DAM-safe reflection fallback. Drop the now-dead argument-cast emission and update the generator tests accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f5797f94-192e-479e-a45f-2b2f8094c93b

Copilot

Review details

Files reviewed: 22/22 changed files
Comments generated: 3
Review effort level: Medium

Copilot

Review details

Files reviewed: 22/22 changed files
Comments generated: 0 new
Review effort level: Low

…void sources - Include the requested DynamicDataSourceType in the DynamicDataSourceResolver registration/lookup key so an accessor generated for one attribute cannot satisfy a different attribute naming the same declaring type/member with an incompatible source kind (e.g. a Property registration answering a Method request instead of taking the reflection path that reports the missing method). - Skip source-gen accessors for void / by-ref / pointer-returning methods (the emitted (object?)Type.Source() would not compile); those keep the reflection fallback as before. - Simplify the resolver to carry only the member kind (methods are always parameterless now) and thread the requested source type through the model, emitted DynamicDataSourceReflectionInfo, and module-init registration. - Tests: registration-key discrimination (framework), void-method and source-kind-mismatch skips (generator). Update Internal/Public API baselines. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f5797f94-192e-479e-a45f-2b2f8094c93b

Copilot

Review details

Files reviewed: 22/22 changed files
Comments generated: 4
Review effort level: Medium

Copilot

Review details

Files reviewed: 22/22 changed files
Comments generated: 7
Review effort level: Medium

…apes, exception parity) - Skip registration when the attribute carries an undefined DynamicDataSourceType cast value (e.g. (DynamicDataSourceType)99), which would emit uncompilable DynamicDataSourceType.99; those keep the runtime fallback. - Reject source/display-name members whose generated direct access would not compile or box: static abstract/virtual (interface) members (CS8926) and pointer / function-pointer / ref-like property, method-return, and field types. - Preserve reflection exception semantics: property and method (and custom display-name) accessors now wrap thrown user exceptions in TargetInvocationException, matching PropertyInfo.GetValue / MethodInfo.Invoke; field reads stay a plain expression (FieldInfo.GetValue does not wrap). - Remove the now-unused testClassSymbol parameter from BuildMethod (IDE0060). - Tests: undefined-enum skip, field-not-wrapped, and property/display-name wrapping assertions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f5797f94-192e-479e-a45f-2b2f8094c93b

github-actions · 2026-07-13T13:52:14Z

🧪 Test quality grade — PR #9861

Grade	Test	Notes
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ DynamicDataRegistration_ Compiles`	Single "no compile errors" assertion is meaningful but consider adding a containment check on expected generated output to verify the correct code was emitted.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ DoesNotEmitDynamicDataAccessor_ ForMethodSourceWithArguments`	Embedded `userCode` string pushes body past ~30 lines; consider extracting shared stubs into named constants to reduce visual noise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ DoesNotEmitDynamicDataAccessor_ ForUndefinedSourceType`	Embedded `userCode` string pushes body past ~30 lines; consider extracting shared stubs into named constants to reduce visual noise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ DoesNotEmitDynamicDataAccessor_ ForVoidMethodSource`	Embedded `userCode` string pushes body past ~30 lines; consider extracting shared stubs into named constants to reduce visual noise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ DoesNotEmitDynamicDataAccessor_ WhenSourceKindMismatchesExplicitSourceType`	Embedded `userCode` string pushes body past ~30 lines; consider extracting shared stubs into named constants to reduce visual noise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ EmitsDynamicDataAccessor_ ForCrossTypeSource`	Embedded `userCode` string pushes body past ~30 lines; consider extracting shared stubs into named constants to reduce visual noise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ EmitsDynamicDataAccessor_ ForFieldSource`	Embedded `userCode` string pushes body past ~30 lines; the comment explaining no TargetInvocationException wrapping is useful — assertions are precise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ EmitsDynamicDataAccessor_ ForParameterlessMethodSource`	Embedded `userCode` string pushes body past ~30 lines; consider extracting shared stubs into named constants to reduce visual noise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ EmitsDynamicDataAccessor_ ForPropertySource`	Embedded `userCode` string pushes body past ~30 lines; five precise containment assertions — consider extracting shared stubs into named constants.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ EmitsDisplayNameAccessor_ WhenCustomDisplayNameSpecified`	Embedded `userCode` string pushes body past ~30 lines; consider extracting shared stubs into named constants to reduce visual noise.
B (80–89)	new `MSTestReflectionMetadataGeneratorTests. Generator_ SkipsUnresolvableDynamicDataSource`	Single containment assertion is meaningful; a complementary `NotContain` on any accessor emission would strengthen confidence.
A (90–100)	new `DynamicDataSourceResolverTests. GetDataFallsBackToReflectionWhenNoProviderRegistered`	No issues found.
A (90–100)	new `DynamicDataSourceResolverTests. GetDataFallsBackToReflectionWhenProviderRegisteredForDifferentSourceType`	Clear AAA; exception assertion confirms the reflection path is taken rather than the registered provider.
A (90–100)	new `DynamicDataSourceResolverTests. GetDataPassesSourceArgumentsToRegisteredProvider`	Captures provider arguments via closure and asserts their exact values — no issues found.
A (90–100)	new `DynamicDataSourceResolverTests. GetDataUsesRegisteredProviderInsteadOfReflection`	No issues found.
A (90–100)	new `DynamicDataSourceResolverTests. GetDisplayNameFallsBackToReflectionWhenNoProviderRegistered`	No issues found.
A (90–100)	new `DynamicDataSourceResolverTests. GetDisplayNameUsesRegisteredProviderInsteadOfReflection`	No issues found.
A (90–100)	mod `SourceGenerationNonAotTests. SourceGenerationNonAot_ BuildsAndRunsTests_ WithExitCodeZero`	Well-structured integration test; exit-code, file-existence, and summary assertions all updated to reflect the new DynamicData test case.

_{This advisory comment was generated automatically. Grades are heuristic

and informational — they do not block merging. Re-run with

/grade-tests.}

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 75.6 AIC · ⌖ 7.18 AIC · ⊞ 8.9K · [◷]( · ◷)

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Evangelink and others added 5 commits July 11, 2026 14:47

Copilot AI review requested due to automatic review settings July 11, 2026 12:57

Copilot started reviewing on behalf of Evangelink July 11, 2026 12:58 View session

Copilot AI reviewed Jul 11, 2026

View reviewed changes

github-code-quality Bot found potential problems Jul 11, 2026

View reviewed changes

Comment thread src/TestFramework/TestFramework/Attributes/DataSource/DynamicDataOperations.cs

Comment thread src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MtpTestElementFilter.cs

Comment thread src/Analyzers/MSTest.SourceGeneration/Generators/TestClassModelBuilder.cs Fixed

Evangelink marked this pull request as ready for review July 11, 2026 20:07

This comment has been minimized.

Sign in to view

Copilot AI review requested due to automatic review settings July 11, 2026 20:29

Copilot started reviewing on behalf of Evangelink July 11, 2026 20:30 View session

Copilot AI reviewed Jul 11, 2026

View reviewed changes

Comment thread src/Analyzers/MSTest.SourceGeneration/Generators/MetadataRegistryEmitter.cs Outdated

github-code-quality Bot found potential problems Jul 11, 2026

View reviewed changes

Comment thread src/Analyzers/MSTest.SourceGeneration/Generators/TestClassModelBuilder.cs

Comment thread src/Analyzers/MSTest.SourceGeneration/Generators/TestClassModelBuilder.cs

This comment has been minimized.

Sign in to view

Merge branch 'main' into dev/amauryleve/native-aot-trim-warnings-audit

a35a99c

Copilot AI review requested due to automatic review settings July 13, 2026 08:33

Copilot started reviewing on behalf of Evangelink July 13, 2026 08:33 View session