Skip to content

Skip [AssemblyFixtureProvider] under Native AOT + add MSTEST0072 analyzer#9941

Merged
Evangelink merged 6 commits into
mainfrom
dev/amauryleve/aot-il2026-assembly-fixture-provider
Jul 15, 2026
Merged

Skip [AssemblyFixtureProvider] under Native AOT + add MSTEST0072 analyzer#9941
Evangelink merged 6 commits into
mainfrom
dev/amauryleve/aot-il2026-assembly-fixture-provider

Conversation

@Evangelink

@Evangelink Evangelink commented Jul 14, 2026

Copy link
Copy Markdown
Member

Summary

[AssemblyFixtureProvider] cross-assembly discovery walks the runtime assembly reference graph (Assembly.GetReferencedAssemblies() + load-by-name via AssemblyLoadContext). That is fundamentally a reflection/runtime-loading mechanism and cannot be made reflection-free by the source generator. Under Native AOT it also surfaced an IL2026 — the last MSTest-owned trim/AOT warning a consumer hit when publishing a Native AOT test app with MSTestSourceGenMode=ReflectionFree and warnings-as-errors.

Rather than paper over it with a suppression, this PR makes the behavior explicit: skip the feature when dynamic code is unsupported, and surface it both at build time (analyzer) and at run time (trace warning).

Changes

1. Skip discovery when dynamic code is unsupported (TypeCache.ProviderDiscovery.cs)
Guard DiscoverFixturesFromProviders on RuntimeFeature.IsDynamicCodeSupported. Because ILC constant-folds that switch to false under AOT, the guarded reflection path is statically removed — so the IL2026 disappears with no suppression needed (same pattern already used in DataSerializationHelper).

2. Best-effort runtime warning
When discovery is skipped, emit a trace warning so consumers are not silently deprived of their fixtures. It scans the already-loaded assemblies (AppDomain.GetAssemblies, metadata-only via CustomAttributeData, with per-assembly failure isolation) — it deliberately does not walk the reference graph, so it stays AOT-safe. Consequently it can only see markers on assemblies that happen to be loaded (e.g. a self-applied marker on the test assembly); an unloaded referenced provider is not detectable at run time AOT-safely. Referenced providers are instead covered at build time by the analyzer. The warning is emitted through MTP's diagnostic logger (--diagnostic output).

3. New analyzer MSTEST0072AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzer
Warns at build time when the project opts into an AOT flavor detectable at build time — PublishAot (Native AOT) or RunAOTCompilation (Blazor WebAssembly AOT) — and [AssemblyFixtureProvider] is in play. It inspects both the compilation's own assembly attributes (reported at the attribute location) and referenced assemblies' attributes (reported as a no-location diagnostic), so the documented default usage — the attribute placed on a referenced fixture library consumed by an AOT test project — is covered. PublishAot and RunAOTCompilation are exposed to analyzers via new CompilerVisibleProperty entries in MSTest.TestAdapter.targets. Includes resources (+ regenerated xlf for all 13 locales), AnalyzerReleases.Unshipped.md entry, and unit tests (including the referenced-assembly and RunAOTCompilation scenarios).

4. Acceptance testAssemblyFixtureProviderNativeAotTests verifies that under PublishAot=true (which sets IsDynamicCodeSupported=false for a managed build) the referenced provider's AssemblyInitialize/AssemblyCleanup are skipped while the test still passes.

Notes

  • Analyzer coverage. MSTEST0072 detects [AssemblyFixtureProvider] whether it is declared in the compilation being built or on a referenced library, and it triggers on both build-time-detectable AOT flavors (PublishAot, RunAOTCompilation). The only case it cannot cover is a runtime that disables dynamic code without either build property being set (e.g. Mono iOS AOT), where there is no build-time signal to key off; the best-effort runtime trace warning is the fallback there, limited to already-loaded assemblies as described in change Porting latest changes. #2.
  • Supersedes the earlier suppression-based commit on this branch.

Validation

  • dotnet build of the adapter (net8.0) and MSTest.Analyzers → 0 errors (release-tracking analyzer satisfied).
  • New analyzer unit tests (7, incl. referenced-assembly and RunAOTCompilation cases) pass on net8.0.
  • Downstream: with discovery skipped under AOT, the reflection-free Native AOT publish reaches native codegen with no MSTest-owned IL20xx/IL30xx warnings.

Copilot AI review requested due to automatic review settings July 14, 2026 14:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Suppresses the IL2026 trim/AOT warning from assembly fixture provider discovery while extending acceptance-test coverage.

Changes:

  • Isolates and suppresses Assembly.GetReferencedAssemblies().
  • Adds the source file to trim/AOT warning assertions.
Show a summary per file
File Description
TypeCache.ProviderDiscovery.cs Adds the scoped IL2026 suppression helper.
TrimAndAotAssertions.cs Guards against future trim/AOT warnings from provider discovery.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Medium

@github-actions

This comment has been minimized.

AssemblyFixtureProvider cross-assembly discovery walks the runtime assembly reference graph (Assembly.GetReferencedAssemblies + load-by-name), which requires capabilities not available when the runtime cannot generate dynamic code (Native AOT, Mono iOS AOT, Blazor WASM AOT). Guard DiscoverFixturesFromProviders on RuntimeFeature.IsDynamicCodeSupported so the feature is skipped there.

Because the ILC substitutes IsDynamicCodeSupported with a constant under AOT, the guarded reflection path is statically removed, so the previously-surfaced IL2026 (from Assembly.GetReferencedAssemblies) disappears without needing an inline suppression. This supersedes the earlier suppression approach.
…ative AOT

Since [AssemblyFixtureProvider] discovery is now skipped under Native AOT (it relies on walking the runtime assembly reference graph), add an analyzer that warns at build time when a project both opts into PublishAot and declares [assembly: AssemblyFixtureProvider], so the silently-ignored feature is surfaced to the user.

PublishAot is exposed to analyzers via a new CompilerVisibleProperty in MSTest.TestAdapter.targets. The diagnostic reports at each attribute application; includes resources (+ regenerated xlf), release tracking, and unit tests.
Copilot AI review requested due to automatic review settings July 14, 2026 15:37
@Evangelink
Evangelink force-pushed the dev/amauryleve/aot-il2026-assembly-fixture-provider branch from 359ff8b to b54f046 Compare July 14, 2026 15:37
@Evangelink Evangelink changed the title Suppress IL2026 in AssemblyFixtureProvider assembly discovery Skip [AssemblyFixtureProvider] under Native AOT + add MSTEST0072 analyzer Jul 14, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 21/21 changed files
  • Comments generated: 2
  • Review effort level: Medium

@github-actions

This comment has been minimized.

@Evangelink Evangelink added the state/needs-review Awaiting review from the team. label Jul 15, 2026
@Evangelink
Evangelink enabled auto-merge (squash) July 15, 2026 07:29
…ime warning

- MSTEST0072 now also inspects referenced assemblies' attributes, so the documented default usage (attribute on a referenced fixture library) is flagged with a no-location diagnostic on the consuming Native AOT project, not just self-applied attributes.

- Add a runtime warning when discovery is skipped because dynamic code is unsupported, covering Mono iOS AOT and Blazor WebAssembly AOT which the PublishAot-keyed analyzer cannot reach. The check is metadata-only (HasAssemblyFixtureProviderMarker) so it stays AOT-safe.

- Use explicit LINQ filtering (.Where/.Any) in the analyzer loops per code-quality feedback.

- Add unit tests for the referenced-assembly scenario (PublishAot true and false).
Copilot AI review requested due to automatic review settings July 15, 2026 07:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 21/21 changed files
  • Comments generated: 8
  • Review effort level: Medium

Comment thread src/Analyzers/MSTest.Analyzers/Helpers/WellKnownTypeNames.cs
Comment thread src/Analyzers/MSTest.Analyzers/Helpers/DiagnosticIds.cs
@github-actions

This comment has been minimized.

…tance test, encoding, LINQ

- Runtime warning now scans already-loaded assemblies (AppDomain.GetAssemblies, metadata-only) instead of only the test assembly, so a referenced provider library is no longer silent on Mono iOS / Blazor WASM AOT. Stays AOT-safe (no reference-graph walk).

- Add acceptance test AssemblyFixtureProviderNativeAotTests verifying that under PublishAot=true (IsDynamicCodeSupported=false, managed) the provider's AssemblyInitialize/AssemblyCleanup are skipped while the test still passes.

- Restore UTF-8 BOM on all touched .cs files per .editorconfig.

- Use explicit .Where(...) in the referenced-assembly loop.
Copilot AI review requested due to automatic review settings July 15, 2026 08:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 22/22 changed files
  • Comments generated: 2
  • Review effort level: Medium

@github-actions

This comment has been minimized.

- Isolate per-assembly failures in the runtime warning loop (try/catch around the metadata probe) so unresolvable custom-attribute metadata on one loaded assembly cannot abort discovery, matching the normal discovery path. Clarify in comments that the runtime probe is best-effort (already-loaded assemblies only) and that referenced-provider coverage is the analyzer's responsibility.

- Broaden MSTEST0072 to also trigger on RunAOTCompilation (Blazor WebAssembly AOT) in addition to PublishAot, giving build-time detection for another dynamic-code-disabled runtime. Added RunAOTCompilation as a CompilerVisibleProperty and a unit test.

- Replace the referenced-assembly foreach (whose loop variable was unused) with a single .Any(...) check that reports one no-location diagnostic, fixing the useless-assignment warning.
Copilot AI review requested due to automatic review settings July 15, 2026 08:35
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 22/22 changed files
  • Comments generated: 2
  • Review effort level: Medium

Comment thread src/Analyzers/MSTest.Analyzers/Resources.resx Outdated
MSTEST0072 now triggers on RunAOTCompilation (Blazor WebAssembly AOT) as well as PublishAot, so the diagnostic title/message/description no longer say only 'Native AOT'. Reword to 'ahead-of-time compilation (such as Native AOT or Blazor WebAssembly AOT)' and regenerate the XLF files for all locales.
Copilot AI review requested due to automatic review settings July 15, 2026 09:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 22/22 changed files
  • Comments generated: 2
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9941

GradeTestNotes
B (80–89) new AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzerTests.
WhenNotPublishAot_
AttributeOnReferencedAssembly_
NoDiagnostic
Strong negative assertion via RunAsync(); body is ~45 lines due to multi-project string literals — consider extracting shared setup into a helper to reduce per-test size.
B (80–89) new AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzerTests.
WhenPublishAotAndAttributeOnReferencedAssembly_
Diagnostic
Good use of WithNoLocation() for cross-assembly diagnostic; body is ~45 lines — extracting the multi-project scaffolding into a shared helper would improve readability.
A (90–100) new AssemblyFixtureProviderNativeAotTests.
AssemblyFixtureProvider_
WhenDynamicCodeUnsupported_
IsSkipped
Clear AAA; rich assertions: exit code, summary, and positive and negative output checks all verified.
A (90–100) new AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzerTests.
WhenNotPublishAot_
NoDiagnostic
Concise, focused negative test; canonical Roslyn markup assertion pattern. No issues found.
A (90–100) new AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzerTests.
WhenPublishAotAndAttributeIsUsed_
Diagnostic
Clean single-scenario test with precise location-aware diagnostic assertion. No issues found.
A (90–100) new AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzerTests.
WhenPublishAotAndAttributeNotUsed_
NoDiagnostic
Well-scoped negative test; verifies silence when the attribute is absent under AOT. No issues found.
A (90–100) new AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzerTests.
WhenPublishAotAndAttributeUsedMultipleTimes_
DiagnosticOnEach
Verifies per-usage diagnostic with two independent markers; correctly tests the each-occurrence contract. No issues found.
A (90–100) new AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzerTests.
WhenRunAOTCompilationAndAttributeIsUsed_
Diagnostic
Correctly tests the RunAOTCompilation property path (distinct from PublishAot); canonical markup assertion. No issues found.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 44.2 AIC · ⌖ 5.65 AIC · ⊞ 8.9K · [◷]( · )

@Evangelink
Evangelink disabled auto-merge July 15, 2026 10:47
@Evangelink
Evangelink enabled auto-merge (squash) July 15, 2026 10:47
@Evangelink
Evangelink merged commit 179aae0 into main Jul 15, 2026
60 checks passed
@Evangelink
Evangelink deleted the dev/amauryleve/aot-il2026-assembly-fixture-provider branch July 15, 2026 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-review Awaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants