Skip to content

Move libraries tests to xunit3#125019

Closed
agocke wants to merge 71 commits intodotnet:mainfrom
agocke:xunit3
Closed

Move libraries tests to xunit3#125019
agocke wants to merge 71 commits intodotnet:mainfrom
agocke:xunit3

Conversation

@agocke
Copy link
Copy Markdown
Member

@agocke agocke commented Mar 1, 2026

No description provided.

agocke and others added 21 commits January 16, 2026 15:52
Updated Dependencies:
Microsoft.CodeAnalysis, Microsoft.CodeAnalysis.Analyzers, Microsoft.CodeAnalysis.CSharp, Microsoft.Net.Compilers.Toolset (Version 5.5.0-2.26124.102 -> 5.5.0-2.26127.101)
Microsoft.CodeAnalysis.NetAnalyzers, Microsoft.DotNet.ApiCompat.Task, Microsoft.NET.Workload.Emscripten.Current.Manifest-11.0.100.Transport (Version 11.0.100-preview.3.26124.102 -> 11.0.100-preview.3.26127.101)
Microsoft.DotNet.Arcade.Sdk, Microsoft.DotNet.Build.Tasks.Archives, Microsoft.DotNet.Build.Tasks.Feed, Microsoft.DotNet.Build.Tasks.Installers, Microsoft.DotNet.Build.Tasks.Packaging, Microsoft.DotNet.Build.Tasks.TargetFramework, Microsoft.DotNet.Build.Tasks.Templating, Microsoft.DotNet.Build.Tasks.Workloads, Microsoft.DotNet.CodeAnalysis, Microsoft.DotNet.GenAPI, Microsoft.DotNet.GenFacades, Microsoft.DotNet.Helix.Sdk, Microsoft.DotNet.PackageTesting, Microsoft.DotNet.RemoteExecutor, Microsoft.DotNet.SharedFramework.Sdk, Microsoft.DotNet.XliffTasks, Microsoft.DotNet.XUnitExtensions (Version 11.0.0-beta.26124.102 -> 11.0.0-beta.26127.101)
Microsoft.DotNet.Cecil (Version 0.11.5-preview.26124.102 -> 0.11.5-preview.26127.101)
Microsoft.DotNet.XUnitAssert, Microsoft.DotNet.XUnitConsoleRunner (Version 2.9.3-beta.26124.102 -> 2.9.3-beta.26127.101)
Microsoft.NET.Sdk.IL, Microsoft.NETCore.App.Ref, Microsoft.NETCore.ILAsm, runtime.native.System.IO.Ports, System.Reflection.Metadata, System.Reflection.MetadataLoadContext, System.Text.Json (Version 11.0.0-preview.3.26124.102 -> 11.0.0-preview.3.26127.101)
NuGet.Frameworks, NuGet.Packaging, NuGet.ProjectModel, NuGet.Versioning (Version 7.5.0-rc.12502 -> 7.5.0-rc.12801)
System.CommandLine (Version 3.0.0-preview.3.26124.102 -> 3.0.0-preview.3.26127.101)
[[ commit created by automation ]]
The test infrastructure was still using the xunit v2 external console runner
(dotnet exec ... xunit.console.dll) after the switch to xunit v3. Tests now
produce native executables with an embedded in-process runner and should be
invoked directly.

Changes:
- RunScriptCommand runs ./AssemblyName (Unix) or AssemblyName.exe (Windows)
  instead of dotnet exec with the external xunit.console.dll
- Set DOTNET_ROOT to RUNTIME_PATH in RunnerTemplate.sh/.cmd so the test
  executable can locate the shared framework
- Update xunit argument syntax for v3: -noLogo, -noColor, -maxThreads,
  -reporter verbose, -trait- (replaces -notrait)
- Remove Microsoft.DotNet.XUnitConsoleRunner PackageReference for .NETCoreApp
- Skip CopyRunnerToOutputDirectory for .NETCoreApp (runner is embedded)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace all uses of SkipTestException (from Microsoft.DotNet.XUnitV3Extensions)
with SkipException.ForSkip() (from Xunit.Sdk) across 117 test files.

- 'throw new SkipTestException(msg)' → 'throw SkipException.ForSkip(msg)'
- 'catch (SkipTestException)' → 'catch (SkipException)'
- Added 'using Xunit.Sdk;' where needed
- Updated comments and nameof() references

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…pUnless

Where the pattern is a simple 'if (cond) { throw SkipException.ForSkip(msg); }',
convert to 'Assert.SkipWhen(cond, msg)' or 'Assert.SkipUnless(cond, msg)' to
match the xunit v3 idiom used in src/installer host tests.

147 conversions across 75 files. Complex cases (catch blocks, multi-statement
ifs, ternary expressions) retain throw SkipException.ForSkip().

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 1, 2026 01:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

agocke and others added 3 commits March 1, 2026 08:44
xunit v3 changed libraries tests from running via 'dotnet exec
xunit.console.dll' to running './AssemblyName' directly as an embedded
runner executable. When the test archive ZIP is extracted on Helix
machines, the execute permission is not preserved, causing all libraries
test work items to fail with exit code 126 (Permission denied).

Add chmod +x for the test executable in SetScriptCommands, matching the
same approach used by host tests in helixpublish.proj. The chmod runs
in the generated RunTests.sh before test execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 1, 2026 20:45
agocke and others added 2 commits March 22, 2026 21:12
Conflicts resolved:
- ProcessStartInfoTests.Windows.cs: Take upstream's simplified test
  (ShellExecute cases removed, xunit3 Mono skip no longer needed)
- System.Private.CoreLib.Shared.projitems: Take upstream's simplified
  PortableThreadPool.Unix.cs condition

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Conflicts:
#	eng/Version.Details.props
#	eng/Version.Details.xml
Copilot AI review requested due to automatic review settings March 23, 2026 22:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

agocke and others added 2 commits March 24, 2026 00:54
- Use 'dotnet exec' for .NETCoreApp test execution instead of running
  the app host directly, since UseAppHost=false means no native
  executable is produced. .NETFramework tests continue to run their
  embedded executable directly.

- Remove the chmod +x SetScriptCommand that was only needed for app
  host execution.

- Deploy xunit.v3.assert as a runtime-only dependency (ExcludeAssets=
  compile) for .NETCoreApp test projects. This is needed because
  netstandard2.0 support libraries (e.g. ModuleCore) compile against
  xunit.v3.assert (assembly name: xunit.v3.assert.dll), while
  .NETCoreApp test projects compile against Microsoft.DotNet.XUnitAssert
  (assembly name: xunit.assert.dll). Without the runtime deployment,
  the xunit.v3.assert.dll is missing and causes FileNotFoundException
  at test time.

- Simplify ModuleCore.csproj to unconditionally reference xunit.v3.assert
  with PrivateAssets=all, removing dead conditional code (netstandard2.0
  TargetFrameworkIdentifier is never .NETCoreApp).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…er/WASM test execution

- ILLink.Descriptor.xunit.xml: Use assembly name 'xunit.assert' instead
  of package name 'Microsoft.DotNet.XUnitAssert'
- default.rd.xml: Same fix for NativeAOT root descriptor
- WasmTestRunner.cs: Accept '-trait-' (xunit v3) alongside '-notrait' (v2)
- SingleFileTestRunner.cs: Same fix
- sendtohelix-browser.targets, sendtohelix-wasi.targets: Update
  -notrait to -trait- (xunit v3 trait exclusion syntax)
- StaticTestGenerator/Program.cs: Update -notrait to -trait-
- BionicRunOnDevice.sh: Update to run test DLL directly (no
  xunit.console.dll) and use -trait- syntax
- monodroid-nativeaot.cs: Update -notrait to -trait-
- Wasm.Build.Tests.csproj, Wasi.Build.Tests.csproj: Run test DLL
  directly instead of via xunit.console.dll
- helixpublish.proj: Use 'dotnet exec' for installer tests since
  UseAppHost=false means no native exe is generated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 24, 2026 04:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

agocke and others added 2 commits March 24, 2026 06:55
xunit v3's ConsoleRunner uses Assembly.GetEntryAssembly().Location to find
the test assembly file, but in NativeAOT Assembly.Location returns empty
string, causing 'assembly not found' error (exit code 3).

This adds a custom entry point (NativeAotTestEntryPoint.cs) that constructs
the XunitProjectAssembly directly using Environment.ProcessPath, bypassing
the file-based assembly lookup. The xunit auto-generated entry point is
disabled for NativeAOT tests via XunitAutoGeneratedEntryPoint=false.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e NativeAotTestEntryPoint

Replace the separate NativeAotTestEntryPoint.cs with an updated SingleFileTestRunner
that uses xunit v3 APIs (ProjectAssemblyRunner, XunitProjectAssembly). This maintains
the existing pattern of using SingleFileTestRunner for NativeAOT tests.

Key change: SingleFileTestRunner uses typeof(SingleFileTestRunner).Assembly and
Environment.ProcessPath, avoiding Assembly.GetEntryAssembly().Location which returns
empty string in NativeAOT.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 24, 2026 07:57
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

agocke and others added 2 commits March 24, 2026 08:03
XunitProject.RunnerReporter throws UnsetPropertyException when accessed
without being initialized. Set it to DefaultRunnerReporter explicitly
since we bypass ConsoleRunner which normally handles this.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In NativeAOT, Assembly.Location previously returned empty string, which
caused xunit v3's InProcessFrontController and TestAssemblyRunner to fail
with 'Argument was empty (Parameter AssemblyName)' since they derive the
assembly name from Path.GetFileNameWithoutExtension(Assembly.Location).

Return Environment.ProcessPath instead, which is correct since all
assemblies are compiled into the single native executable.

Also update SingleFileTestRunner to use testAssembly.Location directly
instead of Environment.ProcessPath as a workaround.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 24, 2026 09:31
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

agocke and others added 2 commits March 24, 2026 11:42
# Conflicts:
#	src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/Regex.Match.Tests.cs
#	src/native/managed/cdac/tests/DumpTests/Microsoft.Diagnostics.DataContractReader.DumpTests.csproj
#	src/native/managed/cdac/tests/DumpTests/TestConfiguration.cs
Copilot AI review requested due to automatic review settings March 29, 2026 07:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

Copilot AI added a commit that referenced this pull request Mar 29, 2026
Port of #125019

Co-authored-by: agocke <515774+agocke@users.noreply.github.com>
@agocke
Copy link
Copy Markdown
Member Author

agocke commented Mar 29, 2026

Closing in favor of #126277

@agocke agocke closed this Mar 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-Meta linkable-framework Issues associated with delivering a linker friendly framework

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants