Abstract VSTest execution input in PlatformServices (Phase 4 of platform-agnostic effort)#9572
Merged
Evangelink merged 4 commits intoJul 3, 2026
Conversation
Make the MSTestAdapter.PlatformServices execution engine operate on the neutral UnitTestElement model end-to-end instead of the VSTest TestCase object model. - Convert TestCase -> UnitTestElement at the adapter boundary (MSTestExecutor and the integration/unit test harnesses), carrying host execution-context (TCM) properties in a neutral string-keyed ExecutionContextProperties bag and the originating TestCase as an opaque, non-serialized HostRecordingHandle. - TestExecutionManager (RunTestsAsync tests-overload, ExecuteTestsAsync, ExecuteTestsInSourceAsync, ExecuteTestsWithTestRunnerAsync, MatchTestFilter, SendTestResults) and ITestResultRecorder now flow UnitTestElement. - The result-recording bridge records against the opaque host handle when present (byte-for-byte fidelity incl. TCM/host properties for normal results) and falls back to element.ToTestCase() for internally discovered tests. - Fix TestMethod.CloneWithUpdatedSource to assign the source on the clone. - Deployment stays TestCase-based; the engine materializes test cases only at that single boundary call (deferred to a later phase). No behavior change: executed set, order, parallelization grouping, TCM/TRX properties and recorded results remain identical. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Reuse a single materialized VSTest test case per internally-discovered test (UnitTestElement.GetOrCreateHostTestCase) so deployment, test-start and every reported result share one instance, restoring the historical one-test-case-per- discovered-test invariant and avoiding repeated Id hashing on the sources path. - Add UnitTestElement tests for WithUpdatedSource (clone gets the new source, the original is left unmutated) and GetOrCreateHostTestCase (host-handle reuse and lazy materialization caching). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Per review direction, keep the latent TestMethod/UnitTestElement.CloneWithUpdatedSource bug (assigns source on 'this' instead of the clone) untouched for the existing ToTestCase/test-case-filter bridge callers, and route the neutral execution engine's source resolution through a new, correct CloneWithSource that updates the returned clone only. The bug is tracked by #9573 and referenced in comments next to both coexisting clone methods. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Per review decision, keep the Phase 5 ITestResultRecorder (and its bridge) TestCase-based and unchanged. The execution engine now threads the test's host TestCase as an OPAQUE handle (obtained via UnitTestElement.GetOrCreateHostTestCase) into the recorder and the deployment service, reading nothing VSTest-specific off it. This keeps recorded results byte-for-byte identical to baseline (the unchanged recorder receives the exact same TestCase), while filtering, parallelization grouping, source resolution and TCM->TestContext remain neutral. Neutralizing the recorder and deployment (removing the opaque TestCase handle) is deferred to a later boundary-neutralization phase. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Evangelink
marked this pull request as ready for review
July 3, 2026 11:41
Evangelink
merged commit Jul 3, 2026
4f02b6e
into
dev/amauryleve/vstest-decoupling-base
20 of 22 checks passed
Evangelink
deleted the
dev/amauryleve/abstract-vstest-execution-input-platform
branch
July 3, 2026 11:41
This was referenced Jul 3, 2026
Evangelink
added a commit
that referenced
this pull request
Jul 5, 2026
…orm-agnostic effort) (#9572) Co-authored-by: Amaury Leveque <amauryleve@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Evangelink
added a commit
that referenced
this pull request
Jul 5, 2026
…ervices Squashed rebase of the vstest-decoupling PlatformServices stack onto main. Phases 1, 2 and 5 already landed on main via #9548/#9567/#9550; this commit carries the remaining net-new work: - Phase 3 (#9566): abstract the VSTest discovery sink (IUnitTestElementSink). - Phase 4 (#9572): abstract VSTest execution input. - Phase 6a (#9576): neutralize deployment input (DeploymentContext). - Phase 6b (#9579): neutralize test result recording (ITestResultRecorder). - Phase 6c (#9585): neutralize test message logging. - Phase 6c2: neutralize run-settings input in the host layer (settingsXml). - Phase 6d-1: move test-case filter parsing to the adapter boundary (ITestElementFilterProvider / TestElementFilterProvider). - Phase 6d-2: remove IRunContext/IDiscoveryContext from PlatformServices. - Phase 6e-1 (#9622): remove IFrameworkHandle from the execution engine. - Phase 6e-2 (#9623): relocate VSTest logger/sink bridges to the adapter. - Phase 6e-3a (#9624): neutralize the trait type on UnitTestElement. Result: MSTestAdapter.PlatformServices no longer references the VSTest run/discovery context or result object model; those types live only at the MSTest.TestAdapter boundary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Evangelink
added a commit
that referenced
this pull request
Jul 5, 2026
…ervices Squashed rebase of the vstest-decoupling PlatformServices stack onto main. Phases 1, 2 and 5 already landed on main via #9548/#9567/#9550; this commit carries the remaining net-new work: - Phase 3 (#9566): abstract the VSTest discovery sink (IUnitTestElementSink). - Phase 4 (#9572): abstract VSTest execution input. - Phase 6a (#9576): neutralize deployment input (DeploymentContext). - Phase 6b (#9579): neutralize test result recording (ITestResultRecorder). - Phase 6c (#9585): neutralize test message logging. - Phase 6c2: neutralize run-settings input in the host layer (settingsXml). - Phase 6d-1: move test-case filter parsing to the adapter boundary (ITestElementFilterProvider / TestElementFilterProvider). - Phase 6d-2: remove IRunContext/IDiscoveryContext from PlatformServices. - Phase 6e-1 (#9622): remove IFrameworkHandle from the execution engine. - Phase 6e-2 (#9623): relocate VSTest logger/sink bridges to the adapter. - Phase 6e-3a (#9624): neutralize the trait type on UnitTestElement. Result: MSTestAdapter.PlatformServices no longer references the VSTest run/discovery context or result object model; those types live only at the MSTest.TestAdapter boundary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Part of the multi-PR effort to make
src/Adapter/MSTestAdapter.PlatformServicesplatform-agnostic by removing its dependency on the VSTest object model (Microsoft.TestPlatform.ObjectModel). VSTest coupling moves up into the adapter layer (src/Adapter/MSTest.TestAdapter); the platform-services engine stops reading VSTest-specific data off the test.This is Phase 4 (execution input): it makes the execution engine (
TestExecutionManager+ partials) operate on the neutralUnitTestElementmodel for everything that reads test metadata.What
TestCase→UnitTestElementnow happens at the adapter boundary (MSTestExecutor, and the unit/integration test harnesses). Each element carries:ExecutionContextPropertiesbag holding the host execution-context (a.k.a. TCM) properties.TcmTestPropertiesProvider.GetTcmPropertiesnow returnsIReadOnlyDictionary<string, object?>keyed by property id — lossless becauseGetTestContextPropertiesonly ever consumedTestProperty.Id.[NonSerialized]HostRecordingHandleholding the originating VSTestTestCase.RunTestsAsync(tests overload),ExecuteTestsAsync,ExecuteTestsInSourceAsync,ExecuteTestsWithTestRunnerAsync, andMatchTestFilterno longer read VSTest test-case properties. Filtering, parallelization grouping (element.DoNotParallelize/element.TestMethod.FullClassName), ordering (managed type/method gated byHasManagedMethodAndTypeProperties, matching the oldGetManagedType()/GetManagedMethod()semantics), source resolution (UnitTestElement.WithUpdatedSource), and TCM→TestContext(the neutral bag) all run off the neutral model.TestCaseDiscoverySinknow collectsUnitTestElementdirectly (noTestCaseround-trip).VSTest
TestCaseremaining — as an opaque handle onlyThe engine still hands the test's
TestCaseto two boundary services — the result recorder (ITestResultRecorder, unchanged from Phase 5) and the deployment service (ITestDeployment,TestCase-based) — but treats it as an opaque handle: it obtains it viaUnitTestElement.GetOrCreateHostTestCase()(the host's original test case when present, else a single cached materialized one) and reads nothing VSTest-specific off it. This deliberately keeps recorded results byte-for-byte identical to baseline: the unchanged recorder receives the exact sameTestCasethe engine passed before this change, so TRX/TCM association and any host/data-collector-injected properties are preserved. This is not "100%TestCase-free" — see remaining work.No behavior change
Pure refactor: the executed test set, order, parallelization grouping, TCM/TRX properties, and recorded results (fields, Id, outcome, messages, attachments, timings) are preserved for both the
RunTestsAsync(tests)andRunTestsAsync(sources)paths. The filterIdis filename-based (source-path-independent), so source resolution does not affect the filtered set.Source resolution and a tracked latent bug
Correct element-based source resolution is required for the deployment path (issue #6713 — the executed assembly must load from the deployment directory).
TestMethod.CloneWithUpdatedSourcehas a latent bug: it assignsAssemblyName/MethodInfoonthisinstead of the returned clone, tolerated only by the current call order. Rather than change that shared primitive here, this PR leaves the buggy method untouched for the existingToTestCase/test-case-filter bridge callers and adds a new, correctCloneWithSourcethat the neutral engine uses viaWithUpdatedSource. The bug is tracked by #9573 and referenced in comments next to both coexisting clone methods. Correct deployment-directory loading is verified byRunTestsForTestShouldLoadSourceFromDeploymentDirectoryIfDeployed.Remaining work (later phases)
Neutralizing the two boundary services — so the engine no longer holds a
TestCasehandle at all — is deferred and is the prerequisite for Phase 7 (cutting theMicrosoft.TestPlatform.ObjectModeldependency from PlatformServices):element → TestCasereconstruction, incl. the host-handle / TCM / data-collector fidelity, out of the engine) — a dedicated recorder piece.ITestDeployment.Deploy, which also flowsIFrameworkHandle) — folds into the deployment/contexts phase (Phase 6).PlatformServicesfor now:TestResultRecorderExtensions,UnitTestElementSinkExtensions,TcmTestPropertiesProvider,TestCaseExtensions/UnitTestElement.ToTestCase.thisinstead of the returned clone #9573 is addressed.Verification
build.cmd -c Debuggreen across all TFMs (net462, net8.0, net9.0, UWP, WinUI).MSTestAdapter.PlatformServices.UnitTests(897) andMSTestAdapter.UnitTests(21) green; the deployment-directory test passes explicitly. Recorded-result fidelity is covered byMSTestExecutorTests(assertsRecordStartagainst the original hostTestCase) and the integration harness's recorded-result assertions — the recorder is unchanged and receives the sameTestCaseas baseline.MSTest.IntegrationTests(discover→run harness) green — the only failing test seen wasOutputIsNotMixedWhenTestsRunInParallel, a known[CICondition(ConditionMode.Exclude)]timing-flaky parallelism assertion unrelated to this change.Stacked PR
Base =
dev/amauryleve/vstest-decoupling-base(the accumulation base that already contains Phase 2 #9555 and Phase 3 #9566). Phase 1 + Phase 5 are already onmain. This should be reviewed/merged after the earlier phases reachmain.