Native MTP integration for MSTest — RFC 018 + Phases 1–5 (opt-in native path)#9706
Merged
Merged
Conversation
…bridge) Proposes a staged roadmap to plug MSTest directly into Microsoft.Testing.Platform as a native ITestFramework, replacing the Microsoft.Testing.Extensions.VSTestBridge indirection on the MTP code path. Documents the existing neutral seams (IUnitTestElementSink, ITestResultRecorder, IAdapterMessageLogger, ITestElementFilterProvider, DeploymentContext), a 6-phase plan, capability parity checklist, testing strategy, and risks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds RFC 018, a design/roadmap document proposing that MSTest plug directly into Microsoft.Testing.Platform (MTP) as a first-class ITestFramework, retiring the Microsoft.Testing.Extensions.VSTestBridge indirection on the MTP code path. It is a documentation-only artifact (status "Under discussion") with no production code changes, intended to let maintainers steer the approach before implementation. The RFC documents the current double-conversion architecture, the existing neutral seams the engine already uses, a 6-phase independently-shippable roadmap, a capability/feature-parity checklist, and the testing strategy, risks, and open questions.
Changes:
- Documents current vs. target MTP architecture, highlighting the wasteful
neutral model → VSTest TestCase/TestResult → TestNodedouble conversion and the resulting information loss (emptyAssemblyFullName/ReturnTypeFullName). - Lays out a 6-phase roadmap that keeps the bridge as default until the native path reaches parity, with a feature-parity checklist and dual-run testing strategy.
- Follows the established
docs/RFCs/convention; the bridge package is not removed (NUnit/Expecto/third-party adapters still use it) — only MSTest stops depending on it.
Show a summary per file
| File | Description |
|---|---|
| docs/RFCs/018-Native-MTP-Integration-For-MSTest.md | New RFC describing the native MTP integration design, phasing, parity checklist, testing strategy, risks, and open questions. All referenced types, seams, file paths, and the quoted in-code comment were verified as accurate against the codebase. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Medium
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
| # | Dimension | Verdict |
|---|---|---|
| 17 | Documentation Accuracy | 🟢 2 NIT |
| 21 | Scope & PR Discipline | 🟢 1 NIT |
✅ 20/22 dimensions N/A (no code changes). 2 dimensions reviewed — only minor nits found.
Summary: This is a well-structured, thoroughly researched RFC. The claims about existing neutral seams (IUnitTestElementSink, ITestResultRecorder, IAdapterMessageLogger, ITestElementFilterProvider) are verified against the codebase, the MSTestBridgedTestFramework information-loss quote is accurate (line 100–106 in that file), and the 6-phase roadmap with the capability parity checklist covers all bridge features I can identify. The design principle of "boundary replacement, not engine rewrite" is well-supported by the architecture diagrams.
- Path convention: use forward slashes for cross-platform consistency (line 84)
- Consider linking a tracking issue for the 6-phase roadmap
Adds the first, unwired building block for plugging MSTest directly into Microsoft.Testing.Platform instead of routing through the VSTest bridge: - MSTestTestNodeConverter: builds MTP TestNodes directly from MSTest's neutral models (UnitTestElement + framework TestResult), faithfully porting the mapping the bridge does today via ObjectModelConverters/TestResultExtensions/UnitTestElementExtensions (outcome, timing, std out/err, TRX categories/messages/exception/type-name, traits, file location, TestMethodIdentifierProperty). - MtpUnitTestElementSink / MtpTestResultRecorder: MTP-native IUnitTestElementSink / ITestResultRecorder that publish TestNodeUpdateMessage on the message bus. - MSTestTestNodeException: native message+stacktrace carrier (replaces the bridge's VSTestException on the MSTest path). - UnitTestElementExtensions.GetTestId: exposes the stable test id neutrally, without materializing a VSTest TestCase. The bridge remains the default; these seams are covered by 24 unit tests and are wired in a later phase. No behavior change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Evangelink
changed the title
This comment has been minimized.
This comment has been minimized.
When the MSTEST_EXPERIMENTAL_NATIVE_MTP opt-in is set, MSTest publishes discovery and result TestNodes directly via the Phase 1 native converter+seams, bypassing the bridge's ObjectModelConverters / FrameworkHandlerAdapter / TestCaseDiscoverySinkAdapter. The bridge remains the default; only context/filter/config/command-line parsing still flows through it. Changes: - VSTestBridge base: expose the session UID as an internal auto-property (bridge already has InternalsVisibleTo MSTest.TestAdapter; no public API change). - MSTestDiscoverer: add an IUnitTestElementSink overload and share a discovery core so the native sink is used directly instead of discoverySink.ToUnitTestElementSink(). - MSTestExecutor: add a Func<MSTestSettings,ITestResultRecorder> overload and share a run core so a native recorder is used instead of frameworkHandle.ToTestResultRecorder(); the framework handle is still used for message logging and apartment-state handling. - MSTestBridgedTestFramework: flag-gated native wiring. Validated: full build 0/0; MSTestAdapter.UnitTests 45/45; native-path acceptance OutputTests(6)+TrxReportTests(3)+Inconclusive/Ignore/DataSource(38) all green; default bridge-path OutputTests(6) unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Evangelink
changed the title
Contributor
🧪 Test quality grade — PR #970624 new test methods in
This advisory comment was generated automatically. Grades are heuristic
|
Evangelink
commented
Jul 7, 2026
Evangelink
commented
Jul 7, 2026
0101
approved these changes
Jul 7, 2026
- Make the neutral discovery/result seams asynchronous end-to-end (per review): ITestResultRecorder (RecordStartAsync/RecordEmptyResultAsync/RecordResultAsync) and IUnitTestElementSink (SendTestElementAsync). Thread async through UnitTestDiscoverer (DiscoverTestsAsync/DiscoverTestsInSourceAsync/SendTestCasesAsync) and TestExecutionManager.SendTestResultsAsync. The MTP implementations now await the message-bus publish instead of blocking with GetAwaiter().GetResult(); the VSTest implementations return completed tasks (sync fast-path preserves STA behavior). - Add the UTF-8 BOM required by .editorconfig to the five new .cs files. - Use forward slashes for the seam path in RFC 018. Validated: full build 0/0; unit tests MSTestAdapter.UnitTests 45/45 and MSTestAdapter.PlatformServices.UnitTests 898/898; acceptance default-path threading/output/trx 41 green; native-path threading/output/trx/outcome/data-driven 75 green. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
0101
approved these changes
Jul 7, 2026
Evangelink
enabled auto-merge (squash)
July 8, 2026 10:16
github-actions Bot
added a commit
that referenced
this pull request
Jul 8, 2026
Apply two project-convention improvements to files added in recent PRs: - MSTestTestNodeConverter.cs (added in #9706): replace Substring calls with range operators per csharp_style_prefer_range_operator = true - TestMethodRunner.DataSource.cs (added in #9729): replace == null with is null per the project's 'always use is null / is not null' rule No behavioral change; builds verified to succeed with 0 errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Jul 8, 2026
Evangelink
changed the title
This was referenced Jul 8, 2026
Merged
github-actions Bot
added a commit
that referenced
this pull request
Jul 9, 2026
- AzureFoundry: add DefaultAzureCredential/managed identity auth details and required environment variables (PR #9707) - MSTestTestFramework: new entry documenting the native MTP ITestFramework for MSTest introduced by RFC 018 (PRs #9706, #9743, #9748, #9755) - VSTestBridge: note MSTest no longer depends on it on the MTP path as of MSTest 4.3 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Evangelink
pushed a commit
that referenced
this pull request
Jul 9, 2026
- AzureFoundry: add DefaultAzureCredential/managed identity auth details and required environment variables (PR #9707) - MSTestTestFramework: new entry documenting the native MTP ITestFramework for MSTest introduced by RFC 018 (PRs #9706, #9743, #9748, #9755) - VSTestBridge: note MSTest no longer depends on it on the MTP path as of MSTest 4.3 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Works toward removing the
Microsoft.Testing.Extensions.VSTestBridgedependency from MSTest's Microsoft.Testing.Platform (MTP) code path, so MSTest plugs into MTP natively. Delivered as a reviewable, phased series — all behind the opt-inMSTEST_EXPERIMENTAL_NATIVE_MTPswitch, so the shipping default (the bridge) is unchanged.docs/RFCs/018-...md).TestNodeproduction seam (MSTestTestNodeConverter,MtpUnitTestElementSink,MtpTestResultRecorder).MSTestTestFrameworkthat handles the MTP request directly: native filtering, runsettings/config, message logging, and session lifecycle. No VSTest bridge object-model adapters on the native request path.Why
On the MTP path, MSTest round-trips through the VSTest object model (context/filter/runsettings/handle/sink adapters) even though its engine already runs on neutral models. Going native removes that plumbing.
Phases 3–5 in the latest commit
A native
MSTestTestFramework : ITestFramework, IDataProducerreplaces the bridge framework on the flag path and reuses MSTest's existingMSTestDiscoverer/MSTestExecutorengine. New (all#if !WINDOWS_UWP):MSTestFilterContext(MSTestRunContext/MSTestDiscoveryContext) — nativeIRunContext/IDiscoveryContextthat builds the VSTestITestCaseFilterExpressionfrom the MTPITestExecutionFilter+--filter+ runsettings<TestCaseFilter>(reusingMicrosoft.TestPlatform.Filter.Source, now referenced directly). MSTest's existingTestMethodFiltermatching is unchanged.MSTestRunSettings— nativeIRunSettings: reads runsettings, patches MTP defaults (DesignMode/ResultsDirectory/--test-parameter), and warns on unsupported entries (reusing the bridge's already-localizedExtensionResources).MSTestFrameworkHandle— anIFrameworkHandlethat only forwards messages toIOutputDevice(results flow through the native recorder).AddMSTestbranches the framework factory on the flag; the bridge's shared command-line/config/env-var registrations are kept for now (retired with the dependency in the final step).Validation
.\build.cmd— 0 warnings, 0 errors across all TFMs.MSTestAdapter.UnitTests) + 898 (MSTestAdapter.PlatformServices.UnitTests), all green.FilterTests,RunsettingsTests(incl. localization),OutputTests,TrxReportTests,ThreadingTests(STA),ServerModeTests,DeploymentItem,DataSourceTests,Inconclusive/Ignore,TestRunParameters,TimeoutTests.FilterTests+RunsettingsTests(45) — unchanged.Remaining — Phase 6 (separate, gated)
Flip the default to native and drop the
VSTestBridgedependency from MSTest. This is gated on:--filter/--settings/--test-parameteroption providers and the runsettings config/env-var providers), so the package reference can be removed.The bridge package itself is not removed — NUnit/Expecto/3rd-party adapters still use it. Only MSTest stops depending on it.