Add testconfig.json JSON schema for SchemaStore publication

[Evangelink]

What

Adds a draft-07 JSON Schema for testconfig.json, the configuration file consumed by Microsoft.Testing.Platform (MTP) and MSTest, plus a maintainer doc.

Implements Task 1 of #9381 (publish the MSTest testconfig.json schema to SchemaStore).

Files

• docs/testconfig.schema.json — the schema. Covers the full mstest:* surface (timeout, execution, parallelism, output, deployment, assemblyResolution, orderTestsByNameInClass), the platformOptions:* host options, and the environmentVariables section. Enums, ranges and per-key descriptions are derived from the actual parsers (MSTestSettings.Configuration.cs, MSTestAdapterSettings.cs, RunConfigurationSettings.cs, PlatformConfigurationConstants.cs).
• docs/testconfig.schema.md — how to reference the schema, the SchemaStore submission steps, how versioning works (rolling main URL; no in-file version), and a "keep in sync" pointer to the parser source-of-truth files.

Validation

The schema was validated against the canonical testconfig.json example used in the repo's acceptance tests, and confirmed to reject bad enums, sub-minimum timeouts, unknown keys, and missing required path in assemblyResolution.

SchemaStore registration

The companion SchemaStore catalog PR is open: SchemaStore/schemastore#5842. It is a self-hosted/external registration pointing at https://raw.githubusercontent.com/microsoft/testfx/main/docs/testconfig.schema.json, so it only goes green once this PR merges to main and the URL resolves.

Follow-ups

Tasks 2–5 from #9381 are tracked separately: #9402, #9401, #9403, #9404.

[Copilot]

Pull request overview

Adds a draft-07 JSON Schema for testconfig.json (for MSTest + Microsoft.Testing.Platform) plus maintainer documentation intended to support SchemaStore publication and IDE completion/validation.

Changes:

• Introduces docs/testconfig.schema.json defining the schema for mstest, platformOptions, and environmentVariables.
• Adds docs/testconfig.schema.md documenting how to reference the schema and how to publish/register it in SchemaStore.

Show a summary per file

File	Description
docs/testconfig.schema.json	Adds the draft-07 JSON schema for testconfig.json keys and value constraints.
docs/testconfig.schema.md	Documents schema usage, SchemaStore registration, and versioning/sync guidance.

Copilot's findings

• Files reviewed: 2/2 changed files
• Comments generated: 2

[Evangelink]

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review summary

All enum values cross-checked against the actual parser code — parallelism.scope, launchDebuggerOnAssertionFailure, and executionApartmentState are all correct. The platformOptions key paths all match PlatformConfigurationConstants. The overall schema structure, $schema dialect, $id URL, and the .md documentation are solid.

One factual correctness issue needs to be fixed before merge:

🔴 MAJOR — collectSourceInformation not wired up in JSON config

mstest.execution.collectSourceInformation is documented as a valid testconfig.json key, but no JSON configuration parser in the codebase reads mstest:execution:collectSourceInformation. A grep across the entire repo shows every occurrence of CollectSourceInformation lives in XML runsettings paths or XML-path code comments. The three JSON parsers invoked for testconfig.json — MSTestSettings.SetSettingsFromConfig, RunConfigurationSettings.SetRunConfigurationSettingsFromConfig, and MSTestAdapterSettings.ToSettings(IConfiguration) — do not call any ParseBooleanSetting/TryParse on that key. Users who set this in their testconfig.json will get a silent no-op.

Fix: remove the property from the schema (or clearly document it is .runsettings-only) until the JSON-path implementation is in place. Inline comment is left on line 125.

🟡 NIT — includeSubDirectories type union

"type": ["boolean", "string"] leaks how IConfiguration works internally. JSON authors should write true/false; "type": "boolean" is the correct schema type. Inline comment is left on line 193.

[Evangelink]

MAJOR — collectSourceInformation is not implemented in the JSON config path

collectSourceInformation under mstest.execution is documented here as a valid key, but none of the JSON configuration parsers actually read it:

• MSTestSettings.SetSettingsFromConfig (the 18-setting switch in MSTestSettings.Configuration.cs) does not call ParseBooleanSetting for execution:collectSourceInformation.
• RunConfigurationSettings.SetRunConfigurationSettingsFromConfig only reads mstest:execution:executionApartmentState.
• MSTestAdapterSettings.ToSettings(IConfiguration) only reads the deployment:* and assemblyResolution keys.

A grep of the full codebase confirms all occurrences of CollectSourceInformation are in XML runsettings paths or XML path comments (part of <RunConfiguration>, not <MSTest>), never in a configuration["mstest:execution:collectSourceInformation"] lookup.

A user who adds "collectSourceInformation": false to their testconfig.json will get no effect — the setting will be silently ignored.

Recommendation: Remove this property from the schema until it is wired up in the JSON config path, or add an "x-not-yet-supported": true / note in the description that this key is only honoured when using .runsettings.

[Evangelink]

NIT — ["boolean", "string"] union leaks an IConfiguration implementation detail

IConfiguration always surfaces all values as strings internally, so from the configuration layer's perspective true and "true" are indistinguishable. However, from a JSON author's perspective the correct way to express this is a plain JSON boolean:

{ "path": "C:\\foo", "includeSubDirectories": true }

Exposing "type": ["boolean", "string"] implicitly tells tooling that "includeSubDirectories": "true" is also a valid document, which makes schema-driven editors accept "yes" or "1" without complaint (those are strings that would silently fail bool.TryParse).

Recommendation: Use "type": "boolean" only.