From 5c57d2f99130b3e1b6d63d25b1ff975fba64e296 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Mon, 1 Jun 2026 16:38:55 +0200 Subject: [PATCH 1/4] Audit MTP CLI options and environment variables Inventoried every CommandLineOptionsProvider and EnvironmentVariableConstants entry in microsoft/testfx and reconciled the docs. - cli-options: replace removed --debug-wait-attach with --debug (microsoft/testfx#6516); add --filter-uid, --treenode-filter, --no-banner; clarify --list-tests text|json argument. - config: rename diagnostic env vars to current names (TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX, TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE) with back-compat notes; add TESTINGPLATFORM_NOBANNER, DOTNET_NOLOGO, TESTINGPLATFORM_WAIT_ATTACH_DEBUGGER, TESTINGPLATFORM_LAUNCH_ATTACH_DEBUGGER. - troubleshooting: sync diagnostic env vars table with renames. - crash-hang-dumps: add --crash-report, --crash-report-if-supported, --crash-sequence, --hangdump-type-if-supported; expand --hangdump-type (TFM-specific values + None), --hangdump-timeout formats and --hangdump-filename placeholders. - terminal-output: add --ansi, --show-stdout, --show-stderr. - test-reports: add all Azure DevOps options (--report-azdo-flaky-history, --report-azdo-demote-known-flaky, --report-azdo-quarantine-file, --report-azdo-upload-artifacts plus -include/-exclude/-name, --publish-azdo-test-results, --publish-azdo-run-name). - retry: add --retry-failed-tests-delay. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../microsoft-testing-platform-cli-options.md | 23 +++++++++++-- .../microsoft-testing-platform-config.md | 33 ++++++++++++++++--- ...osoft-testing-platform-crash-hang-dumps.md | 12 ++++--- .../microsoft-testing-platform-retry.md | 7 ++-- ...rosoft-testing-platform-terminal-output.md | 5 ++- ...microsoft-testing-platform-test-reports.md | 15 +++++++-- ...rosoft-testing-platform-troubleshooting.md | 6 ++-- 7 files changed, 80 insertions(+), 21 deletions(-) diff --git a/docs/core/testing/microsoft-testing-platform-cli-options.md b/docs/core/testing/microsoft-testing-platform-cli-options.md index 96b507d709a84..a1cc265dd5bb9 100644 --- a/docs/core/testing/microsoft-testing-platform-cli-options.md +++ b/docs/core/testing/microsoft-testing-platform-cli-options.md @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) CLI options reference description: Find platform and extension command-line options for MTP in one place. author: Evangelink ms.author: amauryleve -ms.date: 02/24/2026 +ms.date: 06/01/2026 ai-usage: ai-assisted --- @@ -53,6 +53,13 @@ This article gives a central entry point for MTP command-line options. Specifies a [*testconfig.json*](microsoft-testing-platform-config.md) file. +- **`--debug`** + + Pauses test execution at startup so you can attach a debugger to the test process. Equivalent to setting the `TESTINGPLATFORM_WAIT_ATTACH_DEBUGGER` [environment variable](./microsoft-testing-platform-config.md#testingplatform_wait_attach_debugger-environment-variable) to `1`. Not supported on browser platforms. + + > [!NOTE] + > Before MTP 2.0, this option was named `--debug-wait-attach`. The old name was removed; use `--debug` instead. + - **`--diagnostic`** Enables the diagnostic logging. The default log level is `Trace`. The file is written in the output directory with the following name format, `log_[MMddHHssfff].diag`. @@ -77,6 +84,10 @@ This article gives a central entry point for MTP command-line options. Exit the test process if dependent process exits. PID must be provided. +- **`--filter-uid`** + + Filters the tests to run by their test node UIDs. Accepts one or more UIDs. + - **`--help`** Prints out a description of how to use the command. @@ -98,7 +109,7 @@ This article gives a central entry point for MTP command-line options. - **`--list-tests`** - List available tests. Tests won't be executed. + Lists the available tests without executing them. Optionally takes an argument that controls the output format: `text` (default, human-readable) or `json`. - **`--maximum-failed-tests`** @@ -111,6 +122,10 @@ This article gives a central entry point for MTP command-line options. Specifies the minimum number of tests that are expected to run. By default, at least one test is expected to run. +- **`--no-banner`** + + Disables the startup banner, the copyright message, and the telemetry banner. The same effect can be achieved through the `TESTINGPLATFORM_NOBANNER` or `DOTNET_NOLOGO` [environment variables](./microsoft-testing-platform-config.md#environment-variables). + - **`--results-directory`** The directory where the test results are going to be placed. If the specified directory doesn't exist, it's created. The default is `TestResults` in the directory that contains the test application. @@ -119,6 +134,10 @@ This article gives a central entry point for MTP command-line options. A global test execution timeout. Takes one argument as string in the format `[h|m|s]` where `` is float. +- **`--treenode-filter`** + + Filters the tests to run by using a tree filter expression. Tree filters offer richer matching than `--filter` for advanced scenarios. + ## Extension options by scenario Use the following table to find extension options quickly. diff --git a/docs/core/testing/microsoft-testing-platform-config.md b/docs/core/testing/microsoft-testing-platform-config.md index ffb4e3cdcf3c2..c9d458324ec6c 100644 --- a/docs/core/testing/microsoft-testing-platform-config.md +++ b/docs/core/testing/microsoft-testing-platform-config.md @@ -3,7 +3,8 @@ title: Microsoft.Testing.Platform (MTP) config options description: Learn how to configure MTP using testconfig.json configuration settings and environment variables. author: Evangelink ms.author: amauryleve -ms.date: 05/13/2026 +ms.date: 06/01/2026 +ai-usage: ai-assisted --- # Microsoft.Testing.Platform (MTP) configuration settings @@ -163,18 +164,40 @@ Defines the verbosity level when diagnostics are enabled. The available values a The output directory of the diagnostic logging. If not specified, the file is generated in the default *TestResults* directory. -### `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` environment variable +### `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` environment variable -The prefix for the log file name. Defaults to `"log_"`. +The prefix for the log file name. Defaults to `"log_"`. Matches the `--diagnostic-file-prefix` command-line option. -### `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` environment variable +> [!NOTE] +> The legacy `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` takes precedence. + +### `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` environment variable + +Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. Matches the `--diagnostic-synchronous-write` command-line option. -Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. +> [!NOTE] +> The legacy `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` takes precedence. ### `TESTINGPLATFORM_EXITCODE_IGNORE` environment variable A semicolon-separated list of exit codes to ignore. When an exit code is ignored, the process returns `0` instead. For example, `TESTINGPLATFORM_EXITCODE_IGNORE=2;8` ignores test failures and no-tests-ran scenarios. +### `TESTINGPLATFORM_NOBANNER` environment variable + +When set to `1` or `true`, suppresses the startup banner, the copyright message, and the telemetry banner. Equivalent to the `--no-banner` command-line option. The `DOTNET_NOLOGO` environment variable has the same effect. + +### `DOTNET_NOLOGO` environment variable + +When set to `1` or `true`, suppresses the startup banner, the copyright message, and the telemetry banner. This is the standard .NET CLI environment variable and is honored by MTP. See also `TESTINGPLATFORM_NOBANNER`. + +### `TESTINGPLATFORM_WAIT_ATTACH_DEBUGGER` environment variable + +When set to `1`, the test process pauses at startup and waits for a debugger to attach before proceeding. Equivalent to the `--debug` command-line option. Not supported on browser platforms. + +### `TESTINGPLATFORM_LAUNCH_ATTACH_DEBUGGER` environment variable + +When set to `1`, the test process calls `Debugger.Launch()` at startup, which prompts the system to launch a just-in-time debugger and attach it to the process. Use this variable to debug startup-time issues (for example, server-mode handshake) that occur before you can manually attach. On non-Windows platforms, the behavior depends on the configured JIT debugger. + > [!NOTE] > Diagnostic-related environment variables take precedence over their corresponding `--diagnostic-*` command-line arguments. diff --git a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md index b7e77fb258e13..eb3bcb4decf63 100644 --- a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md +++ b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) crash and hang dumps description: Learn about the MTP extensions for collecting crash dumps and hang dumps during test execution. author: evangelink ms.author: amauryleve -ms.date: 02/25/2026 +ms.date: 06/01/2026 ai-usage: ai-assisted --- @@ -32,6 +32,9 @@ builder.TestHostControllers.AddCrashDumpProvider(); | `--crashdump` | Generates a dump file when the test host process crashes. Supported in .NET 6.0+. | | `⁠-⁠-⁠crashdump-⁠filename` | Specifies the file name of the dump. | | `--crashdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Triage`, `Full`. Defaults as `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | +| `--crash-report` | [Linux/macOS only] Generates a JSON crash report when the test process crashes. Combine with `--crashdump` to also generate a dump file. Requires .NET 7+ when used alone; .NET 6+ when combined with `--crashdump`. Not supported on Windows because the .NET runtime ignores the underlying `DOTNET_EnableCrashReport` and `DOTNET_EnableCrashReportOnly` environment variables on Windows ([dotnet/runtime#80191](https://github.com/dotnet/runtime/issues/80191)). | +| `--crash-report-if-supported` | Same as `--crash-report` but silently ignored (with an informational message) on platforms where crash report generation isn't supported. Use this option to keep the same command line across CI matrices that include Windows. Mutually exclusive with `--crash-report`. | +| `--crash-sequence` | Controls whether a sequence file listing the tests that started and ended during the test session is generated alongside the crash dump or crash report. The file makes it possible to identify the tests that were running at the time of the crash without inspecting the dump. Valid values are `on` (default; also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). | > [!CAUTION] > The extension isn't compatible with .NET Framework and will be silently ignored. For .NET Framework support, you enable the postmortem debugging with Sysinternals ProcDump. For more information, see [Enabling Postmortem Debugging: Window Sysinternals ProcDump](/windows-hardware/drivers/debugger/enabling-postmortem-debugging#window-sysinternals-procdump). The postmortem debugging solution will also collect process crash information for .NET so you can avoid the use of the extension if you're targeting both .NET and .NET Framework test applications. @@ -52,6 +55,7 @@ builder.TestHostControllers.AddHangDumpProvider(); | Option | Description | |--|--| | `--hangdump` | Generates a dump file in case the test host process hangs. | -| `-⁠-hangdump-filename` | Specifies the file name of the dump. | -| `--hangdump-timeout` | Specifies the duration of no activity after which the dump is generated. The timeout value is specified in one of the following formats:
`1.5h`, `1.5hour`, `1.5hours`
`90m`, `90min`, `90minute`, `90minutes`
`5400s`, `5400sec`, `5400second`, `5400seconds`. Defaults to `30m` (30 minutes). | -| `--hangdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Triage`, `Full`. Defaults as `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | +| `-⁠-hangdump-filename` | Specifies the file name of the dump. Supports the following placeholders: `{pname}` (test application name), `{pid}` (process ID), `{asm}` (entry assembly name), `{tfm}` (target framework moniker), `{time}` (timestamp). The legacy `%p` token (process ID) is also supported for backward compatibility. | +| `--hangdump-timeout` | Specifies the duration of no activity after which the dump is generated. The timeout value is specified in one of the following formats:
`1.5h`, `1.5hour`, `1.5hours`
`90m`, `90min`, `90minute`, `90minutes`
`5400s`, `5400sec`, `5400second`, `5400seconds`
`500ms`, `500mil`, `500millisecond`, `500milliseconds`
`1d`, `1day`, `1days`. A bare number (with no suffix) is interpreted as milliseconds. Defaults to `30m` (30 minutes). | +| `--hangdump-type` | Specifies the type of the dump. Valid values on .NET (Core) are `Mini`, `Heap`, `Full`, `Triage`, `None`. On .NET Framework, the valid values are `Mini`, `Heap`, `Full`, `None` (`Triage` is unavailable). Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | +| `--hangdump-type-if-supported` | Same as `--hangdump-type` but silently falls back (with an informational message) to the closest supported dump type when the requested type isn't available on the current runtime. For example, `Triage` is only supported on .NET (Core) and falls back to `Mini` on .NET Framework. Use this option to keep the same command line across CI matrices that mix .NET (Core) and .NET Framework. Valid values are `Mini`, `Heap`, `Full`, `Triage`, `None`. Mutually exclusive with `--hangdump-type`. | diff --git a/docs/core/testing/microsoft-testing-platform-retry.md b/docs/core/testing/microsoft-testing-platform-retry.md index 63bc25c9d4b08..81282f6e91403 100644 --- a/docs/core/testing/microsoft-testing-platform-retry.md +++ b/docs/core/testing/microsoft-testing-platform-retry.md @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) retry description: Learn about retrying failed tests with MTP. author: evangelink ms.author: amauryleve -ms.date: 02/25/2026 +ms.date: 06/01/2026 ai-usage: ai-assisted --- @@ -36,8 +36,9 @@ This extension is intended for integration tests where the test depends heavily | Option | Description | |---------------------------------------|--------------------------------------------------------------------------------------------------| | `--retry-failed-tests` | Reruns any failed tests until they pass or until the maximum number of attempts is reached. Required to activate the extension. | -| `--retry-failed-tests-max-percentage` | Avoids rerunning tests when the percentage of failed test cases crosses the specified threshold. Can't be combined with `--retry-failed-tests-max-tests`. | -| `--retry-failed-tests-max-tests` | Avoids rerunning tests when the number of failed test cases crosses the specified limit. Can't be combined with `--retry-failed-tests-max-percentage`. | +| `--retry-failed-tests-max-percentage` | Avoids rerunning tests when the percentage of failed test cases crosses the specified threshold. Can't be combined with `--retry-failed-tests-max-tests`. Requires `--retry-failed-tests`. | +| `--retry-failed-tests-max-tests` | Avoids rerunning tests when the number of failed test cases crosses the specified limit. Can't be combined with `--retry-failed-tests-max-percentage`. Requires `--retry-failed-tests`. | +| `--retry-failed-tests-delay` | Adds a delay between retries. The delay is expressed as a time value, for example `200`, `500ms`, `1s`, `2.5m`, `1h`, `1d`. Default unit is milliseconds. Requires `--retry-failed-tests`. | Both threshold options (`--retry-failed-tests-max-percentage` and `--retry-failed-tests-max-tests`) require `--retry-failed-tests` to also be set. diff --git a/docs/core/testing/microsoft-testing-platform-terminal-output.md b/docs/core/testing/microsoft-testing-platform-terminal-output.md index c193d7fdf00b2..d9ff280fe3962 100644 --- a/docs/core/testing/microsoft-testing-platform-terminal-output.md +++ b/docs/core/testing/microsoft-testing-platform-terminal-output.md @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) terminal output description: Learn about the built-in terminal test reporter in MTP, including output modes, ANSI support, and progress indicators. author: evangelink ms.author: amauryleve -ms.date: 02/25/2026 +ms.date: 06/01/2026 ai-usage: ai-assisted --- @@ -47,4 +47,7 @@ The progress bar is written based on the selected mode: |---|---| | `--no-progress` | Disable reporting progress to screen. | | `--no-ansi` | Disable outputting ANSI escape characters to screen. | +| `--ansi` | Control whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). | | `--output` | Output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. | +| `--show-stdout` | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. | +| `--show-stderr` | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. | diff --git a/docs/core/testing/microsoft-testing-platform-test-reports.md b/docs/core/testing/microsoft-testing-platform-test-reports.md index 19bb3b9ba4709..17c7d8f140291 100644 --- a/docs/core/testing/microsoft-testing-platform-test-reports.md +++ b/docs/core/testing/microsoft-testing-platform-test-reports.md @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) test reports description: Learn about the MTP extensions for generating test report files (TRX, Azure DevOps). author: evangelink ms.author: amauryleve -ms.date: 02/25/2026 +ms.date: 06/01/2026 ai-usage: ai-assisted --- @@ -56,7 +56,16 @@ builder.TestHost.AddAzureDevOpsProvider(); | Option | Description | |---|---| -| `--report-azdo` | Enable outputting errors / warnings in CI builds. | -| `--report-azdo-severity` | Severity to use for the reported event. Options are: `error` (default) and `warning`. | +| `--report-azdo` | Enables the Azure DevOps report generator. Errors and warnings are written to the output in a format that Azure DevOps understands. | +| `--report-azdo-severity` | Severity to use for reported events. Valid values are `error` (default) and `warning`. | +| `--report-azdo-flaky-history` | Queries Azure DevOps test result history for the past N days (1-90) and annotates reported failures with flakiness context. Requires `--report-azdo`. | +| `--report-azdo-demote-known-flaky` | Demotes failures that are flaky enough in the Azure DevOps history window (default threshold is 25%) from errors to warnings. Requires `--report-azdo` and `--report-azdo-flaky-history`. | +| `--report-azdo-quarantine-file` | Path to a text file that lists quarantined test fully qualified names or glob patterns. Matching failures are reported as warnings. Requires `--report-azdo`. | +| `--report-azdo-upload-artifacts` | Uploads test result files and/or adds build tags to Azure DevOps. Valid values are `off` (default), `tags-only`, `files`, and `all`. | +| `--report-azdo-upload-artifact-include` | Includes files in the Azure DevOps artifact upload using glob patterns relative to the test results directory. Defaults to `**/*`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. | +| `--report-azdo-upload-artifact-exclude` | Excludes files from the Azure DevOps artifact upload using glob patterns relative to the test results directory. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. | +| `--report-azdo-upload-artifact-name` | Overrides the Azure DevOps artifact container name. Defaults to `TestResults_{assemblyName}_{tfm}`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. | +| `--publish-azdo-test-results` | Publishes test results live to the Azure DevOps **Tests** tab. | +| `--publish-azdo-run-name` | Sets a custom Azure DevOps test run name for live test-result publishing. Requires `--publish-azdo-test-results`. | The extension automatically detects that it is running in continuous integration (CI) environment by checking the `TF_BUILD` environment variable. diff --git a/docs/core/testing/microsoft-testing-platform-troubleshooting.md b/docs/core/testing/microsoft-testing-platform-troubleshooting.md index 162a3f15bb156..2c81a4d827dfb 100644 --- a/docs/core/testing/microsoft-testing-platform-troubleshooting.md +++ b/docs/core/testing/microsoft-testing-platform-troubleshooting.md @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) troubleshooting description: Troubleshoot MTP issues, exit codes, and known problems. author: Evangelink ms.author: amauryleve -ms.date: 05/25/2026 +ms.date: 06/01/2026 ai-usage: ai-assisted --- @@ -64,8 +64,8 @@ You can also enable the diagnostic logs using the environment variables: | `TESTINGPLATFORM_DIAGNOSTIC` | If set to `1`, enables the diagnostic logging. | | `TESTINGPLATFORM_DIAGNOSTIC_VERBOSITY` | Defines the verbosity level. The available values are `Trace`, `Debug`, `Information`, `Warning`, `Error`, or `Critical`. | | `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_DIRECTORY` | The output directory of the diagnostic logging, if not specified the file is generated in the default _TestResults_ directory. | -| `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` | The prefix for the log file name. Defaults to `"log_"`. | -| `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` | Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. | +| `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` | The prefix for the log file name. Defaults to `"log_"`. (Legacy name `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` is still honored for backward compatibility.) | +| `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` | Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. (Legacy name `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` is still honored for backward compatibility.) | > [!NOTE] > Environment variables take precedence over the command line arguments. From 93ccd625a302cc5119d7a45adbabb4bb896a1230 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Mon, 1 Jun 2026 16:44:46 +0200 Subject: [PATCH 2/4] Annotate MTP CLI options and env vars with introduction version Adds 'Available in MTP starting with version X.Y.Z.' notes for every recently added option/env var, so readers can tell at a glance which MTP release to target. Versions referenced: - 1.6.0: TESTINGPLATFORM_WAIT_ATTACH_DEBUGGER, TESTINGPLATFORM_LAUNCH_ATTACH_DEBUGGER - 1.8.0: --filter-uid - 1.9.0: --debug (replacing the 1.6.0 --debug-wait-attach), Azure DevOps extension stable (--report-azdo, --report-azdo-severity) - 2.1.0: --hangdump-type 'None' value - 2.2.1: --show-stdout, --show-stderr - 2.3.0: --list-tests json output, --ansi, --crash-report, --crash-report-if-supported, --crash-sequence, --hangdump-type-if-supported, --retry-failed-tests-delay, all Azure DevOps options beyond report-azdo/severity, TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX, TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE Also corrects an inaccurate note on --debug: the option was introduced in 1.9.0 (not 'before 2.0'). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../testing/microsoft-testing-platform-cli-options.md | 8 +++++++- docs/core/testing/microsoft-testing-platform-config.md | 10 ++++++++-- .../microsoft-testing-platform-crash-hang-dumps.md | 10 +++++----- docs/core/testing/microsoft-testing-platform-retry.md | 2 +- .../microsoft-testing-platform-terminal-output.md | 6 +++--- .../testing/microsoft-testing-platform-test-reports.md | 3 +++ .../microsoft-testing-platform-troubleshooting.md | 4 ++-- 7 files changed, 29 insertions(+), 14 deletions(-) diff --git a/docs/core/testing/microsoft-testing-platform-cli-options.md b/docs/core/testing/microsoft-testing-platform-cli-options.md index a1cc265dd5bb9..0254db5dc9fe0 100644 --- a/docs/core/testing/microsoft-testing-platform-cli-options.md +++ b/docs/core/testing/microsoft-testing-platform-cli-options.md @@ -58,7 +58,7 @@ This article gives a central entry point for MTP command-line options. Pauses test execution at startup so you can attach a debugger to the test process. Equivalent to setting the `TESTINGPLATFORM_WAIT_ATTACH_DEBUGGER` [environment variable](./microsoft-testing-platform-config.md#testingplatform_wait_attach_debugger-environment-variable) to `1`. Not supported on browser platforms. > [!NOTE] - > Before MTP 2.0, this option was named `--debug-wait-attach`. The old name was removed; use `--debug` instead. + > This option is available in MTP starting with version 1.9.0. It replaces the previous `--debug-wait-attach` option (introduced in MTP 1.6.0); the old name was removed and must no longer be used. - **`--diagnostic`** @@ -88,6 +88,9 @@ This article gives a central entry point for MTP command-line options. Filters the tests to run by their test node UIDs. Accepts one or more UIDs. + > [!NOTE] + > This option is available in MTP starting with version 1.8.0. + - **`--help`** Prints out a description of how to use the command. @@ -111,6 +114,9 @@ This article gives a central entry point for MTP command-line options. Lists the available tests without executing them. Optionally takes an argument that controls the output format: `text` (default, human-readable) or `json`. + > [!NOTE] + > The `json` output format is available in MTP starting with version 2.3.0. + - **`--maximum-failed-tests`** Specifies the maximum number of tests failures that, when reached, will stop the test run. Support for this switch requires framework authors to implement the `IGracefulStopTestExecutionCapability` capability. The exit code when reaching that amount of test failures is 13. For more information, see [MTP exit codes](microsoft-testing-platform-troubleshooting.md#exit-codes). diff --git a/docs/core/testing/microsoft-testing-platform-config.md b/docs/core/testing/microsoft-testing-platform-config.md index c9d458324ec6c..ce71ab4bb1bae 100644 --- a/docs/core/testing/microsoft-testing-platform-config.md +++ b/docs/core/testing/microsoft-testing-platform-config.md @@ -169,14 +169,14 @@ The output directory of the diagnostic logging. If not specified, the file is ge The prefix for the log file name. Defaults to `"log_"`. Matches the `--diagnostic-file-prefix` command-line option. > [!NOTE] -> The legacy `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` takes precedence. +> This environment variable name is available in MTP starting with version 2.3.0. The legacy `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` takes precedence. ### `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` environment variable Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. Matches the `--diagnostic-synchronous-write` command-line option. > [!NOTE] -> The legacy `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` takes precedence. +> This environment variable name is available in MTP starting with version 2.3.0. The legacy `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` takes precedence. ### `TESTINGPLATFORM_EXITCODE_IGNORE` environment variable @@ -194,10 +194,16 @@ When set to `1` or `true`, suppresses the startup banner, the copyright message, When set to `1`, the test process pauses at startup and waits for a debugger to attach before proceeding. Equivalent to the `--debug` command-line option. Not supported on browser platforms. +> [!NOTE] +> This environment variable is available in MTP starting with version 1.6.0. + ### `TESTINGPLATFORM_LAUNCH_ATTACH_DEBUGGER` environment variable When set to `1`, the test process calls `Debugger.Launch()` at startup, which prompts the system to launch a just-in-time debugger and attach it to the process. Use this variable to debug startup-time issues (for example, server-mode handshake) that occur before you can manually attach. On non-Windows platforms, the behavior depends on the configured JIT debugger. +> [!NOTE] +> This environment variable is available in MTP starting with version 1.6.0. + > [!NOTE] > Diagnostic-related environment variables take precedence over their corresponding `--diagnostic-*` command-line arguments. diff --git a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md index eb3bcb4decf63..78190f773799e 100644 --- a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md +++ b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md @@ -32,9 +32,9 @@ builder.TestHostControllers.AddCrashDumpProvider(); | `--crashdump` | Generates a dump file when the test host process crashes. Supported in .NET 6.0+. | | `⁠-⁠-⁠crashdump-⁠filename` | Specifies the file name of the dump. | | `--crashdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Triage`, `Full`. Defaults as `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | -| `--crash-report` | [Linux/macOS only] Generates a JSON crash report when the test process crashes. Combine with `--crashdump` to also generate a dump file. Requires .NET 7+ when used alone; .NET 6+ when combined with `--crashdump`. Not supported on Windows because the .NET runtime ignores the underlying `DOTNET_EnableCrashReport` and `DOTNET_EnableCrashReportOnly` environment variables on Windows ([dotnet/runtime#80191](https://github.com/dotnet/runtime/issues/80191)). | -| `--crash-report-if-supported` | Same as `--crash-report` but silently ignored (with an informational message) on platforms where crash report generation isn't supported. Use this option to keep the same command line across CI matrices that include Windows. Mutually exclusive with `--crash-report`. | -| `--crash-sequence` | Controls whether a sequence file listing the tests that started and ended during the test session is generated alongside the crash dump or crash report. The file makes it possible to identify the tests that were running at the time of the crash without inspecting the dump. Valid values are `on` (default; also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). | +| `--crash-report` | [Linux/macOS only] Generates a JSON crash report when the test process crashes. Combine with `--crashdump` to also generate a dump file. Requires .NET 7+ when used alone; .NET 6+ when combined with `--crashdump`. Not supported on Windows because the .NET runtime ignores the underlying `DOTNET_EnableCrashReport` and `DOTNET_EnableCrashReportOnly` environment variables on Windows ([dotnet/runtime#80191](https://github.com/dotnet/runtime/issues/80191)). Available in MTP starting with version 2.3.0. | +| `--crash-report-if-supported` | Same as `--crash-report` but silently ignored (with an informational message) on platforms where crash report generation isn't supported. Use this option to keep the same command line across CI matrices that include Windows. Mutually exclusive with `--crash-report`. Available in MTP starting with version 2.3.0. | +| `--crash-sequence` | Controls whether a sequence file listing the tests that started and ended during the test session is generated alongside the crash dump or crash report. The file makes it possible to identify the tests that were running at the time of the crash without inspecting the dump. Valid values are `on` (default; also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. | > [!CAUTION] > The extension isn't compatible with .NET Framework and will be silently ignored. For .NET Framework support, you enable the postmortem debugging with Sysinternals ProcDump. For more information, see [Enabling Postmortem Debugging: Window Sysinternals ProcDump](/windows-hardware/drivers/debugger/enabling-postmortem-debugging#window-sysinternals-procdump). The postmortem debugging solution will also collect process crash information for .NET so you can avoid the use of the extension if you're targeting both .NET and .NET Framework test applications. @@ -57,5 +57,5 @@ builder.TestHostControllers.AddHangDumpProvider(); | `--hangdump` | Generates a dump file in case the test host process hangs. | | `-⁠-hangdump-filename` | Specifies the file name of the dump. Supports the following placeholders: `{pname}` (test application name), `{pid}` (process ID), `{asm}` (entry assembly name), `{tfm}` (target framework moniker), `{time}` (timestamp). The legacy `%p` token (process ID) is also supported for backward compatibility. | | `--hangdump-timeout` | Specifies the duration of no activity after which the dump is generated. The timeout value is specified in one of the following formats:
`1.5h`, `1.5hour`, `1.5hours`
`90m`, `90min`, `90minute`, `90minutes`
`5400s`, `5400sec`, `5400second`, `5400seconds`
`500ms`, `500mil`, `500millisecond`, `500milliseconds`
`1d`, `1day`, `1days`. A bare number (with no suffix) is interpreted as milliseconds. Defaults to `30m` (30 minutes). | -| `--hangdump-type` | Specifies the type of the dump. Valid values on .NET (Core) are `Mini`, `Heap`, `Full`, `Triage`, `None`. On .NET Framework, the valid values are `Mini`, `Heap`, `Full`, `None` (`Triage` is unavailable). Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | -| `--hangdump-type-if-supported` | Same as `--hangdump-type` but silently falls back (with an informational message) to the closest supported dump type when the requested type isn't available on the current runtime. For example, `Triage` is only supported on .NET (Core) and falls back to `Mini` on .NET Framework. Use this option to keep the same command line across CI matrices that mix .NET (Core) and .NET Framework. Valid values are `Mini`, `Heap`, `Full`, `Triage`, `None`. Mutually exclusive with `--hangdump-type`. | +| `--hangdump-type` | Specifies the type of the dump. Valid values on .NET (Core) are `Mini`, `Heap`, `Full`, `Triage`, `None`. On .NET Framework, the valid values are `Mini`, `Heap`, `Full`, `None` (`Triage` is unavailable). Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). The `None` value is available in MTP starting with version 2.1.0. | +| `--hangdump-type-if-supported` | Same as `--hangdump-type` but silently falls back (with an informational message) to the closest supported dump type when the requested type isn't available on the current runtime. For example, `Triage` is only supported on .NET (Core) and falls back to `Mini` on .NET Framework. Use this option to keep the same command line across CI matrices that mix .NET (Core) and .NET Framework. Valid values are `Mini`, `Heap`, `Full`, `Triage`, `None`. Mutually exclusive with `--hangdump-type`. Available in MTP starting with version 2.3.0. | diff --git a/docs/core/testing/microsoft-testing-platform-retry.md b/docs/core/testing/microsoft-testing-platform-retry.md index 81282f6e91403..a572554a6557f 100644 --- a/docs/core/testing/microsoft-testing-platform-retry.md +++ b/docs/core/testing/microsoft-testing-platform-retry.md @@ -38,7 +38,7 @@ This extension is intended for integration tests where the test depends heavily | `--retry-failed-tests` | Reruns any failed tests until they pass or until the maximum number of attempts is reached. Required to activate the extension. | | `--retry-failed-tests-max-percentage` | Avoids rerunning tests when the percentage of failed test cases crosses the specified threshold. Can't be combined with `--retry-failed-tests-max-tests`. Requires `--retry-failed-tests`. | | `--retry-failed-tests-max-tests` | Avoids rerunning tests when the number of failed test cases crosses the specified limit. Can't be combined with `--retry-failed-tests-max-percentage`. Requires `--retry-failed-tests`. | -| `--retry-failed-tests-delay` | Adds a delay between retries. The delay is expressed as a time value, for example `200`, `500ms`, `1s`, `2.5m`, `1h`, `1d`. Default unit is milliseconds. Requires `--retry-failed-tests`. | +| `--retry-failed-tests-delay` | Adds a delay between retries. The delay is expressed as a time value, for example `200`, `500ms`, `1s`, `2.5m`, `1h`, `1d`. Default unit is milliseconds. Requires `--retry-failed-tests`. Available in MTP starting with version 2.3.0. | Both threshold options (`--retry-failed-tests-max-percentage` and `--retry-failed-tests-max-tests`) require `--retry-failed-tests` to also be set. diff --git a/docs/core/testing/microsoft-testing-platform-terminal-output.md b/docs/core/testing/microsoft-testing-platform-terminal-output.md index d9ff280fe3962..92c233ec8752a 100644 --- a/docs/core/testing/microsoft-testing-platform-terminal-output.md +++ b/docs/core/testing/microsoft-testing-platform-terminal-output.md @@ -47,7 +47,7 @@ The progress bar is written based on the selected mode: |---|---| | `--no-progress` | Disable reporting progress to screen. | | `--no-ansi` | Disable outputting ANSI escape characters to screen. | -| `--ansi` | Control whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). | +| `--ansi` | Control whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. | | `--output` | Output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. | -| `--show-stdout` | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. | -| `--show-stderr` | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. | +| `--show-stdout` | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. Available in MTP starting with version 2.2.1. | +| `--show-stderr` | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. Available in MTP starting with version 2.2.1. | diff --git a/docs/core/testing/microsoft-testing-platform-test-reports.md b/docs/core/testing/microsoft-testing-platform-test-reports.md index 17c7d8f140291..210d8263f4a14 100644 --- a/docs/core/testing/microsoft-testing-platform-test-reports.md +++ b/docs/core/testing/microsoft-testing-platform-test-reports.md @@ -68,4 +68,7 @@ builder.TestHost.AddAzureDevOpsProvider(); | `--publish-azdo-test-results` | Publishes test results live to the Azure DevOps **Tests** tab. | | `--publish-azdo-run-name` | Sets a custom Azure DevOps test run name for live test-result publishing. Requires `--publish-azdo-test-results`. | +> [!NOTE] +> The Azure DevOps extension became stable in MTP 1.9.0 (`--report-azdo` and `--report-azdo-severity`). All other options in the table — `--report-azdo-flaky-history`, `--report-azdo-demote-known-flaky`, `--report-azdo-quarantine-file`, `--report-azdo-upload-artifacts`, `--report-azdo-upload-artifact-include`, `--report-azdo-upload-artifact-exclude`, `--report-azdo-upload-artifact-name`, `--publish-azdo-test-results`, and `--publish-azdo-run-name` — are available in MTP starting with version 2.3.0. + The extension automatically detects that it is running in continuous integration (CI) environment by checking the `TF_BUILD` environment variable. diff --git a/docs/core/testing/microsoft-testing-platform-troubleshooting.md b/docs/core/testing/microsoft-testing-platform-troubleshooting.md index 2c81a4d827dfb..6a66c259c9c00 100644 --- a/docs/core/testing/microsoft-testing-platform-troubleshooting.md +++ b/docs/core/testing/microsoft-testing-platform-troubleshooting.md @@ -64,8 +64,8 @@ You can also enable the diagnostic logs using the environment variables: | `TESTINGPLATFORM_DIAGNOSTIC` | If set to `1`, enables the diagnostic logging. | | `TESTINGPLATFORM_DIAGNOSTIC_VERBOSITY` | Defines the verbosity level. The available values are `Trace`, `Debug`, `Information`, `Warning`, `Error`, or `Critical`. | | `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_DIRECTORY` | The output directory of the diagnostic logging, if not specified the file is generated in the default _TestResults_ directory. | -| `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` | The prefix for the log file name. Defaults to `"log_"`. (Legacy name `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` is still honored for backward compatibility.) | -| `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` | Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. (Legacy name `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` is still honored for backward compatibility.) | +| `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` | The prefix for the log file name. Defaults to `"log_"`. Available in MTP starting with version 2.3.0; the legacy name `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` is still honored for backward compatibility. | +| `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` | Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. Available in MTP starting with version 2.3.0; the legacy name `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` is still honored for backward compatibility. | > [!NOTE] > Environment variables take precedence over the command line arguments. From c65232b895f4b5ae1f417dac30ad558861a1dc93 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Mon, 1 Jun 2026 09:22:33 -0700 Subject: [PATCH 3/4] Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Genevieve Warren <24882762+gewarren@users.noreply.github.com> --- .../testing/microsoft-testing-platform-config.md | 4 ++-- .../microsoft-testing-platform-crash-hang-dumps.md | 8 ++++---- .../core/testing/microsoft-testing-platform-retry.md | 2 +- .../microsoft-testing-platform-terminal-output.md | 12 ++++++------ 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/core/testing/microsoft-testing-platform-config.md b/docs/core/testing/microsoft-testing-platform-config.md index ce71ab4bb1bae..82d94a17eba9a 100644 --- a/docs/core/testing/microsoft-testing-platform-config.md +++ b/docs/core/testing/microsoft-testing-platform-config.md @@ -169,14 +169,14 @@ The output directory of the diagnostic logging. If not specified, the file is ge The prefix for the log file name. Defaults to `"log_"`. Matches the `--diagnostic-file-prefix` command-line option. > [!NOTE] -> This environment variable name is available in MTP starting with version 2.3.0. The legacy `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` takes precedence. +> This environment variable name is available in MTP starting with version 2.3.0. The legacy `TESTINGPLATFORM_DIAGNOSTIC_OUTPUT_FILEPREFIX` environment variable is still honored for backward compatibility but is deprecated and might be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_FILE_PREFIX` takes precedence. ### `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` environment variable Forces the built-in file logger to synchronously write logs. Useful for scenarios where you don't want to lose any log entries (if the process crashes). This does slow down the test execution. Matches the `--diagnostic-synchronous-write` command-line option. > [!NOTE] -> This environment variable name is available in MTP starting with version 2.3.0. The legacy `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` environment variable is still honored for backward compatibility but is deprecated and may be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` takes precedence. +> This environment variable name is available in MTP starting with version 2.3.0. The legacy `TESTINGPLATFORM_DIAGNOSTIC_FILELOGGER_SYNCHRONOUSWRITE` environment variable is still honored for backward compatibility but is deprecated and might be removed in a future major version. When both variables are set, `TESTINGPLATFORM_DIAGNOSTIC_SYNCHRONOUS_WRITE` takes precedence. ### `TESTINGPLATFORM_EXITCODE_IGNORE` environment variable diff --git a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md index 78190f773799e..b04593960ed10 100644 --- a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md +++ b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md @@ -30,9 +30,9 @@ builder.TestHostControllers.AddCrashDumpProvider(); | Option | Description | |--|--| | `--crashdump` | Generates a dump file when the test host process crashes. Supported in .NET 6.0+. | -| `⁠-⁠-⁠crashdump-⁠filename` | Specifies the file name of the dump. | +| `--crashdump-filename` | Specifies the file name of the dump. | | `--crashdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Triage`, `Full`. Defaults as `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | -| `--crash-report` | [Linux/macOS only] Generates a JSON crash report when the test process crashes. Combine with `--crashdump` to also generate a dump file. Requires .NET 7+ when used alone; .NET 6+ when combined with `--crashdump`. Not supported on Windows because the .NET runtime ignores the underlying `DOTNET_EnableCrashReport` and `DOTNET_EnableCrashReportOnly` environment variables on Windows ([dotnet/runtime#80191](https://github.com/dotnet/runtime/issues/80191)). Available in MTP starting with version 2.3.0. | +| `--crash-report` | [Linux/macOS only] Generates a JSON crash report when the test process crashes. Combine with `--crashdump` to also generate a dump file. Requires .NET 7+ when used alone or .NET 6+ when combined with `--crashdump`. Not supported on Windows because the .NET runtime ignores the underlying `DOTNET_EnableCrashReport` and `DOTNET_EnableCrashReportOnly` environment variables on Windows ([dotnet/runtime#80191](https://github.com/dotnet/runtime/issues/80191)). Available in MTP starting with version 2.3.0. | | `--crash-report-if-supported` | Same as `--crash-report` but silently ignored (with an informational message) on platforms where crash report generation isn't supported. Use this option to keep the same command line across CI matrices that include Windows. Mutually exclusive with `--crash-report`. Available in MTP starting with version 2.3.0. | | `--crash-sequence` | Controls whether a sequence file listing the tests that started and ended during the test session is generated alongside the crash dump or crash report. The file makes it possible to identify the tests that were running at the time of the crash without inspecting the dump. Valid values are `on` (default; also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. | @@ -55,7 +55,7 @@ builder.TestHostControllers.AddHangDumpProvider(); | Option | Description | |--|--| | `--hangdump` | Generates a dump file in case the test host process hangs. | -| `-⁠-hangdump-filename` | Specifies the file name of the dump. Supports the following placeholders: `{pname}` (test application name), `{pid}` (process ID), `{asm}` (entry assembly name), `{tfm}` (target framework moniker), `{time}` (timestamp). The legacy `%p` token (process ID) is also supported for backward compatibility. | +| `--hangdump-filename` | Specifies the file name of the dump. Supports the following placeholders: `{pname}` (test application name), `{pid}` (process ID), `{asm}` (entry assembly name), `{tfm}` (target framework moniker), `{time}` (timestamp). The legacy `%p` token (process ID) is also supported for backward compatibility. | | `--hangdump-timeout` | Specifies the duration of no activity after which the dump is generated. The timeout value is specified in one of the following formats:
`1.5h`, `1.5hour`, `1.5hours`
`90m`, `90min`, `90minute`, `90minutes`
`5400s`, `5400sec`, `5400second`, `5400seconds`
`500ms`, `500mil`, `500millisecond`, `500milliseconds`
`1d`, `1day`, `1days`. A bare number (with no suffix) is interpreted as milliseconds. Defaults to `30m` (30 minutes). | -| `--hangdump-type` | Specifies the type of the dump. Valid values on .NET (Core) are `Mini`, `Heap`, `Full`, `Triage`, `None`. On .NET Framework, the valid values are `Mini`, `Heap`, `Full`, `None` (`Triage` is unavailable). Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). The `None` value is available in MTP starting with version 2.1.0. | +| `--hangdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Full`, `Triage` (not available on .NET Framework), `None`. Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). The `None` value is available in MTP starting with version 2.1.0. | | `--hangdump-type-if-supported` | Same as `--hangdump-type` but silently falls back (with an informational message) to the closest supported dump type when the requested type isn't available on the current runtime. For example, `Triage` is only supported on .NET (Core) and falls back to `Mini` on .NET Framework. Use this option to keep the same command line across CI matrices that mix .NET (Core) and .NET Framework. Valid values are `Mini`, `Heap`, `Full`, `Triage`, `None`. Mutually exclusive with `--hangdump-type`. Available in MTP starting with version 2.3.0. | diff --git a/docs/core/testing/microsoft-testing-platform-retry.md b/docs/core/testing/microsoft-testing-platform-retry.md index a572554a6557f..275edcf0b06df 100644 --- a/docs/core/testing/microsoft-testing-platform-retry.md +++ b/docs/core/testing/microsoft-testing-platform-retry.md @@ -38,7 +38,7 @@ This extension is intended for integration tests where the test depends heavily | `--retry-failed-tests` | Reruns any failed tests until they pass or until the maximum number of attempts is reached. Required to activate the extension. | | `--retry-failed-tests-max-percentage` | Avoids rerunning tests when the percentage of failed test cases crosses the specified threshold. Can't be combined with `--retry-failed-tests-max-tests`. Requires `--retry-failed-tests`. | | `--retry-failed-tests-max-tests` | Avoids rerunning tests when the number of failed test cases crosses the specified limit. Can't be combined with `--retry-failed-tests-max-percentage`. Requires `--retry-failed-tests`. | -| `--retry-failed-tests-delay` | Adds a delay between retries. The delay is expressed as a time value, for example `200`, `500ms`, `1s`, `2.5m`, `1h`, `1d`. Default unit is milliseconds. Requires `--retry-failed-tests`. Available in MTP starting with version 2.3.0. | +| `--retry-failed-tests-delay` | Adds a delay between retries. The delay is expressed as a time value, for example `200`, `500ms`, `1s`, `2.5m`, `1h`, or `1d`. Default unit is milliseconds. Requires `--retry-failed-tests`. Available in MTP starting with version 2.3.0. | Both threshold options (`--retry-failed-tests-max-percentage` and `--retry-failed-tests-max-tests`) require `--retry-failed-tests` to also be set. diff --git a/docs/core/testing/microsoft-testing-platform-terminal-output.md b/docs/core/testing/microsoft-testing-platform-terminal-output.md index 92c233ec8752a..83a78d95ab0f0 100644 --- a/docs/core/testing/microsoft-testing-platform-terminal-output.md +++ b/docs/core/testing/microsoft-testing-platform-terminal-output.md @@ -45,9 +45,9 @@ The progress bar is written based on the selected mode: | Option | Description | |---|---| -| `--no-progress` | Disable reporting progress to screen. | -| `--no-ansi` | Disable outputting ANSI escape characters to screen. | -| `--ansi` | Control whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. | -| `--output` | Output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. | -| `--show-stdout` | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. Available in MTP starting with version 2.2.1. | -| `--show-stderr` | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, `None`. Default is `All`. Available in MTP starting with version 2.2.1. | +| `--no-progress` | Disables reporting progress to screen. | +| `--no-ansi` | Disables outputting ANSI escape characters to screen. | +| `--ansi` | Controls whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. | +| `--output` | Specifies the output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. | +| `--show-stdout` | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. Available in MTP starting with version 2.2.1. | +| `--show-stderr` | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. Available in MTP starting with version 2.2.1. | From 2395b3043a629ab3f5b7407978d6b673989f2ad4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Tue, 2 Jun 2026 10:14:24 +0200 Subject: [PATCH 4/4] Address Copilot review feedback on crash/hang dump options - Drop 'silently' from --crash-report-if-supported and --hangdump-type-if-supported descriptions: the option emits an informational message, so the behavior isn't silent (Copilot comment). - Apply Oxford commas to remaining value/placeholder lists in --crashdump-type, --crash-sequence, --hangdump-filename, --hangdump-type, and --hangdump-type-if-supported for consistency. - Tighten --crashdump-type wording ('Defaults to', Oxford comma in value list). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../microsoft-testing-platform-crash-hang-dumps.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md index b04593960ed10..d35511573a6ae 100644 --- a/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md +++ b/docs/core/testing/microsoft-testing-platform-crash-hang-dumps.md @@ -31,10 +31,10 @@ builder.TestHostControllers.AddCrashDumpProvider(); |--|--| | `--crashdump` | Generates a dump file when the test host process crashes. Supported in .NET 6.0+. | | `--crashdump-filename` | Specifies the file name of the dump. | -| `--crashdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Triage`, `Full`. Defaults as `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | +| `--crashdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Triage`, and `Full`. Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). | | `--crash-report` | [Linux/macOS only] Generates a JSON crash report when the test process crashes. Combine with `--crashdump` to also generate a dump file. Requires .NET 7+ when used alone or .NET 6+ when combined with `--crashdump`. Not supported on Windows because the .NET runtime ignores the underlying `DOTNET_EnableCrashReport` and `DOTNET_EnableCrashReportOnly` environment variables on Windows ([dotnet/runtime#80191](https://github.com/dotnet/runtime/issues/80191)). Available in MTP starting with version 2.3.0. | -| `--crash-report-if-supported` | Same as `--crash-report` but silently ignored (with an informational message) on platforms where crash report generation isn't supported. Use this option to keep the same command line across CI matrices that include Windows. Mutually exclusive with `--crash-report`. Available in MTP starting with version 2.3.0. | -| `--crash-sequence` | Controls whether a sequence file listing the tests that started and ended during the test session is generated alongside the crash dump or crash report. The file makes it possible to identify the tests that were running at the time of the crash without inspecting the dump. Valid values are `on` (default; also accepts `true`, `enable`, `1`) or `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. | +| `--crash-report-if-supported` | Same as `--crash-report` but ignored, with an informational message, on platforms where crash report generation isn't supported. Use this option to keep the same command line across CI matrices that include Windows. Mutually exclusive with `--crash-report`. Available in MTP starting with version 2.3.0. | +| `--crash-sequence` | Controls whether a sequence file listing the tests that started and ended during the test session is generated alongside the crash dump or crash report. The file makes it possible to identify the tests that were running at the time of the crash without inspecting the dump. Valid values are `on` (default; also accepts `true`, `enable`, and `1`) or `off` (also accepts `false`, `disable`, and `0`). Available in MTP starting with version 2.3.0. | > [!CAUTION] > The extension isn't compatible with .NET Framework and will be silently ignored. For .NET Framework support, you enable the postmortem debugging with Sysinternals ProcDump. For more information, see [Enabling Postmortem Debugging: Window Sysinternals ProcDump](/windows-hardware/drivers/debugger/enabling-postmortem-debugging#window-sysinternals-procdump). The postmortem debugging solution will also collect process crash information for .NET so you can avoid the use of the extension if you're targeting both .NET and .NET Framework test applications. @@ -55,7 +55,7 @@ builder.TestHostControllers.AddHangDumpProvider(); | Option | Description | |--|--| | `--hangdump` | Generates a dump file in case the test host process hangs. | -| `--hangdump-filename` | Specifies the file name of the dump. Supports the following placeholders: `{pname}` (test application name), `{pid}` (process ID), `{asm}` (entry assembly name), `{tfm}` (target framework moniker), `{time}` (timestamp). The legacy `%p` token (process ID) is also supported for backward compatibility. | +| `--hangdump-filename` | Specifies the file name of the dump. Supports the following placeholders: `{pname}` (test application name), `{pid}` (process ID), `{asm}` (entry assembly name), `{tfm}` (target framework moniker), and `{time}` (timestamp). The legacy `%p` token (process ID) is also supported for backward compatibility. | | `--hangdump-timeout` | Specifies the duration of no activity after which the dump is generated. The timeout value is specified in one of the following formats:
`1.5h`, `1.5hour`, `1.5hours`
`90m`, `90min`, `90minute`, `90minutes`
`5400s`, `5400sec`, `5400second`, `5400seconds`
`500ms`, `500mil`, `500millisecond`, `500milliseconds`
`1d`, `1day`, `1days`. A bare number (with no suffix) is interpreted as milliseconds. Defaults to `30m` (30 minutes). | -| `--hangdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Full`, `Triage` (not available on .NET Framework), `None`. Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). The `None` value is available in MTP starting with version 2.1.0. | -| `--hangdump-type-if-supported` | Same as `--hangdump-type` but silently falls back (with an informational message) to the closest supported dump type when the requested type isn't available on the current runtime. For example, `Triage` is only supported on .NET (Core) and falls back to `Mini` on .NET Framework. Use this option to keep the same command line across CI matrices that mix .NET (Core) and .NET Framework. Valid values are `Mini`, `Heap`, `Full`, `Triage`, `None`. Mutually exclusive with `--hangdump-type`. Available in MTP starting with version 2.3.0. | +| `--hangdump-type` | Specifies the type of the dump. Valid values are `Mini`, `Heap`, `Full`, `Triage` (not available on .NET Framework), and `None`. Defaults to `Full`. For more information, see [Types of mini dumps](../diagnostics/collect-dumps-crash.md#types-of-mini-dumps). The `None` value is available in MTP starting with version 2.1.0. | +| `--hangdump-type-if-supported` | Same as `--hangdump-type` but falls back, with an informational message, to the closest supported dump type when the requested type isn't available on the current runtime. For example, `Triage` is only supported on .NET (Core) and falls back to `Mini` on .NET Framework. Use this option to keep the same command line across CI matrices that mix .NET (Core) and .NET Framework. Valid values are `Mini`, `Heap`, `Full`, `Triage`, and `None`. Mutually exclusive with `--hangdump-type`. Available in MTP starting with version 2.3.0. |