From 3ce10ddccaca24aa43cef417aee81d7e8639ea8e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Tue, 9 Jun 2026 08:03:19 +0200 Subject: [PATCH] docs: Update glossary - weekly full scan (2026-06-09) Add 7 new terms introduced in recent changes: - ArtifactNamingHelper: template-based artifact naming helper used by HangDump/CrashDump extensions - AzureFoundry: new MTP extension for Azure AI Foundry (Azure OpenAI) integration - HtmlReport: self-contained HTML test report extension (was missing from glossary) - IChatClientProvider: new AI provider abstraction interface in Microsoft.Testing.Platform.AI - JUnitReport: new MTP extension for JUnit XML test report generation (experimental) - Microsoft.Testing.Platform.AI: new AI extensibility abstractions package for MTP - testconfig.json: MTP per-project configuration file with environmentVariables support Based on commit 1b64756 which introduced Microsoft.Testing.Extensions.AzureFoundry, Microsoft.Testing.Extensions.JUnitReport, Microsoft.Testing.Platform.AI packages, and related docs (RFCs 001-AI-Extensibility, 002-TestConfig-EnvironmentVariables, ArtifactNamingHelper.md). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/glossary.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/glossary.md b/docs/glossary.md index 3a0d4fb08c..12602e7f7d 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -12,10 +12,18 @@ A .NET fluent-assertion library ([NuGet: AwesomeAssertions](https://www.nuget.or An MTP struct (`ArgumentArity.cs`) that defines the minimum and maximum number of values a command-line option accepts. Provides five predefined constants: `Zero` (0,0), `ZeroOrOne` (0,1), `ZeroOrMore` (0,∞), `ExactlyOne` (1,1), and `OneOrMore` (1,∞). Used by `ICommandLineOptionsProvider` implementations to declare option shapes. +### ArtifactNamingHelper + +A shared static helper compiled into MTP extensions via file linking (no NuGet service registration or InternalsVisibleTo required) that provides template-based naming for test artifact files (dump files, report files, etc.). Templates are strings containing `{placeholder}` tokens (case-sensitive, lowercase): `{pname}` (process name), `{pid}` (process ID), `{asm}` (entry-assembly name), `{tfm}` (target framework moniker, best-effort runtime detection), and `{time}` (high-precision UTC timestamp). Legacy `%p` patterns from earlier hang-dump versions continue to work. Custom per-call overrides can replace default placeholder values via a `Dictionary`. Used by the [HangDump](#hangdump) and [CrashDump](#crashdump) extensions. + ### AzureDevOpsReport An MTP extension (`Microsoft.Testing.Extensions.AzureDevOpsReport`) that formats and reports test results to Azure DevOps pipelines. It generates pipeline-compatible output including TFM and test name details for richer CI reporting. +### AzureFoundry + +An MTP extension (`Microsoft.Testing.Extensions.AzureFoundry`) that integrates [Azure AI Foundry](https://azure.microsoft.com/products/ai-foundry) (Azure OpenAI) with Microsoft.Testing.Platform as an [IChatClientProvider](#ichatclientprovider) implementation. It reads Azure OpenAI connection settings from environment variables and supplies AI chat-client capabilities to any testing extension that consumes the [Microsoft.Testing.Platform.AI](#microsofttestingplatformai) abstractions. This is the reference implementation of the `Microsoft.Testing.Platform.AI` abstractions. + ## C ### CodeCoverage @@ -60,8 +68,16 @@ A specific code component (function, struct, or class) selected for formal verif An MTP extension (`Microsoft.Testing.Extensions.HangDump`) that captures a process memory dump when a test exceeds a configured timeout. Helps diagnose deadlocks, infinite loops, or unexpectedly slow tests. +### HtmlReport + +An MTP extension (`Microsoft.Testing.Extensions.HtmlReport`) that generates a self-contained HTML test report at the end of a test session. The report inlines all CSS, JavaScript, and test data into a single `.html` file with no external dependencies, making it suitable for archiving as a CI artifact, attaching to PR comments, or sharing by email. Features include failed-test-first ordering, free-text search, sort/filter by outcome or duration, an expandable per-test detail panel (error message, stack trace, stdout/stderr), and automatic light/dark theme following the system preference. Pagination keeps the report usable for very large test runs. Currently **experimental** — CLI option, layout, and on-disk format may change without notice. Enable via the `--report-html` CLI option. + ## I +### IChatClientProvider + +An MTP interface (`Microsoft.Testing.Platform.AI.IChatClientProvider`) that defines the contract for AI provider integrations in the testing platform. Exposes four members: `IsAvailable` (whether required configuration, e.g. environment variables, is present), `HasToolsCapability` (whether the provider supports tool/function calling, e.g. MCP tools), `ModelName` (the model in use), and `CreateChatClientAsync` (factory that returns an `IChatClient` from [Microsoft.Extensions.AI](https://www.nuget.org/packages/Microsoft.Extensions.AI)). Extensions that need AI capabilities consume an injected `IChatClientProvider` rather than implementing provider-specific logic themselves. The interface is shipped in [Microsoft.Testing.Platform.AI](#microsofttestingplatformai); the reference implementation is [AzureFoundry](#azurefoundry). + ### Informal Spec (FV) An intermediate artifact in the [Formal Verification (FV)](#formal-verification-fv) workflow that documents the behavioural properties of an [FV Target](#fv-target) in plain English (or structured natural language), before writing formal [Lean 4](#lean-4) proofs. An informal spec lists preconditions, postconditions, edge-case expectations, and any confirmed bugs discovered during analysis. It corresponds to Phase 2 of the FV target lifecycle and lives in the `formal-verification/specs/` directory. @@ -76,6 +92,10 @@ An MSBuild property (`true` schema and is accepted by Jenkins (`junit` step), GitLab CI (`junit:` artifact reports), Azure DevOps (`PublishTestResults@2` with `testResultsFormat: 'JUnit'`), CircleCI, GitHub Actions test reporters, and most other CI tooling. MTP's hierarchical [TestNode](#testnode) tree is preserved as a `` element inside each ``, allowing tools to reconstruct hierarchy. Auto-registers via MSBuild by setting `true`. Currently **experimental** — the API, CLI options, and on-disk format may change without notice. Enable via `--report-junit`; override filename with `--report-junit-filename`. + ## L ### Lean 4 @@ -124,6 +144,10 @@ See **Microsoft.Testing.Platform**. A lightweight, extensible test platform for .NET that serves as a modern alternative to VSTest. MTP ships as a NuGet package (`Microsoft.Testing.Platform`) and provides the core infrastructure for running tests: command-line parsing, test session management, result reporting, and an extension model. Test frameworks (e.g., MSTest, xUnit adapters) and extensions (e.g., CrashDump, HangDump) plug into MTP. +### Microsoft.Testing.Platform.AI + +A NuGet package (`Microsoft.Testing.Platform.AI`) that provides AI extensibility abstractions for Microsoft.Testing.Platform. It defines the [IChatClientProvider](#ichatclientprovider) interface and leverages [Microsoft.Extensions.AI](https://www.nuget.org/packages/Microsoft.Extensions.AI) types so that test frameworks and extensions can consume Large Language Model (LLM) capabilities — flaky test analysis, crash dump analysis, test failure root-cause analysis, and more — without implementing provider-specific logic. This package ships the **abstractions only**; an AI provider implementation such as [Microsoft.Testing.Extensions.AzureFoundry](#azurefoundry) must also be registered to supply actual AI capabilities. See `docs/microsoft.testing.platform/001-AI-Extensibility.md` for the design RFC. + ## N ### NopFilter @@ -158,6 +182,10 @@ Request for Comments document in the `docs/RFCs/` folder. RFCs describe design d ## T +### testconfig.json + +The per-project configuration file for Microsoft.Testing.Platform, placed at the project root and read at test startup. Supports multiple top-level sections; a key one is `environmentVariables`, which declares environment variables to set on the test host process — mirroring the `` element of legacy `.runsettings` and removing the need to write a custom `ITestHostEnvironmentVariableProvider` (introduced in RFC 002, `docs/RFCs/002-TestConfig-EnvironmentVariables.md`). When the `environmentVariables` section is present and non-empty, MTP activates the **controller process model**: the launching process becomes the controller, injects the declared variables into `ProcessStartInfo`, and spawns the actual test host as a child process. + ### TestNode A core MTP class (`Microsoft.Testing.Platform.Extensions.Messages.TestNode`) that represents a single test item — either discovered or executed. Each `TestNode` carries a unique `Uid` (`TestNodeUid`), a human-readable `DisplayName`, and a [PropertyBag](#propertybag) of typed properties (state, timing, file location, metadata, etc.). `TestNode` instances are published to the `IMessageBus` by test framework adapters during discovery and execution phases.