Add event-level task telemetry (build/tasks/details) by jankratochvilcz · Pull Request #13609 · dotnet/msbuild

jankratochvilcz · 2026-04-27T11:32:15Z

Context

Per-task execution details are already published to the VS telemetry sink via the Activity-based path (IActivityTelemetryDataHolder -> TelemetryComplexProperty, which accepts arbitrary objects and serializes them itself). The CLI/SDK telemetry path goes through TelemetryEventArgs, whose Properties are IDictionary<string, string?> -- so the same data has to be reduced to strings before it can leave the engine.

Changes Made

New telemetry event build/tasks/details emitted from BuildManager after the existing BuildTelemetry event. Gated by the existing MSBUILDTELEMETRYEXCLUDETASKSDETAILS env var (Traits.ExcludeTasksDetailsFromTelemetry).
Properties on the event:
- TaskCount - number of distinct tasks in the payload (capped at 100).
- TotalTaskCount - total number of distinct tasks observed.
- Tasks - JSON array of TaskDetailInfo records (top 100 by ExecutionsCount), ready for Kusto mv-expand.
Serialization uses System.Text.Json.JsonSerializer with JsonIgnoreCondition.WhenWritingNull (so FactoryName / TaskHostRuntime are omitted when null). The helper lives in src/Build/TelemetryInfra/TasksDetailsTelemetry.cs so it can use System.Text.Json directly (Microsoft.Build only targets net472+net10, both of which already reference it -- no new package dependency).

Testing

Added Telemetry_Tests covering: null/empty data, count properties, ordering by ExecutionsCount desc, custom-name / factory-name hashing, top-100 bound, TaskHostRuntime inclusion, valid JSON shape, null-property omission, JSON-special-char safety, numeric/boolean primitive round-trip, and an end-to-end BuildManager test that verifies the event is emitted with the expected shape.
All GetTasksDetailsProperties_* tests pass on net472 and net10.

Notes

Requires a matching SDK-side change to merge build/tasks/details into the aggregated build/tasks telemetry event.

…emetry The new per-task telemetry path (PR #13609) needed to convert List<TaskDetailInfo> to a JSON string because the CLI telemetry sink (TelemetryEventArgs.Properties) requires IDictionary<string, string?>, unlike the VS path which accepts arbitrary objects via TelemetryComplexProperty. The original code hand-rolled ~95 lines of JSON construction in Framework's TelemetryDataUtils to avoid taking a System.Text.Json dependency on the netstandard2.0 TFM. This change moves the serialization helper out of Framework and into src/Build (Microsoft.Build only targets net472 + net10, both of which already have System.Text.Json available), so we can use JsonSerializer.Serialize directly with no new package references. - New: src/Build/TelemetryInfra/TasksDetailsTelemetry.cs holds GetTasksDetailsProperties + TasksDetailsEventName. - Framework: GetTasksDetails flipped from private to internal so the new helper can reuse it; ~135 LOC of manual JSON deleted. - BuildManager: now references TasksDetailsTelemetry.TasksDetailsEventName. - Tests: 3 new tests lock in the JsonSerializer behavior (null property omission via JsonIgnoreCondition.WhenWritingNull, JSON-special-char safety, and primitive-type round-trip for numeric/boolean fields). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…shared fixture Per review: ProducesCorrectProperties mixed three independent assertions (counts, ordering, custom-name hashing). Split into: - GetTasksDetailsProperties_ReportsTaskCounts - GetTasksDetailsProperties_OrdersTasksByExecutionsCountDescending - GetTasksDetailsProperties_HashesCustomTaskAndFactoryNames Common three-task fixture setup moved into BuildThreeTaskFixtureProperties(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a new CLI/SDK-compatible telemetry event (build/tasks/details) that emits per-task execution details as string properties (including a JSON payload), aligning the non-Activity telemetry path with the richer VS Activity-based telemetry.

Changes:

Emit new build/tasks/details event from BuildManager.EndBuild() when task-details telemetry isn’t excluded.
Add TasksDetailsTelemetry helper to produce TelemetryEventArgs-compatible string properties, including a capped JSON array of task details.
Add unit tests validating the properties/JSON shape and an end-to-end BuildManager emission test.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/Framework/Telemetry/TelemetryDataUtils.cs	Exposes task-details conversion helper to friend assemblies for reuse.
src/Build/TelemetryInfra/TasksDetailsTelemetry.cs	New serializer/helper producing `TaskCount`, `TotalTaskCount`, and JSON `Tasks` properties.
src/Build/Microsoft.Build.csproj	Includes the new telemetry helper in the Build assembly compilation.
src/Build/BackEnd/BuildManager/BuildManager.cs	Emits the new `build/tasks/details` telemetry event at end of build (gated by trait).
src/Build.UnitTests/Telemetry/Telemetry_Tests.cs	Adds coverage for serialization behavior and end-to-end event emission.

JanProvaznik · 2026-05-28T11:57:25Z

is this the same one as VMR? the test fails look real @jankratochvilcz

…ix-pr-13609-ci

Avoid IL2026 and IL3050 in AOT-compatible builds by serializing task details with generated System.Text.Json metadata. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

github-actions · 2026-07-23T07:54:17Z

🔍 Skill Validator Results

⚠️ Warnings or advisories found

Scope	Checked
Skills	6
Agents	1
Total	7
Severity	Count
---	---:
❌ Errors	0
⚠️ Warnings	4
ℹ️ Advisories	0

Summary

Level	Finding
ℹ️	Found 6 skill(s)
ℹ️	[cswin32-com] 📊 cswin32-com: 4,376 BPE tokens [chars/4: 4,230] (standard ~), 14 sections, 6 code blocks
ℹ️	[cswin32-com] ⚠ Skill is 4,376 BPE tokens (chars/4 estimate: 4,230) — approaching "comprehensive" range where gains diminish.
ℹ️	[dotnet-aot-compat] 📊 dotnet-aot-compat: 3,905 BPE tokens [chars/4: 4,192] (standard ~), 25 sections, 9 code blocks
ℹ️	[dotnet-aot-compat] ⚠ Skill is 3,905 BPE tokens (chars/4 estimate: 4,192) — approaching "comprehensive" range where gains diminish.
ℹ️	[merge-dependency-updates] 📊 merge-dependency-updates: 2,052 BPE tokens [chars/4: 1,913] (detailed ✓), 16 sections, 7 code blocks
ℹ️	[cswin32-interop] 📊 cswin32-interop: 3,530 BPE tokens [chars/4: 3,311] (standard ~), 14 sections, 3 code blocks
ℹ️	[cswin32-interop] ⚠ Skill is 3,530 BPE tokens (chars/4 estimate: 3,311) — approaching "comprehensive" range where gains diminish.
ℹ️	[release] 📊 release: 3,668 BPE tokens [chars/4: 3,588] (standard ~), 13 sections, 1 code blocks
ℹ️	[release] ⚠ Skill is 3,668 BPE tokens (chars/4 estimate: 3,588) — approaching "comprehensive" range where gains diminish.

Full validator output

```text Found 6 skill(s) [cswin32-com] 📊 cswin32-com: 4,376 BPE tokens [chars/4: 4,230] (standard ~), 14 sections, 6 code blocks [cswin32-com] ⚠ Skill is 4,376 BPE tokens (chars/4 estimate: 4,230) — approaching "comprehensive" range where gains diminish. [dotnet-aot-compat] 📊 dotnet-aot-compat: 3,905 BPE tokens [chars/4: 4,192] (standard ~), 25 sections, 9 code blocks [dotnet-aot-compat] ⚠ Skill is 3,905 BPE tokens (chars/4 estimate: 4,192) — approaching "comprehensive" range where gains diminish. [merge-dependency-updates] 📊 merge-dependency-updates: 2,052 BPE tokens [chars/4: 1,913] (detailed ✓), 16 sections, 7 code blocks [cswin32-interop] 📊 cswin32-interop: 3,530 BPE tokens [chars/4: 3,311] (standard ~), 14 sections, 3 code blocks [cswin32-interop] ⚠ Skill is 3,530 BPE tokens (chars/4 estimate: 3,311) — approaching "comprehensive" range where gains diminish. [release] 📊 release: 3,668 BPE tokens [chars/4: 3,588] (standard ~), 13 sections, 1 code blocks [release] ⚠ Skill is 3,668 BPE tokens (chars/4 estimate: 3,588) — approaching "comprehensive" range where gains diminish. [running-unit-tests] 📊 running-unit-tests: 2,379 BPE tokens [chars/4: 2,319] (detailed ✓), 10 sections, 6 code blocks ✅ All checks passed (6 skill(s)) Found 1 agent(s) Validated 1 agent(s) ✅ All checks passed (1 agent(s)) ```

Clear MSBUILDTELEMETRYEXCLUDETASKSDETAILS for the event-emission test so CI agent configuration does not suppress the event under test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

Run the BuildManager telemetry test with task details both enabled and excluded while isolating each case from the agent environment. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

The worker node can inherit telemetry exclusion settings that differ from the test host, so this integration test cannot make a reliable assertion in CI. Direct task-details aggregation and serialization tests retain deterministic coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

baronfel

Let's gooooo

Automated flaky-test quarantine (2026-07-27). --- ## Quarantine: `SingleBuild_CoordinatorCapsMaxNodeCount` **Flaky-test key** (automated de-duplication — do not edit): ```text flaky-test-id: Microsoft.Build.Coordinator.UnitTests.CoordinatorIntegration_Tests.SingleBuild_CoordinatorCapsMaxNodeCount ``` Tracked by #14516 - **Action**: quarantine (6a) — added `[ActiveIssue("https://github.com/dotnet/msbuild/issues/14516")]` - **File**: `src/MSBuild.Coordinator.UnitTests/CoordinatorIntegration_Tests.cs` - **Distinct sources**: 3 (3 PRs: #13609, #14240, #14470; 0 rolling builds) - **Legs**: CoreOnLinux, CoreOnWindows, FullOnWindows Release - **TFMs**: net10.0, net11.0, net472 - **Error hashes**: 3 distinct (`4039adb7`, `55b6331b`, `ba217936`) - **First seen**: 2026-07-17 | **Last seen**: 2026-07-23 - [Sample build](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1523327) --- ## Also tracked this run (new issues filed; quarantine eligible next run) - `Microsoft.Build.BuildCheck.UnitTests.EndToEndTests.TFMinNonSdkCheckTest` (3 rolling builds, 4 error hashes) - `Microsoft.Build.Engine.UnitTests.MSBuildServer_Tests.ServerSpawnAndReuseAreLoggedToBuildLog` (3 rolling builds, 3 error hashes) - `Microsoft.Build.UnitTests.CommunicationUtilitiesTests.GetEnvVars` (3 rolling builds, 2 error hashes) --- ## Skipped (possible regression — human triage needed) The following tests showed a **single stable error hash** failing across the same set of 4 rolling builds with no PR sources, which may indicate a regression rather than a flake: - `Microsoft.Build.UnitTests.CodeTaskFactoryTests.*` (7 tests, hash `2f59b830`, builds 1506834/1514373/1517682/1527742) - `Microsoft.Build.UnitTests.CodeTaskFactoryTests.EmbedsGenerated*` (2 tests, hash `d888cdeb`, same builds) These were **not quarantined** to avoid masking a real bug. Human investigation recommended. > [!WARNING] > <details> > <summary>Firewall blocked 2 domains</summary> > > The following domains were blocked by the firewall during workflow execution: > > - `awmgmcpg` > - `southcentralus0.in.applicationinsights.azure.com` >> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter: > > ```yaml > network: > allowed: > - defaults > - "awmgmcpg" > - "southcentralus0.in.applicationinsights.azure.com" > ``` > > See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information. > > </details> > Generated by [Flaky Test Triage](https://github.com/dotnet/msbuild/actions/runs/30267259222) · sonnet46 · 153.8 AIC · ⌖ 6.72 AIC · ⊞ 14.7K · [◷](https://github.com/search?q=repo%3Adotnet%2Fmsbuild+%22gh-aw-workflow-id%3A+flaky-test-detector.agent%22&type=pullrequests)    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Aleš Prokop <ales.prokop.dev@gmail.com>

Draft of event-level telemetry

6f5853e

jankratochvilcz commented Apr 27, 2026

View reviewed changes

Comment thread src/Framework/Telemetry/TelemetryDataUtils.cs Outdated

Comment thread src/Build/BackEnd/BuildManager/BuildManager.cs

Jan Kratochvil and others added 2 commits May 25, 2026 16:52

jankratochvilcz changed the title ~~Draft of event-level telemetry~~ Add event-level task telemetry (uild/tasks/details) May 25, 2026

jankratochvilcz marked this pull request as ready for review May 25, 2026 15:04

Copilot AI review requested due to automatic review settings May 25, 2026 15:04

jankratochvilcz changed the title ~~Add event-level task telemetry (uild/tasks/details)~~ Add event-level task telemetry (build/tasks/details) May 25, 2026

Copilot started reviewing on behalf of jankratochvilcz May 25, 2026 15:04 View session

jankratochvilcz mentioned this pull request May 25, 2026

Forward MSBuild build/tasks/details telemetry event dotnet/sdk#54092

Merged

Copilot AI reviewed May 25, 2026

View reviewed changes

Comment thread src/Build/TelemetryInfra/TasksDetailsTelemetry.cs

Comment thread src/Build/TelemetryInfra/TasksDetailsTelemetry.cs

Comment thread src/Build.UnitTests/Telemetry/Telemetry_Tests.cs

Comment thread src/Build.UnitTests/Telemetry/Telemetry_Tests.cs Outdated

baronfel approved these changes May 26, 2026

View reviewed changes

This was referenced May 27, 2026

Add per-task telemetry event (build/tasks/details) end-to-end [VMR review preview] jankratochvilcz/dotnet#1

Closed

Add per-task telemetry event (build/tasks/details) end-to-end dotnet/dotnet#6861

Open

JanProvaznik and others added 3 commits May 28, 2026 13:57

Merge branch 'main' into jan/telemetry-sdk

c60bcbe

Merge remote-tracking branch 'origin/pr-13609' into jankratochvilcz-f…

e52e46f

…ix-pr-13609-ci

jankratochvilcz requested review from a team as code owners July 23, 2026 07:53

jankratochvilcz added 2 commits July 23, 2026 10:27

jankratochvilcz closed this Jul 23, 2026

jankratochvilcz reopened this Jul 23, 2026

jankratochvilcz added 5 commits July 23, 2026 11:15

Stabilize task telemetry environment test

c33f1cb

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

Clear cached task telemetry exclusion in test

9af9e1b

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

Read task telemetry exclusion dynamically

7293d6e

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

Test task telemetry using process configuration

032cb1b

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 197b045b-3774-41a5-a8b7-7748bcf906c5

baronfel approved these changes Jul 23, 2026

View reviewed changes

This was referenced Jul 25, 2026

[Flaky Test] Microsoft.Build.Coordinator.UnitTests.CoordinatorIntegration_Tests.SingleBuild_CoordinatorCapsMaxNodeCount #14516

Open

[Flaky Test] Quarantine 1 flaky test #14542

Merged

github-actions Bot mentioned this pull request Jul 29, 2026

[Flaky Test] Microsoft.Build.UnitTests.BackEnd.TaskExecutionHost_Tests.TaskExceptionHandlingTest #14459

Open

jankratochvilcz merged commit ca770cf into main Jul 31, 2026
12 checks passed

jankratochvilcz deleted the jan/telemetry-sdk branch July 31, 2026 09:06

dotnet-maestro Bot mentioned this pull request Aug 1, 2026

[main] Source code updates from dotnet/msbuild dotnet/dotnet#8085

Merged

Navigation Menu

Uh oh!

Add event-level task telemetry (build/tasks/details) - #13609

Add event-level task telemetry (build/tasks/details)#13609
jankratochvilcz merged 13 commits into
mainfrom
jan/telemetry-sdk

jankratochvilcz commented Apr 27, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JanProvaznik commented May 28, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

Uh oh!

baronfel left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

jankratochvilcz commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Changes Made

Testing

Notes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JanProvaznik commented May 28, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

🔍 Skill Validator Results

Summary

Uh oh!

baronfel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jankratochvilcz commented Apr 27, 2026 •

edited

Loading