Add auto-merge workflow for docs PRs based on Learn Build statuses#82
Merged
Conversation
- New workflow (automerge-docs.yml) triggers on issue_comment from learn-build-service-prod bot on automation/update-api-docs PRs - Checks PoliCheck (must show 'No issues found') and Build Report (no errors, no new warnings vs baseline) - Uses the JSON build log API for reliable structured warning data - Baseline file (known-warnings.txt) tracks 120 known xref-not-found warnings from external library type references - Fix update-docs.yml to skip force-push when automation branch already has identical content (compares git tree SHAs) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- compare_warnings now uses Counter (multiset) instead of set, so duplicate warnings with the same text are tracked individually. Previously 'Gdk.RGBA' appearing 8 times vs 2 in baseline was reported as 0 new (set dedup), now correctly reports 6 new. - Add automation/write-api-docs to the list of automation branches eligible for auto-merge checking. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Check commit statuses (SUCCESS/ERROR/PENDING) and check runs (COMPLETED+SUCCESS) before proceeding with comment analysis. Blocks merge if any check is not green. Tested against: - PR 70: all green -> passes - PR 75: OpenPublishing.Build ERROR -> blocks - PR 81: all green but new warnings -> blocks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collapse duplicate warnings (120 lines -> 63 unique rows with counts). Format: file,code,message,count — easier to read and edit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace comment-based approach with status-driven design: - Read build report URLs directly from GitHub status targetUrls (OpenPublishing.Build and PoliCheck Scan statuses) - No more parsing PR comments or matching commit SHAs in markdown - Trigger workflow on 'status' event instead of 'issue_comment' - First step finds the PR for the commit SHA, then runs the check The PoliCheck pass/fail is now determined by its status state (SUCCESS vs ERROR), and the build warnings are fetched from the OpenPublishing.Build targetUrl -> build report -> JSON build log. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The workflow now filters by automation/* branch prefix in the 'Find PR' step, so the script never runs for non-automation PRs. Script is now purely about validating statuses and warnings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The status event payload includes: - context: 'OpenPublishing.Build' or 'PoliCheck Scan' - branches[]: array of branch names the commit belongs to - sha: the commit SHA - target_url: direct link to the build report Use branches from the event to: 1. Filter in job 'if' — skip entirely if no automation/ branch 2. Look up PR by branch name (gh pr list --head) instead of SHA search Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The old automation/update-api-docs branch is gone (PR #71). Only the write-api-docs branch exists now. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
contains() on an array checks for exact element membership. The toJSON() wrapper was converting to string first, making it a substring match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes from Opus 4.7 and GPT-5.5 code reviews:
1. Baseline tamper protection: checkout main (not PR branch) so a PR
cannot edit known-warnings.csv to bypass the warning gate.
2. TOCTOU race prevention: pin merge to validated SHA with
--match-head-commit; verify event SHA matches PR HEAD before
proceeding; skip stale status events.
3. Require both statuses: fail closed unless both OpenPublishing.Build
AND PoliCheck Scan are present and green in the rollup.
4. Fail closed on schema changes: assert build_log_error_items exists,
is a list, items are dicts with message_severity. Missing key =
refuse to merge (was silently treating as zero warnings).
5. Shell injection prevention: pass all event data through env vars
(BRANCHES_JSON, EVENT_SHA, PR_NUMBER, REASON, MERGE_SHA) instead
of $\{\{ \}\} interpolation in run: blocks.
6. URL host validation: allowlist buildapi.docs.microsoft.com before
fetching any build report or JSON log URL.
7. Comment on all failures: post PR comment for any failure mode
(not just new warnings), with link to workflow run.
8. CSV baseline validation: verify required columns exist, validate
count values are integers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
DRY_RUN env var defaults to true. When enabled, the workflow runs all checks but posts a comment instead of merging. Also supports workflow_dispatch with a dry_run input toggle. To go live: change DRY_RUN to false in the workflow file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When non-green checks are only PENDING/IN_PROGRESS, exit 0 (wait silently) instead of exit 1 (post blocked comment). Prevents spurious 'auto-merge blocked' comments when the first status arrives before the second. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
Learn Build status updates of commit 69c733c: ✅ Validation status: passed
For more details, please refer to the build report. |
Contributor
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
mattleibow
added a commit
that referenced
this pull request
Jun 30, 2026
Go Live: publish latest API docs (#171) Publish the accumulated SkiaSharp / HarfBuzzSharp API documentation from main to the live (published) branch. This is the first Go Live since #66 and rolls up a large batch of docs automation, a full regeneration, and content/cross-reference cleanup. Highlights in this batch: - Docs automation: automated API docs writer (#92, hardened in #155), daily update workflow, go-live workflow (#65), and a Learn Build status-based auto-merge gate (#82, #83) backed by check-learn-build.py and a .github/known-warnings.csv baseline. - Regeneration: frameworks docs switched to latest-only monikers (#141), baseline reset to the Windows-generated output (#142), and stub regeneration moved to Linux via Mono (#147). This drops older, no-longer-shipping member/type pages (e.g. pre-v1.68 view APIs, the removed Android ISKRenderer interface, and the SKPaint text properties that moved to SKFont). - Content: filled API documentation placeholders (#150, #151, #172) and fixed broken cross-references (#152). - Cross-reference cleanup (#173): removed/repointed 36 obsolete xref-not-found references left behind by the latest-only-moniker regeneration — dangling links in 2019-era remarks to members that were since removed or relocated. Without this the Go Live build reported 160 warnings and the auto-merge gate (correctly) blocked the publish. Build health at publish: 0 errors, 124 warnings, 0 suggestions. All 124 remaining warnings are expected xref-not-found references to external framework types (OpenTK, Gdk/Cairo/Graphene, ElmSharp/Tizen.NUI, Windows.UI.Xaml, Microsoft.UI.Xaml, SharpVk, Vortice) that Learn cannot resolve and which are tracked in the known-warnings.csv baseline — 0 new warnings versus baseline, so the gate passes. Co-authored-by: Matthew Leibowitz <mattleibow@live.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a GitHub Actions workflow that automatically validates and merges docs PRs from the
automation/write-api-docsbranch when Learn Build bot statuses are green and no new warnings appear.Currently in dry-run mode — comments what it would do instead of merging. Set
DRY_RUN: falsein the workflow to go live.How it works
statusevent whenOpenPublishing.BuildorPoliCheck Scanreports onautomation/write-api-docscheck-learn-build.pywhich:OpenPublishing.BuildandPoliCheck Scanto be presentknown-warnings.csvbaseline (120 knownxref-not-foundwarnings)Security hardening
Reviewed by Opus 4.7 and GPT-5.5, with all findings fixed:
mainfor the baseline, not the PR branch--match-head-commitpins merge to the validated SHAbuildapi.docs.microsoft.comFiles
.github/workflows/automerge-docs.yml.github/scripts/check-learn-build.py.github/known-warnings.csv