From d13494481ebc943521fab7314809b996483cf8fd Mon Sep 17 00:00:00 2001 From: Milos Kotlar Date: Mon, 1 Jun 2026 12:02:43 +0200 Subject: [PATCH 1/2] [ci-scan] Treat maintainer-closed test-disable PRs as do-not-disable signal Extends Step 4.7 to also search for recent closed-not-merged [ci-scan] test-disable PRs targeting the same test or KBE. A MEMBER/OWNER close of a prior test-disable PR is itself a do-not-disable signal, so the scanner does not re-file the same disable after a maintainer rejects it. Motivated by PR #128793, which re-filed the stackoverflowtester interpreter-mode disable that BrzVlad had closed in PR #128737 with a pushback to investigate the assert rather than mute the test. Neither the prior fix in PR #128760 (which added referenced-issue comment scanning) nor the existing open-PR dedup in Step 4.4 caught it: the do-not-disable signal lived on the closed PR, not on KBE #127899 itself. No 'gh aw compile' needed; the scanner loads this file at runtime via '{{#runtime-import}}'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/ci-failure-scan.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/ci-failure-scan.md b/.github/workflows/ci-failure-scan.md index 4544e9fb38a06d..a67ff6b096f95a 100644 --- a/.github/workflows/ci-failure-scan.md +++ b/.github/workflows/ci-failure-scan.md @@ -264,6 +264,7 @@ Read the candidate KBE / tracker body + the latest 5 comments (not just the most - Body or recent comment from any `MEMBER`/`OWNER` mentions one of (case-insensitive): `please don't disable`, `do not mute`, `do not disable`, `keep failing`, `investigation in progress`, `fix-forward`, `fix forward`, `should be supported`, `will investigate`, `wait for #`, `landing in #`. - Issue carries a label semantically equivalent to "do not mute" (verify the label exists in `dotnet/runtime` before relying on it; do not invent labels). - Most recent area-owner comment within the last 14 days opposes disabling on procedural grounds. +- A prior `[ci-scan]` test-disable PR for the same test (or same KBE `#`) was **closed without merge** by a `MEMBER`/`OWNER` in the last 30 days. Search `is:pr is:closed -is:merged "" "[ci-scan]"` and `is:pr is:closed -is:merged "#" "[ci-scan]"`; for each hit, fetch the PR and check (a) the `closed` event actor's association and (b) any `MEMBER`/`OWNER` comment for the keywords above. A maintainer close of a prior test-disable PR is itself a do-not-disable signal; re-filing requires fresh evidence such as a new maintainer comment on the KBE greenlighting the disable, or a clearly different failure signature. Record `-> skipped: do-not-disable, prior PR # closed by @`. When in doubt -> skip the test-disable and let the next run revisit. From d7e52da4a11a3b8c5266d378af64151c40f82748 Mon Sep 17 00:00:00 2001 From: Milos Kotlar Date: Mon, 1 Jun 2026 12:08:15 +0200 Subject: [PATCH 2/2] Address review: scope query to last 30 days and use comment association The previous wording lacked a closed:>= filter and referenced the closed event actor's association, which is not directly available without a separate events API call. Scope the search with closed:>=<30-days-ago> and rely on the PR comments' authorAssociation field (already returned by the comments API) to identify maintainer pushback. The combination of a maintainer pushback comment plus a non-merge close is the do-not-disable signal; the closer does not need to be the same person. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/ci-failure-scan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci-failure-scan.md b/.github/workflows/ci-failure-scan.md index a67ff6b096f95a..c9820ddb7ab4ce 100644 --- a/.github/workflows/ci-failure-scan.md +++ b/.github/workflows/ci-failure-scan.md @@ -264,7 +264,7 @@ Read the candidate KBE / tracker body + the latest 5 comments (not just the most - Body or recent comment from any `MEMBER`/`OWNER` mentions one of (case-insensitive): `please don't disable`, `do not mute`, `do not disable`, `keep failing`, `investigation in progress`, `fix-forward`, `fix forward`, `should be supported`, `will investigate`, `wait for #`, `landing in #`. - Issue carries a label semantically equivalent to "do not mute" (verify the label exists in `dotnet/runtime` before relying on it; do not invent labels). - Most recent area-owner comment within the last 14 days opposes disabling on procedural grounds. -- A prior `[ci-scan]` test-disable PR for the same test (or same KBE `#`) was **closed without merge** by a `MEMBER`/`OWNER` in the last 30 days. Search `is:pr is:closed -is:merged "" "[ci-scan]"` and `is:pr is:closed -is:merged "#" "[ci-scan]"`; for each hit, fetch the PR and check (a) the `closed` event actor's association and (b) any `MEMBER`/`OWNER` comment for the keywords above. A maintainer close of a prior test-disable PR is itself a do-not-disable signal; re-filing requires fresh evidence such as a new maintainer comment on the KBE greenlighting the disable, or a clearly different failure signature. Record `-> skipped: do-not-disable, prior PR # closed by @`. +- A prior `[ci-scan]` test-disable PR for the same test (or same KBE `#`) was **closed without merge** within the last 30 days. Search `is:pr is:closed -is:merged "" "[ci-scan]" closed:>=<30-days-ago>` and `is:pr is:closed -is:merged "#" "[ci-scan]" closed:>=<30-days-ago>` (compute `<30-days-ago>` as the ISO date 30 days before the scan run). For each hit, fetch the PR comments and skip if any commenter with `authorAssociation` `MEMBER` or `OWNER` used any of the keywords listed above. The combination of a maintainer pushback comment plus a non-merge close is the do-not-disable signal; the closer does not need to be the same maintainer. Re-filing requires fresh evidence such as a new maintainer comment on the KBE greenlighting the disable, or a clearly different failure signature. Record `-> skipped: do-not-disable, prior PR # closed without merge after maintainer pushback`. When in doubt -> skip the test-disable and let the next run revisit.