Skip to content

fix(smoke-codex): eliminate safe_outputs instability on schedule runs and wrong-PR comments#22039

Merged
pelikhan merged 2 commits into
mainfrom
copilot/fix-safe-outputs-instability
Mar 20, 2026
Merged

fix(smoke-codex): eliminate safe_outputs instability on schedule runs and wrong-PR comments#22039
pelikhan merged 2 commits into
mainfrom
copilot/fix-safe-outputs-instability

Conversation

Copilot AI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Schedule-triggered runs had no PR context, causing safe_outputs to fail when the agent attempted add_comment. Separately, PR-triggered runs were failing because the agent searched GitHub for a PR number instead of using the one from context, sometimes landing on the wrong PR.

Changes

.github/workflows/smoke-codex.md — Output section rewritten:

  • Event branching: prompt now gates behavior on ${{ github.event_name }}
    • pull_requestadd_comment + label/unassign actions
    • schedule / workflow_dispatchnoop with results summary (no PR exists)
  • Explicit PR number: all safe-output tools now receive item_number: ${{ github.event.pull_request.number }} directly, preventing the agent from searching GitHub and resolving to the wrong PR
- **If `${{ github.event_name }}` is `pull_request`**: Use `add_comment` with
  `item_number: ${{ github.event.pull_request.number }}` (use this exact number —
  do NOT search GitHub for a PR)

- **If `${{ github.event_name }}` is `schedule` or `workflow_dispatch`**: Call
  `noop` with a brief summary of test results (no pull request to comment on)

.github/workflows/smoke-codex.lock.yml — recompiled from updated markdown.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE x_amd64/link git rev-�� --show-toplevel x_amd64/link /usr/bin/git -json .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw --ignore-path ../../../.prettiif ! cd actions/setup/js && npm run check:pkg-json 2>&1 | grep -q "All matched files use Prettier code style"; then \ echo "JSON files are not formatted. Run 'make fmt-json' to fix."; \ exit 1; \ fi sh -c "prettier" --wriGOSUMDB git 64/bin/go rror /opt/hostedtoolc-c /usr/bin/git go (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw --ignore-path ../../../.prettilint:cjs sh -c "prettier" --wriGOSUMDB git 64/bin/go --show-toplevel /opt/hostedtoolc-c /usr/bin/git go (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name k/gh-aw/gh-aw/cm-errorsas k/gh-aw/gh-aw/cm-ifaceassert 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc-buildtags -o /tmp/go-build348-errorsas -trimpath 64/bin/go -p github.com/pmeza-atomic -lang=go1.16 go (http block)
  • https://api.github.com/repos/actions-ecosystem/action-add-labels/contents/action.yaml
    • Triggering command: /usr/bin/gh gh api /repos/actions-ecosystem/action-add-labels/contents/action.yaml?ref=18f1af5e3544586314bbe15c0273249c770b2daf --jq .content (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions-ecosystem/action-add-labels/contents/action.yaml?ref=18f1af5e3544586314bbe15c0273249c770b2daf --jq .content .js' --ignore-path .prettierignore GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env ithub/workflows GO111MODULE cal/bin/sh GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions-ecosystem/action-add-labels/contents/action.yml
    • Triggering command: /usr/bin/gh gh api /repos/actions-ecosystem/action-add-labels/contents/action.yml?ref=18f1af5e3544586314bbe15c0273249c770b2daf --jq .content (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions-ecosystem/action-add-labels/contents/action.yml?ref=18f1af5e3544586314bbe15c0273249c770b2daf --jq .content .js' --ignore-pa-f GO111MODULE ache/go/1.25.0/x-f GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE k/_temp/uv-pytho-f GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --git-dir x_amd64/vet /usr/bin/git -json GO111MODULE x_amd64/vet git rev-�� --git-dir x_amd64/vet /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha sistency_GoAndJavaScript4234454424/001/test-simple-frontmatter.md -buildtags ache/node/24.14.0/x64/bin/node -errorsas -ifaceassert -nilfunc /tmp/go-build1310047830/b406/console.test t-41�� bility_SameInputSameOutput1113185681/001/stability-test.md -test.v=true /usr/bin/git -test.timeout=10git -test.run=^Test -test.short=true--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env -json GO111MODULE ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/x64/pkg/tool/linu.github/workflows/test.md (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel x_amd64/vet /usr/bin/git -json GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git -json .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git Onlymin-integritgit 0047830/b136/vetrev-parse ceutil.test git rev-�� --show-toplevel ceutil.test /usr/bin/git se 0047830/b279/vetrev-parse 64/pkg/tool/linu--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha user.email test@example.com /usr/bin/git -json GO111MODULE x_amd64/vet git bran�� --show-current x_amd64/vet /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha add origin /usr/bin/git -json GO111MODULE x_amd64/vet git push�� -u origin /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git se 0047830/b047/vetrev-parse .cfg git rev-�� --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git -json GO111MODULE 0/x64/bin/node git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha "prettier" --cheGOSUMDB GOPROXY 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc-buildtags -V=f�� (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha --check scripts/**/*.js 64/bin/go .prettierignore (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GOMOD 64/bin/go tierignore (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha k/gh-aw/gh-aw/.github/workflows/bot-detection.md -trimpath /usr/bin/git -p main -lang=go1.25 git add test.txt -dwarf=false /usr/bin/git go1.25.0 -c=4 nch,headSha,disp--show-toplevel git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE x_amd64/vet git conf�� user.name Test User /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq .object.sha -json GO111MODULE $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE go env ithub/workflows GO111MODULE repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq .object.sha ithub/workflows GO111MODULE $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/uv/0.10.12/x86_64/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha -unreachable=false /tmp/go-build1310047830/b049/vet.cfg 0047830/b369/vet.cfg GOSUMDB GOWORK 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet -uns�� k/gh-aw/gh-aw/.github/workflows /tmp/go-build1310047830/b225/vet.cfg /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet 2595903/b428/_pkgit GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha ../../../**/*.jsGOSUMDB !../../../pkg/woGOWORK 64/bin/go ../../../.prettiprettier ortcfg /usr/bin/git go env h ../../../.prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .visibility .js' --ignore-pa-f GO111MODULE ache/go/1.25.0/x-f GOINSECURE GOMOD GOMODCACHE go env ithub/workflows GO111MODULE k/gh-aw/node_mod-f GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha 5436-27975/test-2727207212 /tmp/go-build1310047830/b094/vet.cfg 0047830/b372/vet.cfg 16cd4a5c5f3f40b6git GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet -uns�� -unreachable=false /tmp/go-build1310047830/b243/vet.cfg /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet 2595903/b419/_pkgit GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha jpq_JbW5I /tmp/go-build1310047830/b055/vet.cfg 0047830/b365/vet.cfg f98c39224891bbadgit GO111MODULE 64/bin/go ortcfg -uns�� g/timeutil/format.go g/timeutil/format_test.go ache/go/1.25.0/x64/pkg/tool/linux_amd64/link -json GO111MODULE 64/bin/go ache/go/1.25.0/x64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 .go 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile env g_.a GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE ole GOMODCACHE 64/pkg/tool/linutest@example.com (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE EC/vKfaaJsUtRlxju1NbgiD/uvljh3C4vdN3eqAa1K0H (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path k/gh-aw/gh-aw/pkGOSUMDB k/gh-aw/gh-aw/pkGOWORK 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc/tmp/go-build1310047830/b213/vet.cfg -o /tmp/go-build3482595903/b392/_pkGOINSECURE -trimpath 64/bin/go -p main -lang=go1.25 go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 github.com/githu-atomic -lang=go1.25 go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/vet env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha h ../../../.prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha -json GO111MODULE h GOINSECURE GOMOD GOMODCACHE go env '**/*.ts' '**/*.json' --ignore-p-f GO111MODULE r: $owner, name: $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE sb/WrsaZCqRpvSTiHMe1VZH/ZiMelo4K-4Ses7ApefUz env -json GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc/tmp/go-build1310047830/b215/vet.cfg -o /tmp/go-build3482595903/b427/_pkGOINSECURE -trimpath 64/bin/go -p github.com/githu-atomic -lang=go1.25 go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go **/*.json --ignore-path ../../../.pretti-unreachable=false /opt/hostedtoolc/tmp/go-build1310047830/b221/vet.cfg -o /tmp/go-build3482595903/b431/_pkGOINSECURE -trimpath 64/bin/go -p github.com/githu-atomic -lang=go1.25 go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build1310047830/b400/cli.test /tmp/go-build1310047830/b400/cli.test -test.testlogfile=/tmp/go-build1310047830/b400/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOSUMDB GOWORK 64/bin/go node /hom�� --check scripts/**/*.js 64/bin/go .prettierignore (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name tions-lock.json -errorsas pkg/mod/github.c-ifaceassert 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc-buildtags -o /tmp/go-build348-errorsas -trimpath 64/bin/go -p main -lang=go1.25 go (http block)

If you need me to access, download, or install something from one of these locations, you can either:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…safe_outputs instability

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/2d920ec1-e74c-42f9-afa0-eedb260ee103
Copilot AI changed the title [WIP] Fix safe_outputs instability in smoke codex schedule runs fix(smoke-codex): eliminate safe_outputs instability on schedule runs and wrong-PR comments Mar 20, 2026
Copilot AI requested a review from pelikhan March 20, 2026 23:07
@pelikhan pelikhan marked this pull request as ready for review March 20, 2026 23:08
Copilot AI review requested due to automatic review settings March 20, 2026 23:08
@pelikhan pelikhan merged commit 58acd3e into main Mar 20, 2026
103 checks passed
@pelikhan pelikhan deleted the copilot/fix-safe-outputs-instability branch March 20, 2026 23:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the smoke Codex workflow prompt and compiled lock workflow to prevent safe-outputs failures on non-PR runs and to avoid commenting on the wrong PR by using the PR number from context.

Changes:

  • Updates the smoke-codex prompt output instructions to branch behavior based on the triggering event type and to use an explicit PR number.
  • Recompiles the workflow lock file and adds GH_AW_GITHUB_EVENT_NAME to the environment passed into prompt rendering / placeholder substitution.
  • Modifies the custom safe-output action (add_smoked_label) schema and its invocation wiring in the compiled workflow.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
.github/workflows/smoke-codex.md Updates output instructions to branch on event type and use explicit PR number.
.github/workflows/smoke-codex.lock.yml Recompiled workflow; adds event-name env var and changes custom action schema/invocation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

with:
labels: ${{ fromJSON(steps.process_safe_outputs.outputs.action_add_smoked_label_payload).labels }}
number: ${{ fromJSON(steps.process_safe_outputs.outputs.action_add_smoked_label_payload).number }}
payload: ${{ steps.process_safe_outputs.outputs.action_add_smoked_label_payload }}

Copilot AI Mar 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step switched from passing labels/number to passing a single payload input into actions-ecosystem/action-add-labels@.... The rest of the workflow still produces action_add_smoked_label_payload as a JSON string of inputs, so passing it as payload will not match the action’s expected inputs and will likely break adding the smoked label. Restore the previous mapping (labels and number sourced from fromJSON(...)) or change the action reference to one that actually accepts a payload input and update the tool schema/prompt accordingly.

Suggested change
payload: ${{ steps.process_safe_outputs.outputs.action_add_smoked_label_payload }}
labels: ${{ fromJSON(steps.process_safe_outputs.outputs.action_add_smoked_label_payload).labels }}
number: ${{ fromJSON(steps.process_safe_outputs.outputs.action_add_smoked_label_payload).number }}

Copilot uses AI. Check for mistakes.
Comment on lines +99 to +106
**Check `${{ github.event_name }}` to determine the correct output action:**

If all tests pass:
- Use the `add_labels` safe-output tool to add the label `smoke-codex` to the pull request
- Use the `remove_labels` safe-output tool to remove the label `smoke` from the pull request
- Use the `unassign_from_user` safe-output tool to unassign the user `githubactionagent` from the pull request (this is a fictitious user used for testing)
- **If `${{ github.event_name }}` is `pull_request`**: Use the `add_comment` safe-output tool to add a **very brief** comment (max 5-10 lines) to the triggering pull request, specifying `item_number: ${{ github.event.pull_request.number }}` (use this exact number — do NOT search GitHub for a PR):
- PR titles only (no descriptions)
- ✅ or ❌ for each test result
- Overall status: PASS or FAIL

- **If `${{ github.event_name }}` is `schedule` or `workflow_dispatch`**: Call the `noop` safe-output tool with a brief summary of the test results (there is no pull request to comment on for scheduled or manually triggered runs).

Copilot AI Mar 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

github.event_name is not currently evaluated/interpolated into the prompt (the runtime expression evaluator only exposes a limited github.* context, and event_name isn’t included). As written, the agent will likely see the literal string ${{ github.event_name }} and won’t be able to reliably branch between pull_request vs schedule/workflow_dispatch. Consider switching these checks to an expression that is resolvable (e.g., ${{ env.GH_AW_GITHUB_EVENT_NAME }} now that the lock workflow exports it), or include the event name in the rendered <github-context> section and instruct the agent to branch on that value.

See below for a potential fix:

**Check `${{ env.GH_AW_GITHUB_EVENT_NAME }}` (the GitHub event name) to determine the correct output action:**

- **If `${{ env.GH_AW_GITHUB_EVENT_NAME }}` is `pull_request`**: Use the `add_comment` safe-output tool to add a **very brief** comment (max 5-10 lines) to the triggering pull request, specifying `item_number: ${{ github.event.pull_request.number }}` (use this exact number — do NOT search GitHub for a PR):
  - PR titles only (no descriptions)
  - ✅ or ❌ for each test result
  - Overall status: PASS or FAIL

- **If `${{ env.GH_AW_GITHUB_EVENT_NAME }}` is `schedule` or `workflow_dispatch`**: Call the `noop` safe-output tool with a brief summary of the test results (there is no pull request to comment on for scheduled or manually triggered runs).

If all tests pass and `${{ env.GH_AW_GITHUB_EVENT_NAME }}` is `pull_request`:

Copilot uses AI. Check for mistakes.
- Use the `add_smoked_label` safe-output action tool to add the label `smoked` to the pull request (call it with `{"labels": "smoked", "number": "${{ github.event.pull_request.number }}"}`)

**Important**: If no action is needed after completing your analysis, you **MUST** call the `noop` safe-output tool with a brief explanation. Failing to call any safe-output tool is the most common cause of safe-output workflow failures.
**Important**: You **MUST** always call exactly one safe-output tool. Failing to call any safe-output tool is the most common cause of safe-output workflow failures.

Copilot AI Mar 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Output section asks for multiple safe-output actions on success (add_comment + add_labels/remove_labels/unassign/add_smoked_label), but then states: “You MUST always call exactly one safe-output tool.” This is contradictory and will cause the agent to choose only one action (or skip label cleanup) to satisfy the “exactly one” rule. Align this with the established safe-outputs convention in other workflows (e.g., smoke-copilot.md:174) by requiring at least one safe-output tool call, and using noop only when no other actions are needed.

Suggested change
**Important**: You **MUST** always call exactly one safe-output tool. Failing to call any safe-output tool is the most common cause of safe-output workflow failures.
**Important**: You **MUST** always call at least one safe-output tool, and you may call multiple safe-output tools in a single response when instructed (for example, `add_comment` plus label updates). Use `noop` only when no other safe-output actions are needed. Failing to call any safe-output tool is the most common cause of safe-output workflow failures.

Copilot uses AI. Check for mistakes.
Comment on lines 456 to 466
"description": "Add the 'smoked' label to the current pull request (can only be called once)",
"inputSchema": {
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"labels": {
"description": "The labels' name to be added. Must be separated with line breaks if there're multiple labels.",
"type": "string"
},
"number": {
"description": "The number of the issue or pull request.",
"payload": {
"description": "JSON-encoded payload to pass to the action",
"type": "string"
}
},
"required": [
"labels"
],
"type": "object"
},

Copilot AI Mar 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated dynamic safe-output tool schema for add_smoked_label now only accepts a string payload and sets additionalProperties: true. This no longer matches the prompt instruction in smoke-codex.md (which tells the agent to call add_smoked_label with labels and number fields), and it also diverges from the custom action handler’s documented behavior (export per-input fields that are later mapped to action with: inputs). Regenerate or adjust this schema to explicitly model the expected inputs (e.g., labels, number) and keep additionalProperties: false so invalid fields are rejected.

See below for a potential fix:

                  "additionalProperties": false,
                  "properties": {
                    "labels": {
                      "description": "Labels to add to the current pull request",
                      "type": "array",
                      "items": {
                        "type": "string"
                      }
                    },
                    "number": {
                      "description": "Pull request number to which the labels should be added",
                      "type": "integer"
                    }
                  },
                  "required": [
                    "labels",
                    "number"
                  ],

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[q] fix(smoke-codex): handle schedule runs and explicit PR number to fix safe_outputs instability (closes #22029)

3 participants