[safe-output-health] Safe Output Health Monitor — 2026-07-31 Daily Audit (99.29% success, 1 new pattern)

[github-actions[bot]]

Executive Summary

Audit window: 2026-07-30 ~20:10 UTC → 2026-07-31 ~05:00 UTC (last 24h). Of 144 total workflow runs, 140 carried a safe_outputs job; 139/140 (99.29%) succeeded. 1 in-scope hard failure at the Process Safe Outputs step: §30577084701 (Smoke Copilot, workflow_dispatch dispatched from PR #49116). No agent-job or detection-job failures are included in this report — those are out of scope for this monitor.

This is a healthy day overall: none of the five standing recurring clusters (review-comment 422, Changeset Generator bundle-transport, PR Sous Chef scheduled-target-resolution, assign_to_agent guess-unresolved, target-star fallback family) recurred in this window. One new pattern was identified and logged.

Job / Item Statistics

All safe-output types (create_issue, create_discussion, add_comment, create_pull_request, push_to_pull_request_branch, etc.) are actuated inside a single consolidated GitHub Actions job named safe_outputs, with the actuation logic in one step, Process Safe Outputs (followed by Upload Safe Outputs Items). There are no longer separate per-type jobs in the compiled workflows.

Metric	Value
Total runs (24h)	144
Runs with a safe_outputs job	140
safe_outputs job succeeded	139
safe_outputs job hard-failed (in scope)	1
Success rate	99.29%
Prior day (07-30) success rate	93.75% (1/16 failed)

Error Cluster: NEW — Smoke Copilot zero-item Process Safe Outputs hard-fail

Run: §30577084701 — Smoke Copilot, workflow_dispatch, branch copilot/review-install-copilot-cli-script, dispatched from PR #49116.

• Process Safe Outputs step: failure (90s). Upload Safe Outputs Items still ran afterward and succeeded — partial-failure isolation is working as designed.
• Agent produced zero safe-output items (created_items=0, safe_items=0); actuation posture flipped from this workflow's baseline write_capable to read_only. The agenticworkflows audit MCP tool classified the run "risky" (reason_codes: posture_changed, run_unsuccessful).
• Root cause: unconfirmed. Re-querying the audit tool directly against this run returned only a generic "Workflow Failed" / error_count=1 finding — no step-level stdout/stderr. This is the same recurring observability gap documented in at least 8 prior audits (06-23, 06-26, 07-11, 07-20, 07-23, 07-30, now 07-31).

Root cause analysis and classification reasoning

This failure is structurally similar to the 2026-07-11 firewall_report_process_safe_outputs_hardfail_create_discussion_not_wired pattern (zero items + job hard-fail), but occurs on a different workflow and trigger (a workflow_dispatch smoke test against an in-review PR's own branch, not a scheduled production job). It was logged as a distinct new pattern rather than merged into that entry.

Working hypothesis: PR #49116 modifies the Copilot CLI install script that Smoke Copilot exists to exercise. If the script change altered engine setup/output behavior on that branch, the agent may have produced no parseable output, and Process Safe Outputs choked on malformed/empty input rather than cleanly no-op'ing (as the other 139 runs did without error). This cannot be confirmed without the step's raw output. Production impact: false — this is a smoke test against a PR under review, not scheduled automation, so it does not affect production discussion/issue/PR creation.

Root Cause Analysis (cross-cutting)

The single most-repeated, highest-leverage finding across 8+ consecutive audits remains: the Process Safe Outputs step's raw stdout/stderr is not recoverable from either the pre-bundled run_summary.json log artifacts or the agenticworkflows audit MCP tool — both surface only a generic error count/finding, never the actual thrown error. This has blocked exact confirmation of essentially every in-scope safe_outputs job hard-failure this monitor has ever flagged, including today's.

Recommendations

Critical / Observability (top priority, unaddressed for 8+ audits):

• Pre-bundle the Process Safe Outputs step's stdout/stderr into the downloadable log artifact on failure (or expose it via the audit MCP tool), so future hard-failures can be root-caused directly instead of inferred from item-count/workflow shape.

Bug fixes ready to ship (highest-confidence, still open):

1. pr_review_buffer.cjs:554 — one-line predicate fix: broaden errorMessage.includes("Line could not be resolved") to also match "Path could not be resolved", i.e. (errorMessage.includes("Line could not be resolved") || errorMessage.includes("Path could not be resolved")) && comments.length > 0. Fixes review_path_unresolved_422's Path variant — 6 occurrences since 2026-05-21 (most recently 07-20, 07-23), fix unvalidated for 58+ consecutive audits because the Path variant hasn't recurred since the fix was scoped, but the Line-only fix won't catch it when it does.
2. Safe-outputs bridge process "dubious ownership" git error on create_pull_request / push_to_pull_request_branch with patch-format:bundle — add -c safe.directory=/home/runner/work/gh-aw/gh-aw (or '*') to the bridge's git invocation. Affects the Changeset Generator production release-notes workflow (2 confirmed hard-fails: 06-17, 06-23/06-26), absent/unexercised for ~35 days but unvalidated.

Process improvements:

• Flag PR fix: enforce three-level version priority for Copilot CLI install #49116 to its author/reviewers: a Smoke Copilot failure while testing their branch's install-script change could be an early regression signal in that PR, independent of any gh-aw platform bug.

Work Item Plans

#	Item	Type	Effort	Priority
1	Pre-bundle Process Safe Outputs step log on failure	Observability	Small–Medium	Critical
2	Fix pr_review_buffer.cjs:554 Path-variant predicate + add regression test	Bug fix	XS	High
3	Add safe.directory to safeoutputs bridge git invocation (bundle transport)	Bug fix	Small	High
4	Unify soft-skip-vs-hard-fail handling for missing trigger context across add_labels/remove_labels/update_issue/add_comment vs. create_pull_request_review_comment	Refactor	Medium	Medium
5	Investigate Smoke Copilot zero-item hard-fail root cause once step logs are available	Follow-up	Small	Medium

Historical Context / Trends

• 07-30: 93.75% (1/16 failed, assign_to_agent probable recurrence, unconfirmed).
• 07-29: 100% (19/19).
• 07-24: 97.44% (76/78, PR Sous Chef scheduled-target-resolution double failure).
• 07-23: 99.24% (262/264, review-comment + PR Sous Chef failures).
• 07-31 (today): 99.29% (139/140) — recovers from yesterday's dip, no standing cluster recurred, one new isolated pattern logged.

The Process Safe Outputs step-log observability gap has now blocked root-cause confirmation in every single one of the last 8 audits with an in-scope hard failure (06-23, 06-26, 07-11, 07-20, 07-23, 07-30, 07-31) — it is the single most consequential unaddressed item in this monitor's history.

Metrics / KPIs

• Safe-outputs job success rate (24h): 99.29% (139/140)
• In-scope hard failures: 1
• New pattern clusters identified: 1
• Recurring clusters that reproduced this window: 0
• Standing unvalidated code fixes: 2 (pr_review_buffer.cjs:554, bridge safe.directory)

Next Steps

• Ship the pr_review_buffer.cjs:554 Path-variant predicate fix + regression test.
• Add safe.directory to the safeoutputs bridge's git invocation for bundle-transport pushes.
• Implement step-log pre-bundling for Process Safe Outputs failures (unblocks root-causing this and all prior unconfirmed clusters).
• Continue monitoring for recurrence of today's new Smoke Copilot zero-item pattern on other workflow_dispatch smoke runs.
• Note PR fix: enforce three-level version priority for Copilot CLI install #49116 to its author as a heads-up given the smoke-test failure on that branch.

References:

• §30577084701 — Smoke Copilot, today's sole in-scope failure
• §30511498181 — LintMonster, 07-30 probable assign_to_agent recurrence
• §29946005059 — PR Code Quality Reviewer, 07-23 review-comment 422 hard-fail

Generated by 🔒 Safe Output Health Monitor · agent · 234.2 AIC · ⌖ 17.1 AIC · ⊞ 9.7K · ◷

• expires on Jul 31, 2026, 9:45 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-01T05:45:36.254Z.

Closed by Workflow

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-01T05:45:36.254Z.

Closed by Workflow