[audit-workflows] 🩺 Agentic Workflow Audit — 2026-07-20 (24h): HEALTHY 93.1%

[github-actions[bot]]

🩺 Agentic Workflow Audit — 2026-07-20 (24h)

Verdict: HEALTHY on baseline. Clean 24-hour window of 204 runs → 190 success / 14 failure = 93.1%. Every failure was a 0-turn / driver-exit job failure; zero agent-logic failures, zero missing-tools, zero missing-data, zero MCP failures. pi (29/29) and codex (8/8) ran 100% clean. The long-standing token-reporting gap is resolved — 11.03M tokens were captured this window.

Key metrics

Metric	Value
Runs (24h)	204
Success rate	93.1% (190S / 14F)
Prod-main rate	92.7% (114/123) · 94.2% ex-2 intentional credit tests
Non-main rate	93.8% (76/81)
Tokens	11,034,930
AI credits	~9,763
Action minutes	2,064
Missing tools / data / MCP fails	0 / 0 / 0

Per-engine: pi 29/29 (100%) · codex 8/8 (100%) · gemini 1/1 · antigravity 1/1 · copilot 120/129 (93%) · claude 31/35 (89%).

Trend charts

Workflow health (30d) — success rate holds in the high-80s/low-90s band; today's 93.1% sits at the top of the recent range with no red spike. The failure count (14) is proportionally the lowest since early July.

Token usage (30d) — daily tokens recovered to 11.03M after the mid-July reporting gap that showed nulls; the 7-day moving average is trending back up as real metrics land again. Heaviest consumers are the smoke/AOAI validation suite and codex analyzers rather than production report agents.

Failure clustering — all 14 failures

Every failure was 0-turn / driver-exit (the CLI wrapper or a pre/post-agent infra step exiting non-zero). Notably, 3 failures had tokens > 0, meaning the agent completed its work and the driver step still reddened the run.

Cluster	Count	Signature	Maps to known issue
copilot Execute-CLI 0-turn	7	agent/Execute GitHub Copilot CLI	copilot-sdk-driver-failures (rec→25)
claude Execute-CLI 0-turn	3	agent/Execute Claude Code CLI	claude-agent-job-fail-longrun
Intentional credit-guardrail tests	2	activation/token guardrail + credit-CLI	by design
MCP Gateway start (novel)	1	agent/Start MCP Gateway	new anomaly
Install AWF binary (novel)	1	detection/Install AWF binary	new anomaly
Safe Outputs (dev-branch)	1	safe_outputs/Process Safe Outputs	safe-output-partial-failure-intolerance

Detail:

• copilot (7): CI Optimization Coach, Daily Formal Spec Verifier (tok=14.1k), Daily Issues Report Generator, Daily Max Ai Credits Test*, Smoke Copilot ×2 (dev branches), Tidy.
• claude (3): Smoke Claude ×2 (copilot/update-cli-versions), Workflow Portfolio Analyst (tok=29.5k — agent ran, driver reddened).
• Intentional (2): Daily Credit Limit Test*, Daily Max Ai Credits Test* — designed to trip guardrails; depress prod-main ~1.5pts.
• Novel transients (2): MCP Inspector Agent failed at Start MCP Gateway (tok=0); Daily Code Metrics failed at post-agent Install AWF binary (tok=31.6k, agent completed). No firewall block in either — treated as one-off infra transients on watch, not new persistent issues.
• Safe Outputs (1): Smoke Copilot on copilot/fix-safeoutputs-backend-issue (tok=964k) — expected churn on the branch actively fixing that backend.

Firewall & positive signals

• Smoke-CI-on-push chronic 100%-fail cluster NOT observed this window (only Tidy failed on push) — a notable improvement vs. prior audits.
• Aggregate firewall blocks were benign/expected: proxy.golang.org (133), awmg-mcpg:8080 (41, the bridge), pi.dev (31), localhost:8080 (23). None correlated with the two novel infra failures (both had empty block lists).
• Token-usage reporting: run_summary.TokenUsage populated across 190 runs — the mid-July null-metric gap is effectively closed.

Recommended actions

1. [High] Separate driver-exit from real failures in dashboards. 14/14 reds were 0-turn/driver-exit and 3 occurred after the agent finished its work — the current red count overstates fleet ill-health. Classify Execute * CLI non-zero-exit-after-completion distinctly from agent-logic failures.
2. [Medium] Exclude intentional guardrail tests from prod-main rollups. Daily Credit Limit Test and Daily Max Ai Credits Test are designed to fail and cost ~1.5pts of prod-main rate; tag or filter them.
3. [Watch] Two novel infra transients (Start MCP Gateway, Install AWF binary) — no firewall involvement, monitor for recurrence before escalating.

References

• §29780629585 (this audit)
• §29770233192 (Install AWF binary transient)
• §29770927744 (MCP Gateway start transient)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔍 Agentic Workflow Audit Agent · 274.2 AIC · ⌖ 41.3 AIC · ⊞ 7.6K · ◷

• expires on Jul 21, 2026, 1:52 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #47150.