Skip to content

[agentic-token-optimizer] Instructions Janitor: reduce failure rate and wasted retries (40% of last 10 runs failed) #48930

Description

@github-actions

Target workflow

Instructions Janitor (.github/workflows/instructions-janitor.md, engine: claude, schedule: daily, timeout-minutes: 20)

Selected because it has the highest reliability-driven cost waste among candidates not optimized in the last 14 days (PR Code Quality Reviewer, Matt Pocock Skills Reviewer, and Impeccable Skills Reviewer were excluded as recently optimized or self-adjacent), and its recent run history shows a clear, evidence-backed failure pattern.

Analysis period + runs analyzed

Last 10 scheduled runs (2026-07-20 → 2026-07-29), cross-checked against daily AIC snapshots for 2026-07-03/07/08.

Cost profile

Metric Value
Runs sampled (job logs) 10
Failed runs 4 / 10 (40%)
Successful-run AIC samples 825.124, 467.465, 469.417
Avg AIC (successful runs) ≈587
Timeout budget 20 min
Most recent run timed out at 20 min with zero output (full AIC/cost spent, no PR)

Ranked recommendations

1. Pin an explicit fallback-safe model instead of relying on default model resolution — est. savings ~150 AIC on affected runs
Three of the four failures (2026-07-26, 07-27, 07-28) hit the identical hard error:
API Error: 400 Model "claude-opus-5" has no AI credits pricing and no default pricing is configured. The harness retried 3 times (5s → 10s → 20s backoff, each retry reloading ~50K tokens of cache context) before giving up with exit code 1 — burning tokens on every attempt with no artifact produced. The workflow's engine: claude frontmatter has no explicit model:, so it is exposed to opportunistic routing onto an unpriced model. Pin engine: { id: claude, model: claude-sonnet-5 } (or the org's approved default) to remove this class of failure entirely.
Evidence: §30346190933, §30254981529, §30195965203.

2. Bound the git-history exploration to prevent unbounded timeout runs — est. savings ~470 AIC per timeout avoided
The most recent run (2026-07-29) ran the full 20-minute budget and timed out with ##[error]The action 'Execute Claude Code CLI' has timed out after 20 minutes after burning the full AIC cost (GH_AW_AIC: 429.554 recorded before timeout) with no PR opened. Transcript shows the agent chaining unbounded git log --oneline, per-commit git show --stat, and repeated git log --since=<release-date> calls across both docs/ and .github/aw/, expanding scope commit-by-commit rather than doing one bounded diff pass. Add an explicit instruction bounding the Release/Change audit to a single git log --since=<release-date> --name-only call plus a single batched git diff <release-tag>..HEAD -- docs/ .github/aw/, with a hard cap (e.g., "inspect at most 5 individual commits in detail").
Evidence: run 30439448071 (2026-07-29, no PR opened, full timeout).

3. Add an early no-op short-circuit before running all 4 audits — est. savings ~200 AIC on no-change days
The "Edge Cases" section says "if instructions are already current, exit without edits", but this check is only reachable after already running the Release/Change, Size, Duplication, and Accuracy audits (i.e., after most of the cost is already spent). Move the no-op decision earlier: after the Release/Change audit confirms no relevant docs/code changes and the Size audit confirms all files are under their limits, allow exiting immediately rather than continuing through Duplication and Accuracy passes.

4. Reduce retry-storm cost when the harness detects a fatal, non-retryable API error — est. savings ~100 AIC per occurrence
The three pricing-error failures each triggered 3 full retries (including a wasted "fresh run" restart after --continue disabled permanently) even though the 400 error is deterministic and will not resolve on retry. This is largely a harness-level behavior, but pinning the model (recommendation 1) removes the trigger condition entirely, which is the more actionable fix from the workflow side.

Caveats

  • The 3 pricing-error failures may originate from platform-side model routing rather than the workflow file itself; pinning model: is the most direct workflow-level mitigation available.
  • Sample size is 10 scheduled runs; longer-window verification (30 days) would strengthen confidence but was not available in the provided data inputs.
  • No structural (## Setup extraction or inline sub-agent) optimization is recommended — the workflow's audits are not clearly repeating setup prefixes, and each audit step benefits from the same broad context (release diff, file inventory) the main agent already holds.
Raw failure evidence
  • Run 30439448071 (2026-07-29): ##[error]The action 'Execute Claude Code CLI' has timed out after 20 minutes. — AIC recorded 429.554, no PR.
  • Run 30346190933 (2026-07-28): API Error: 400 Model "claude-opus-5" has no AI credits pricing... × 3 attempts, then Process completed with exit code 1.
  • Run 30254981529 (2026-07-27): same opus pricing error pattern, exit code 1.
  • Run 30195965203 (2026-07-26): same opus pricing error pattern, exit code 1.
  • Runs 30152281437, 30081736996, 29994472550, 29907305874, 29817618715, 29731874747 (2026-07-20 → 2026-07-25): all succeeded.

Generated by Agentic Workflow AIC Usage Optimizer · aut00 · 68.3 AIC · ⊞ 10.4K ·

  • expires on Aug 5, 2026, 7:47 AM UTC-08:00

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions