fix: detect max consecutive cache misses exceeded and surface dedicated failure context - #49070
Conversation
…ed failure context Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Triage Summary
Improves failure-handler messaging for max_cache_misses_exceeded errors. Still draft; no CI runs recorded yet.
|
|
@copilot this is a common error generated by awf, look into the awf events logs and add a unified detection mechanism for all agentic engines. |
…errors for all agentic engines - Add MAX_CACHE_MISSES_EXCEEDED_EVENT_TYPE constant and parseMaxCacheMissesExceededFromEventLog() to ai_credits_context.cjs to detect structured max_cache_misses_exceeded events from the AWF API proxy event logs (engine-agnostic guardrail, all engines share the same proxy) - Add MAX_CACHE_MISSES_EXCEEDED_PATTERN regex and isMaxCacheMissesExceededError() to detect_agent_errors.cjs — the unified detection module — covering both the agent stdio log (text pattern) and the AWF API proxy event log (structured events) - Wire maxCacheMissesExceeded into detectErrors() return value, buildOutputLines() (max_cache_misses_exceeded output), and the main() stderr diagnostics - Update handle_agent_failure.cjs to import MAX_CACHE_MISSES_EXCEEDED_PATTERN from detect_agent_errors.cjs instead of maintaining a duplicate regex - Add 20 new tests to detect_agent_errors.test.cjs covering MAX_CACHE_MISSES_EXCEEDED_PATTERN, isMaxCacheMissesExceededError, detectErrors maxCacheMissesExceeded field, and buildOutputLines max_cache_misses_exceeded output Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in commit The
The result is exposed as a |
There was a problem hiding this comment.
Pull request overview
Adds dedicated failure context for AWF API proxy cache-miss limit failures.
Changes:
- Detects cache-miss-limit signals in runtime and structured logs.
- Renders actionable failure guidance instead of generic engine output.
- Adds detector and template tests.
Show a summary per file
| File | Description |
|---|---|
actions/setup/md/max_cache_misses_exceeded.md |
Adds failure guidance template. |
actions/setup/js/handle_agent_failure.cjs |
Selects dedicated failure context. |
actions/setup/js/handle_agent_failure.test.cjs |
Tests detection and rendering helpers. |
actions/setup/js/detect_agent_errors.cjs |
Adds unified error detection and output. |
actions/setup/js/detect_agent_errors.test.cjs |
Tests text detection and output generation. |
actions/setup/js/ai_credits_context.cjs |
Detects structured proxy events. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 6/6 changed files
- Comments generated: 4
- Review effort level: Medium
| > **Engine Cache Miss Limit Exceeded**: The {engine_label} engine hit the provider's consecutive cache miss limit and could not complete this run. | ||
|
|
||
| This signal was detected from engine runtime logs. | ||
|
|
||
| <details> | ||
| <summary>What caused this</summary> | ||
|
|
||
| The provider enforces a limit on consecutive cache misses. When too many back-to-back requests bypass the prompt cache, the API returns a 403 error and the engine terminates. |
There was a problem hiding this comment.
Addressed in a0f7e24. The template now attributes the failure to the gh-aw API proxy apiProxy.maxCacheMisses guardrail and the proxy's 403 max_cache_misses_exceeded response.
| if (hasEngineMaxCacheMissesExceededSignal(logContent)) { | ||
| core.info("Detected engine max cache misses signal — using dedicated context message"); | ||
| return buildEngineMaxCacheMissesExceededContext(engineLabel); | ||
| } |
There was a problem hiding this comment.
Addressed in a0f7e24. Added buildEngineFailureContext coverage for cache-miss signals from both stdio and structured event logs, including suppression of the generic last-output context.
| `http_400_response_error=${results.http400ResponseError}`, | ||
| `capi_quota_exceeded_error=${effectiveCAPIQuotaExceeded}`, | ||
| `invocation_cap_exceeded=${results.invocationCapExceeded}`, | ||
| `max_cache_misses_exceeded=${results.maxCacheMissesExceeded}`, |
There was a problem hiding this comment.
Addressed in a0f7e24. The detect-agent-errors output is now propagated through main job outputs and conclusion env wiring, and the handler also honors the structured event-log signal.
| function parseMaxCacheMissesExceededFromEventLog(eventLogPathOverride) { | ||
| return iterateJSONLFiles( | ||
| resolveUnknownModelAICreditsLogPaths(eventLogPathOverride), | ||
| false, | ||
| content => content.includes(MAX_CACHE_MISSES_EXCEEDED_EVENT_TYPE), | ||
| (acc, entry) => { | ||
| if (acc) return true; // already detected, short-circuit | ||
| return traverseObjectTree(entry, (_key, value) => value === MAX_CACHE_MISSES_EXCEEDED_EVENT_TYPE) || undefined; |
There was a problem hiding this comment.
Addressed in a0f7e24. Added direct parser coverage for parseMaxCacheMissesExceededFromEventLog, including primary and fallback event-log paths plus negative cases.
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. No ADR enforcement needed: PR does not have the 'implementation' label and has ≤100 new lines of code in business logic directories (0 additions detected in src/, lib/, pkg/, internal/, app/, core/, domain/, services/, api/). |
|
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ Warning threat detection engine error DetailsThe threat detection engine failed to produce results. Review the workflow run logs for details. |
|
✅ Test Quality Sentinel completed test quality analysis. Warning threat detection engine error DetailsThe threat detection engine failed to produce results. Review the workflow run logs for details. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warning
threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.
Details
The threat detection engine failed to produce results.
Review the workflow run logs for details.
Skills-Based Review 🧠
Applied /diagnosing-bugs and /tdd — requesting changes on wiring gaps and test coverage.
📋 Key Themes
Issues
- Missing downstream wiring (
detect_agent_errors.cjsline 222):max_cache_misses_exceededis emitted as a GitHub Actions output but is not consumed byaddMainJobEngineErrorOutputs/buildAgentFailureEngineDetectionVars, so detection never reaches the failure context pipeline. - Missing integration test (
handle_agent_failure.cjsline 2657):buildEngineFailureContextis not exercised through the new early-return branch — tests cover the detector and template builder individually but leave branch ordering and suppression untested. - Wrong attribution in user-facing template (
max_cache_misses_exceeded.mdline 9): The text attributes the limit to "the provider" but the guardrail is the AWF API proxyapiProxy.maxCacheMisses. parseMaxCacheMissesExceededFromEventLoghas no direct tests (ai_credits_context.cjsline 450): analogous parsers in this file have full event-log coverage.
Positive Highlights
- ✅ Regex covers both observable forms (JSON error type + human-readable message)
- ✅ Shared
MAX_CACHE_MISSES_EXCEEDED_PATTERNavoids duplication between detector and failure handler - ✅ Template progressive disclosure is consistent with existing rate-limit and max-runs templates
- ✅ Tests for
hasEngineMaxCacheMissesExceededSignalandbuildEngineMaxCacheMissesExceededContextare thorough
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 48.8 AIC · ⊞ 7K
Comment /matt to run again
|
Warning threat detection engine error DetailsThe threat detection engine failed to produce results. Review the workflow run logs for details. Review SummaryThis PR correctly models the
Please address all four inline comments before merging.
|
There was a problem hiding this comment.
Warning
threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.
Details
The threat detection engine failed to produce results.
Review the workflow run logs for details.
test
🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 38.3 AIC · ⊞ 5.3K
|
Warning threat detection engine error DetailsThe threat detection engine failed to produce results. Review the workflow run logs for details. 🧪 Test Quality Sentinel Report✅ Test Quality Score: 81/100 — Excellent
📊 Metrics (22 tests)
|
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Addressed in a0f7e24. I fixed the four unresolved review items, added the missing propagation/tests, and updated the affected wasm goldens. Local validation passed for |
|
Warning threat detection engine error DetailsThe threat detection engine failed to produce results. Review the workflow run logs for details. @copilot please run the skill, address unresolved review feedback if any, and rerun checks once the branch is up to date.
|
|
🎉 This pull request is included in a new release. Release: |
The
max_cache_misses_exceedederror (HTTP 403) from the Anthropic API was not recognized by the failure handler, so affected runs produced the generic engine failure context instead of actionable guidance.Changes
actions/setup/md/max_cache_misses_exceeded.md— new template with[!WARNING]alert plus two<details>blocks (cause explanation, remediation steps) for progressive disclosure; base error message is suppressed when this firesactions/setup/js/handle_agent_failure.cjs— newENGINE_MAX_CACHE_MISSES_EXCEEDED_REregex matching both the JSON error type (max_cache_misses_exceeded) and the human-readable message (Maximum consecutive cache misses exceeded); newhasEngineMaxCacheMissesExceededSignal()/buildEngineMaxCacheMissesExceededContext()pair wired intobuildEngineFailureContext()between the max-runs and generic error paths — early return suppresses the base error, consistent with the existingengine_rate_limit_429andengine_max_runs_exceededpatternsactions/setup/js/handle_agent_failure.test.cjs— 8 new tests covering empty input, JSON error type, human-readable message, exact production log line, unrelated content, template rendering, label fallback, and label whitespace trimming