Reduce Pelis Advisor AI credit burn with fixed low-cost model + turn cap#5166
Conversation
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (4 files)
Coverage comparison generated by |
There was a problem hiding this comment.
⚠️ Not ready to approve
The new regression test should also assert GH_AW_INFO_MODEL (and a minor lock-file formatting consistency issue should be addressed) to fully prevent configuration drift.
Pull request overview
This PR reduces AI credit consumption for the “Pelis Agent Factory Advisor” scheduled workflow by pinning a lower-cost Copilot model and enforcing a small turn budget, and adds a CI regression test to keep the source and compiled lock workflow in sync.
Changes:
- Pins the workflow engine model to
claude-haiku-4.5and caps execution tomax-turns: 4in the source workflow frontmatter. - Updates the compiled lock workflow to use
claude-haiku-4.5andGH_AW_MAX_TURNS: 4at runtime. - Adds a Jest regression test to assert the model/turn-budget constraints remain enforced.
File summaries
| File | Description |
|---|---|
scripts/ci/pelis-agent-factory-advisor-workflow.test.ts |
Adds a regression test that verifies the source + lock workflow keep the low-cost model and turn cap. |
.github/workflows/pelis-agent-factory-advisor.md |
Updates workflow frontmatter to cap turns and pin the Copilot model. |
.github/workflows/pelis-agent-factory-advisor.lock.yml |
Aligns the compiled workflow runtime env with the pinned model and turn budget. |
Copilot's findings
- Files reviewed: 3/3 changed files
- Comments generated: 2
Note
Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| it('compiles model and turn budget into lock workflow', () => { | ||
| const lock = fs.readFileSync(lockPath, 'utf-8'); | ||
|
|
||
| expect(lock).toContain('COPILOT_MODEL: claude-haiku-4.5'); | ||
| expect(lock).toContain('GH_AW_MAX_TURNS: 4'); |
|
@copilot address review feedback |
|
❌ Contribution Check failed. Please review the logs for details. |
|
🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅ |
|
✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓 |
|
❌ Smoke Claude failed |
|
✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓 Smoke test summary comment posted and label added: FAIL |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
🚀 Security Guard has started processing this pull request |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
✅ Build Test Suite completed successfully! |
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded. |
Smoke Test: Copilot BYOK — PASS ✅
Running in direct BYOK mode (
|
🔬 Smoke Test ResultsPR: Reduce Pelis Advisor AI credit burn with fixed low-cost model + turn cap
Overall: FAIL — pre-computed step outputs were not substituted (template variables unresolved).
|
|
PR titles:
Results:
Overall: FAIL
|
🔍 Smoke Test Results — Auth mode: PAT (COPILOT_GITHUB_TOKEN)
Overall: FAIL — pre-step PR: "Reduce Pelis Advisor AI credit burn with fixed low-cost model + turn cap" ·
|
|
|
🔬 Smoke Test: API Proxy OpenTelemetry Tracing
All 5 scenarios passed. OTEL tracing integration is fully functional in the api-proxy sidecar.
|
Chroot Smoke Test Results
Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environments.
|
Smoke Test Results
Overall: FAIL —
|
|
|
Smoke Test Results
Overall status: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
The Pelis Agent Factory Advisor workflow intermittently aborted due to the
max-ai-creditsguardrail before completing useful analysis. This change tightens its execution budget by pinning a cheaper model and limiting turn count so scheduled runs stay within credits.Workflow budget controls (source of truth)
.github/workflows/pelis-agent-factory-advisor.mdto explicitly constrain runtime:engine.model: claude-haiku-4.5max-turns: 4Compiled workflow parity
.github/workflows/pelis-agent-factory-advisor.lock.ymlso runtime env matches frontmatter intent:GH_AW_INFO_MODEL: claude-haiku-4.5COPILOT_MODEL: claude-haiku-4.5GH_AW_MAX_TURNS: 4Regression guard for future edits
scripts/ci/pelis-agent-factory-advisor-workflow.test.tsto enforce that both source and lock workflow keep the low-cost model + turn cap configuration.