Summary
Smoke test for Claude engine validation — Run §22526414043
Overall Status: PARTIAL — Test 3 (Serena find_symbol) failed due to MCP connection drop; Test 17 skipped
PR used for review testing: #18754 (copilot/fix-safe-outputs-target-repo-support)
Timestamp: 2026-02-28
Core Test Results (#1–10)
| # |
Test |
Status |
Notes |
| 1 |
GitHub MCP — Review last 2 merged PRs |
✅ |
PRs #18876, #18879 found |
| 2 |
SafeInputs GH CLI — Query 2 PRs |
✅ |
PRs #18878, #18754 returned |
| 3 |
Serena MCP — Activate project + find_symbol |
❌ |
Project activated; find_symbol failed (MCP EOF/connection closed) |
| 4 |
Make Build |
✅ |
make build succeeded |
| 5 |
Playwright — Navigate to github.com |
✅ |
Title: "GitHub · Change is constant..." |
| 6 |
Tavily Web Search |
✅ |
3 results for "GitHub Agentic Workflows" |
| 7 |
File Writing |
✅ |
/tmp/gh-aw/agent/smoke-test-claude-22526414043.txt created |
| 8 |
Bash Tool |
✅ |
File contents verified via cat |
| 9 |
Discussion Interaction |
✅ |
Discussion #18881 found, comment added |
| 10 |
Agentic Workflows MCP |
✅ |
smoke-claude workflow: active, engine: claude |
PR Review Safe Output Test Results (#11–17)
| # |
Test |
Status |
Notes |
| 11 |
Update PR body |
✅ |
Appended test message to PR #18754 |
| 12 |
PR Review Comments |
✅ |
2 inline comments on .changeset/ and add_labels.cjs |
| 13 |
Submit PR Review |
✅ |
COMMENT review submitted |
| 14 |
Resolve Review Thread |
✅ |
Thread PRRT_kwDOPc1QR85xU3uQ resolved |
| 15 |
Add Reviewer |
✅ |
copilot added as reviewer |
| 16 |
Push to PR Branch |
✅ |
Smoke test file pushed to copilot/fix-safe-outputs-target-repo-support |
| 17 |
Close PR |
⚠️ |
Skipped — no safe test PR to close |
Test 3 Error Details
Serena activate_project succeeded with response:
Created and activated a new project with name 'gh-aw' at /home/runner/work/gh-aw/gh-aw. Programming languages: go, typescript, bash
However, both find_symbol calls failed:
- First attempt:
MCP error 0: calling "tools/call": EOF
- Second attempt (with relative_path):
MCP error 0: connection closed: calling "tools/call": client is closing: EOF
- Re-activation attempt:
MCP error 0: connection closed: calling "tools/call": client is closing: EOF
The Serena MCP server appears to crash or disconnect after activate_project returns. This is likely an issue with the Serena MCP server configuration or resource limits in this environment.
Agentic Workflows Status Detail
[{
"workflow": "smoke-claude",
"engine_id": "claude",
"compiled": "No",
"status": "active",
"triggers": {
"pull_request": {"names": ["smoke"], "types": ["labeled"]},
"reaction": "heart",
"schedule": "every 12h",
"status-comment": true,
"workflow_dispatch": null
}
}]
References:
💥 [THE END] — Illustrated by Smoke Claude
Summary
Smoke test for Claude engine validation — Run §22526414043
Overall Status: PARTIAL — Test 3 (Serena find_symbol) failed due to MCP connection drop; Test 17 skipped
PR used for review testing: #18754 (
copilot/fix-safe-outputs-target-repo-support)Timestamp: 2026-02-28
Core Test Results (#1–10)
make buildsucceeded/tmp/gh-aw/agent/smoke-test-claude-22526414043.txtcreatedcatsmoke-claudeworkflow: active, engine: claudePR Review Safe Output Test Results (#11–17)
.changeset/andadd_labels.cjsPRRT_kwDOPc1QR85xU3uQresolvedcopilotadded as reviewercopilot/fix-safe-outputs-target-repo-supportTest 3 Error Details
Serena
activate_projectsucceeded with response:However, both
find_symbolcalls failed:MCP error 0: calling "tools/call": EOFMCP error 0: connection closed: calling "tools/call": client is closing: EOFMCP error 0: connection closed: calling "tools/call": client is closing: EOFThe Serena MCP server appears to crash or disconnect after
activate_projectreturns. This is likely an issue with the Serena MCP server configuration or resource limits in this environment.Agentic Workflows Status Detail
[{ "workflow": "smoke-claude", "engine_id": "claude", "compiled": "No", "status": "active", "triggers": { "pull_request": {"names": ["smoke"], "types": ["labeled"]}, "reaction": "heart", "schedule": "every 12h", "status-comment": true, "workflow_dispatch": null } }]References: