diff --git a/.github/plugin/marketplace.json b/.github/plugin/marketplace.json index 536f0f852c..de973ecd63 100644 --- a/.github/plugin/marketplace.json +++ b/.github/plugin/marketplace.json @@ -1109,6 +1109,12 @@ "description": "Security frameworks, accessibility guidelines, performance optimization, and code quality best practices for building secure, maintainable, and high-performance applications.", "version": "1.0.0" }, + { + "name": "shepherd-task", + "source": "plugins/shepherd-task", + "description": "End-to-end task shepherding system that automates the full lifecycle of GitHub issues through Copilot coding agent assignment, CI approval, code review resolution, and merge. Includes orchestration scripts for both Bash and PowerShell.", + "version": "1.0.0" + }, { "name": "signals-dashboard", "source": "extensions/signals-dashboard", diff --git a/.vscode/mcp.json b/.vscode/mcp.json index 6699af5648..c567a2f961 100644 --- a/.vscode/mcp.json +++ b/.vscode/mcp.json @@ -1,6 +1,7 @@ { "servers": { "github-agentic-workflows": { + "disabled": true, "command": "gh", "args": [ "aw", @@ -9,4 +10,4 @@ "cwd": "${workspaceFolder}" } } -} \ No newline at end of file +} diff --git a/dd-3031763-improve-agentic-velocity-remove-before-merge/20260803-dd-3042014-01-review-of-extant-post-mortems.md b/dd-3031763-improve-agentic-velocity-remove-before-merge/20260803-dd-3042014-01-review-of-extant-post-mortems.md new file mode 100644 index 0000000000..342b76a0e5 --- /dev/null +++ b/dd-3031763-improve-agentic-velocity-remove-before-merge/20260803-dd-3042014-01-review-of-extant-post-mortems.md @@ -0,0 +1,79 @@ +# Review of Extant Shepherd-Task Post-Mortems + +**Date:** 2026-08-03 +**Scope:** Seven supplied shepherd-task post-mortems, plus the shepherd-task skills and orchestration scripts in this repository. + +## Executive summary + +The documented difficulties were concentrated in asynchronous boundaries and state detection rather than in the core ability to implement or merge code. The most consequential problems were: + +- **Copilot Coding Agent (CCA):** slow or absent responses to requested corrections, including phase-1 timeouts; one large implementation also required repeated correction cycles. +- **Copilot Code Review Agent (CCRA):** refusal to review PRs over its file-count limit, reviews that reported no reviewable files, and follow-up reviews that were requested but did not arrive within the configured timeout. +- **Copilot CLI:** sessions could be killed when polling caused the agent to go idle; exported session exit codes could report success even when shepherding failed; serialized fail-fast execution left later tasks untouched. +- **GitHub CLI (`gh`):** old versions failed on deprecated Projects Classic GraphQL fields; an invalid REST reviewer identity returned HTTP 422; merge-method assumptions caused deterministic failures; and `gh pr edit` could fail after a mutation had actually succeeded. + +The reports also document successful recovery: local review fixes, workflow reruns, squash-merge fallback, resumable PR state, and independent `gh` verification prevented several failures from becoming incorrect merges. + +## Tool-by-tool findings + +### Copilot Coding Agent (CCA) + +| Where it struggled | Evidence and impact | +|---|---| +| **Did not respond to requested corrections within the phase-1 wait window.** | In the 2026-07-30 21:41 run for [#2169](https://github.com/github/copilot-sdk/issues/2169) / [#2175](https://github.com/github/copilot-sdk/pull/2175), phase 1 found requirement gaps, requested corrections, and timed out after 10 minutes without a CCA push. Phase 2 recovered by fixing the implementation locally, but the PR remained open because the second CCRA review was not acknowledged. | +| **Needed repeated correction cycles on a large implementation.** | In the 2026-07-30 17:26 run for [#2168](https://github.com/github/copilot-sdk/issues/2168) / [#2173](https://github.com/github/copilot-sdk/pull/2173), the report records four dismissed human correction cycles involving generated workflow lock files before CCA reached a clean head. | +| **Produced implementations needing substantial iterative review correction.** | In the Java real-estate run, issue [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) required 7 CCRA rounds and 24 comments; issues [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) and [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) reached the 8-round cap. | + +### Copilot Code Review Agent (CCRA) + +| Where it struggled | Evidence and impact | +|---|---| +| **Could not review oversized PRs.** | In the 2026-07-30 17:26 run, CCRA declined [#2170](https://github.com/github/copilot-sdk/pull/2170) because the PR exceeded the 300-file limit. No inline comments were generated. | +| **Reported no reviewable files.** | In the 2026-07-30 01:56 run for [#2144](https://github.com/github/copilot-sdk/issues/2144) / [#2152](https://github.com/github/copilot-sdk/pull/2152), CCRA said it was unable to review any files, providing zero actionable coverage. | +| **Follow-up reviews did not arrive after successful requests.** | In the 2026-07-30 21:15 run for [#2168](https://github.com/github/copilot-sdk/issues/2168) / [#2173](https://github.com/github/copilot-sdk/pull/2173), two findings were fixed, threads resolved, and CI passed, but no new review appeared within 10 minutes, leaving the PR open. | +| **Created long convergence loops on complex changes.** | The Java real-estate post-mortem records 47 total rounds and 287 inline comments across 9 merged tasks, with two tasks reaching the 8-round cap. The Python final batch was healthier: 20 rounds and 30 comments across 6 tasks, with zero idle-kill failures. | + +### Copilot CLI + +| Where it struggled | Evidence and impact | +|---|---| +| **Went idle while a polling command was running.** | The Python post-mortem describes failed runs where polling exceeded the tool wait, the assistant said it would check back, an `assistant.idle` event appeared, and the session terminated with the PR open. | +| **Reported a misleading success-shaped exported exit code.** | In the 2026-07-30 runs for [#2168](https://github.com/github/copilot-sdk/issues/2168) / [#2173](https://github.com/github/copilot-sdk/pull/2173), the exported session record reported `exitCode: 0` while the final shepherd message said `SHEPHERD FAILED`; the outer caller returned `1`. | +| **Stopped serialized campaigns at the first unrecovered task failure.** | The 21:41 run attempted 1 of 7 tasks; six were never started. The 01:56 run targeted 8 tasks but produced artifacts for only the first task. | +| **Retried deterministic failures.** | In the 21:34 run, the CLI attempted merge-commit mode three times before selecting squash. In the 21:41 run, it repeated the same unacknowledged reviewer-request mechanism three times. | + +### GitHub CLI (`gh`) + +| Where it struggled | Evidence and impact | +|---|---| +| **Installed version was incompatible with the Copilot reviewer flow.** | In the 2026-07-30 21:15 run, `gh` 2.45.0 failed while querying a deprecated Projects Classic GraphQL field during `gh pr edit --add-reviewer`. A standalone `gh` 2.96.0 successfully requested `@copilot`. | +| **The REST fallback used the wrong reviewer identity.** | The same run attempted `copilot-pull-request-reviewer` through the REST collaborator endpoint; attempts returned HTTP 422 because that is the bot’s output identity, not a requestable collaborator. | +| **Could return failure after a mutation succeeded.** | The reports note that `gh pr edit` may complete the reviewer mutation and then fail during its deprecated Projects Classic query. Positive API state, not process status alone, is required. | +| **Assumed an unavailable merge method.** | In the 2026-07-30 21:34 run, `gh pr merge --merge` failed three times because merge commits were disabled; `--squash` succeeded. | +| **Was otherwise a useful independent verifier.** | The orchestrator uses `gh` to verify PR linkage, base branch, CI, unresolved threads, merge state, and issue closure rather than trusting Copilot exit codes. | + +## Cross-cutting conclusions + +1. **The dominant failure mode is asynchronous state ambiguity.** CCA correction pushes, CCRA requests/reviews, CI, and Copilot CLI sessions have different completion signals. +2. **Review coverage is not equivalent to review completion.** A zero-comment review can mean convergence, no reviewable files, or a file-count refusal. +3. **Phase boundaries were sometimes violated during recovery.** The 21:41 report says phase 2 proceeded even though phase 1 had failed its CCA correction wait. +4. **Independent state verification is the strongest design choice.** Actual PR, CI, review-thread, reviewer-request, and merge state are more reliable than session status alone. + +## Recommended priorities + +1. Preflight `gh` version and repository merge settings before starting a campaign. +2. Treat reviewer acknowledgement, review completion, and review content as separate states with distinct retries and timeouts. +3. Persist resumable state containing issue, PR, target head, review ID, request timestamps, resolved threads, and CI status. +4. Propagate semantic shepherd failure into a machine-readable nonzero result. +5. Detect CCRA file-limit and no-reviewable-files responses explicitly. +6. Make batch progress explicit (`merged`, `failed`, `resumable`, `not-started`) and record fail-fast skips. + +## Sources + +- [Java real-estate post-mortem](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/blob/edburns/2-build-out-demo/dd-3017826-java-real-estate-demo-remove-before-merge/dd-3029269-post-mortem-report.md) +- [Python demo post-mortem](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/blob/edburns/28-python-agent-demo/28-python-agent-demo-remove-before-merge/28-python-agent-demo-post-mortem.md) +- [Java runtime run 2144-2151](https://github.com/github/copilot-sdk/blob/edburns/1917-java-embed-rust-cli-runtime-dd-3039924-agentic-run-02/1917-java-embed-rust-cli-runtime-remove-before-merge/shepherd-tasks-20260730-0156/20260730-0200-post-mortem.md) +- [Java runtime run 2167-2168](https://github.com/github/copilot-sdk/blob/edburns/1917-java-embed-rust-cli-runtime-dd-3039924-agentic-run-02/1917-java-embed-rust-cli-runtime-remove-before-merge/shepherd-tasks-20260730-1726/20260730-2032-post-mortem.md) +- [Java runtime run 2169-2175](https://github.com/github/copilot-sdk/blob/edburns/1917-java-embed-rust-cli-runtime-dd-3039924-agentic-run-02/1917-java-embed-rust-cli-runtime-remove-before-merge/shepherd-tasks-20260730-2141/20260730-2247-post-mortem.md) +- [Java runtime retry](https://github.com/github/copilot-sdk/blob/edburns/1917-java-embed-rust-cli-runtime-dd-3039924-agentic-run-02/1917-java-embed-rust-cli-runtime-remove-before-merge/shepherd-tasks-20260730-2115/20260730-2128-post-mortem.md) +- [Java runtime successful merge](https://github.com/github/copilot-sdk/blob/edburns/1917-java-embed-rust-cli-runtime-dd-3039924-agentic-run-02/1917-java-embed-rust-cli-runtime-remove-before-merge/shepherd-tasks-20260730-2134/20260730-2135-post-mortem.md) \ No newline at end of file diff --git a/dd-3031763-improve-agentic-velocity-remove-before-merge/20260803-dd-3042014-02-review-of-prompt-files-in-lieu-of-post-mortems.md b/dd-3031763-improve-agentic-velocity-remove-before-merge/20260803-dd-3042014-02-review-of-prompt-files-in-lieu-of-post-mortems.md new file mode 100644 index 0000000000..f77f89024c --- /dev/null +++ b/dd-3031763-improve-agentic-velocity-remove-before-merge/20260803-dd-3042014-02-review-of-prompt-files-in-lieu-of-post-mortems.md @@ -0,0 +1,91 @@ +# Review of Earlier Shepherd-Task Prompt Logs + +**Date:** 2026-08-03 +**Sources:** Prompt logs under `copilot-sdk-01/1682-java-tool-ergonomics-prompts-remove-before-merge` and `copilot-sdk-02/1810-java-tool-ergonomics-tool-as-lambda-remove-before-merge`. + +## Executive summary + +These earlier prompt logs predate the formal post-mortem process, so they do not provide one uniform run summary. They do, however, record several concrete shepherd executions and the difficulties encountered while building and using the primitive shepherd skill. + +The main findings are: + +- **Copilot Coding Agent (CCA):** PR creation and coding-agent completion were slow enough to exceed fixed 10-minute and 15-minute polls. The agent could continue working after the local poll timed out, creating uncertainty about whether it was stuck or merely slow. +- **Copilot Code Review Agent (CCRA):** review completion was difficult to detect; a 10-minute poll found no review even though a Copilot review later existed. Review comments also exposed real semantic defects that source-text-only tests had missed. +- **Copilot CLI:** the orchestration session repeatedly had to manage long blocking waits, distinguish expected failures from real failures, and recover from phase-state ambiguity. Very large token/session costs were recorded for some runs. +- **GitHub CLI (`gh`):** workflow approval semantics were initially misunderstood; the fork-only approval endpoint was tried before `gh run rerun` was identified as the correct same-repository mechanism. Branch-name/remote-name confusion caused base-branch update failures and false failure reports after successful merges. A stale or incorrect comment lookup also returned HTTP 404. + +## Tool-by-tool findings + +### Copilot Coding Agent (CCA) + +| Where it struggled | Evidence and impact | +|---|---| +| **PR creation exceeded the initial poll window.** | In the 2026-07-01 shepherd run for task [#1876](https://github.com/github/copilot-sdk/issues/1876), the first “PR found” poll ran for 600 seconds and timed out. The issue was assigned to Copilot, but no PR or issue comments were visible at that point. The agent was still working, so the timeout did not distinguish slow progress from a stuck run. | +| **Agent execution exceeded another 10-minute wait.** | In the 2026-06-30 run for [#1840](https://github.com/github/copilot-sdk/issues/1840) / [#1857](https://github.com/github/copilot-sdk/pull/1857), the Copilot cloud-agent workflow was still running after 600 seconds while seven workflow runs awaited approval. The operator had to continue waiting before approving the runs. | +| **Agent execution exceeded a longer 15-minute wait.** | In the same 2026-06-30 prompt log, task [#1841](https://github.com/github/copilot-sdk/issues/1841) required a 900-second wait; the script timed out while the agent was still running and then had to wait again. | +| **Implementation quality required substantial human-directed iteration.** | In the 1682 logs for PR [#1792](https://github.com/github/copilot-sdk/pull/1792), human review identified a single-record schema/handler mismatch, generic-type deserialization loss, primitive optional-parameter hazards, and other semantic issues. These were not merely formatting problems; they required processor changes, new validation, and targeted tests. | +| **Created unrelated or poorly scoped work in one case.** | The 2026-06-24 log records an unexpected PR [#1786](https://github.com/github/copilot-sdk/pull/1786), created by the Copilot SWE Agent against `main`, duplicating the incremental Epic [#1682](https://github.com/github/copilot-sdk/issues/1682) work instead of targeting the feature branch. | + +### Copilot Code Review Agent (CCRA) + +| Where it struggled | Evidence and impact | +|---|---| +| **Review completion was not reliably observable within the polling window.** | For PR [#1877](https://github.com/github/copilot-sdk/pull/1877), the prompt log shows a 600-second poll ending with `TIMEOUT: No Copilot review found`. A subsequent `gh api` query showed a Copilot review (`copilot-pull-request-reviewer[bot]`) already present, demonstrating that the polling predicate or timing was insufficient. | +| **Review results arrived asynchronously relative to CI and agent work.** | For PR [#1857](https://github.com/github/copilot-sdk/pull/1857), the process had to wait for the cloud agent, rerun action-required workflows, watch checks, and separately inspect bot comments before declaring the PR ready. The review state was not a single event that could be trusted from the initial PR state. | +| **Found important semantic defects missed by earlier tests.** | Review comments on [#1792](https://github.com/github/copilot-sdk/pull/1792) identified schema/handler inconsistency for a single record parameter, generic type erasure during Jackson conversion, invalid optional primitive handling, and unsafe reflection/JPMS behavior. The prompt log explicitly concludes that existing tests checked generated source shape but not schema-plus-handler behavior. | +| **Review comments sometimes required additional investigation before remediation.** | Examples include determining whether a reviewer’s generic-erasure comment referred to raw `Class` tokens versus full Jackson `JavaType`, and replacing a raw JSON substring assertion with an `ObjectNode.has("defer")` check. This added review latency but improved correctness. | + +### Copilot CLI + +| Where it struggled | Evidence and impact | +|---|---| +| **Long waits consumed the interactive session and required repeated continuation.** | The 2026-06-30 and 2026-07-01 logs contain repeated 600-, 900-, and 1800-second waits for PR creation, cloud-agent completion, CI, and review. The operator repeatedly had to inspect status and launch another wait rather than receive a reliable terminal result. | +| **Phase completion was initially defined textually rather than structurally.** | The primitive skill used `SHEPHERD COMPLETE` / `SHEPHERD FAILED` text in Copilot output and later had to add explicit instructions that phase 2 must not start after phase 1 failure. This indicates the CLI session and outer script did not initially have a robust machine-readable phase contract. | +| **Expected CI failures had to be manually separated from real failures.** | `gh pr checks --watch --fail-fast` exited 1 on the expected “No remove-before-merge directories” check. The Copilot session then performed a second status query and correctly ignored that check. Without this interpretation, the CLI would have classified a successful phase as failed. | +| **Review polling could produce false negatives.** | The 600-second no-review result for PR [#1877](https://github.com/github/copilot-sdk/pull/1877), followed by discovery of an existing review, shows that the CLI’s polling logic was too narrow or raced with review publication. | +| **Resource use was high.** | One 2026-07-01 shepherd session reports approximately 724.6k tokens and 21m 27s of AI-credit/session usage for a phase-1 run. The logs show many repeated status inspections and waits, indicating significant orchestration overhead. | +| **The CLI could recover successfully when the operator supplied explicit state checks.** | Tasks [#1840](https://github.com/github/copilot-sdk/issues/1840) and [#1842](https://github.com/github/copilot-sdk/issues/1842) reached ready-for-review status after workflow reruns, CI filtering, base-branch correction attempts, and bot-comment checks. | + +### GitHub CLI (`gh`) + +| Where it struggled | Evidence and impact | +|---|---| +| **Workflow approval endpoint semantics were initially wrong.** | The primitive shepherd first investigated a `POST .../actions/runs/{id}/approve` path, then discovered from the actual same-repository behavior that it is fork-only. The correct recovery was `gh run rerun` for `action_required` runs. | +| **`action_required` state was confusing.** | Runs appeared with `status: completed` and `conclusion: action_required`, which did not look like a normal pending run. The operator had to inspect run details and experiment before determining that rerunning the runs was the correct operational equivalent of clicking “Approve workflows to run.” | +| **Base-branch names mixed remote-qualified and repository-qualified forms.** | For PR [#1879](https://github.com/github/copilot-sdk/pull/1879), the PR was successfully merged into `edburns/1810-java-tool-ergonomics-tool-as-lambda`, but the outer script expected `upstream/edburns/1810-java-tool-ergonomics-tool-as-lambda` and reported failure. Attempts to set the base to the remote-qualified name returned `GraphQL: Proposed base branch ... was not found`. The same issue was recorded for PR [#1881](https://github.com/github/copilot-sdk/pull/1881). | +| **The CLI could report a deterministic 404 for stale review-comment identifiers.** | A lookup for comment `3508639895` on PR [#1877](https://github.com/github/copilot-sdk/pull/1877) returned `gh: Not Found (HTTP 404)`. A subsequent collection query found the comment, indicating the identifier/path combination was stale or the endpoint was wrong. | +| **Merge/check commands required repository-specific interpretation.** | The primitive process used `gh pr checks --watch`, but its nonzero exit code included the expected remove-before-merge check. It also needed explicit base-branch inspection and correction rather than assuming the requested string was a valid GitHub branch name. | +| **`gh` was valuable once used as an independent state source.** | `gh issue view`, `gh pr view`, `gh run list`, `gh run rerun`, `gh pr checks`, and `gh api` supplied the evidence needed to distinguish CCA progress, workflow approval, CI completion, review presence, and merge state. | + +## Cross-cutting conclusions + +1. **The primitive system’s central weakness was state-machine ambiguity.** “Assigned,” “agent still working,” “PR exists,” “workflow needs approval,” “review requested,” “review posted,” “checks complete,” and “merged” were often discovered through separate ad hoc queries. +2. **Fixed timeouts were observation checkpoints, not reliable failure boundaries.** Several agents continued successfully after 600 or 900 seconds. A timeout needed to preserve resumable state rather than imply that work had stopped. +3. **Remote agent progress and local CLI progress were coupled in a fragile way.** The local session had to remain active while the cloud agent and review agent operated asynchronously, increasing waiting cost and the risk of false failure. +4. **Repository identifiers needed normalization.** A Git remote name such as `upstream` is not part of a GitHub branch name. Passing `upstream/branch` to `gh pr edit --base` caused avoidable failures and later false-negative verification. +5. **The most valuable review feedback was semantic.** The 1682 logs show that generated-source assertions alone did not prove runtime schema/handler correctness. Review agents found defects only when behavior, serialization, and invocation semantics were considered together. + +## Recommended priorities + +1. Use a structured state record for every task and phase: issue, PR, branch, target base branch, current head SHA, agent status, workflow run IDs, review IDs, and terminal state. +2. Make all waits resumable and configurable; distinguish “not observed within window” from “confirmed failed.” +3. Normalize `BASE_BRANCH` before invoking `gh`: accept a Git remote separately, but pass only the repository branch name to GitHub APIs. +4. Preflight workflow approval behavior and use `gh run rerun` for same-repository `action_required` runs. +5. Poll for reviews by PR, reviewer identity, review ID, and target head, rather than relying on one body format or a single timing window. +6. Treat expected checks as named exceptions in machine-readable evaluation rather than interpreting `gh pr checks` exit code alone. +7. Require integration or execution-level tests for generated tool schema and handler binding, not only generated source-text assertions. +8. Preserve a resumable outcome after every timeout and report whether the task is `in-progress`, `ready`, `failed`, `resumable`, `merged`, or `not-started`. + +## Source inventory + +### Issue 1682 prompt logs + +- `20260615-prompts.md` through `20260618-prompts.md`: initial Java tool-definition investigation, low-level E2E test planning, skill creation, and architecture/ignorance-reduction work. +- `20260622-prompts.md` through `20260626-prompts.md`: schema-generator testing, review feedback, PR [#1792](https://github.com/github/copilot-sdk/pull/1792), single-record schema/handler remediation, and generated-code/JPMS concerns. + +### Issue 1810 prompt logs + +- `20260626-prompts.md` and `20260628-prompts.md`: lambda API design, zero-argument overloads, LangChain4j and Micronaut comparison, and plan formation. +- `20260629-prompts.md`: issue and plan preparation. +- `20260630-prompts.md`: first primitive shepherd execution, workflow rerun discovery, CI/review waiting, and PR [#1857](https://github.com/github/copilot-sdk/pull/1857). +- `20260701-prompts.md`: shepherd skill refinement and executions involving PRs [#1877](https://github.com/github/copilot-sdk/pull/1877), [#1879](https://github.com/github/copilot-sdk/pull/1879), and [#1881](https://github.com/github/copilot-sdk/pull/1881). diff --git a/docs/README.plugins.md b/docs/README.plugins.md index 5511d40691..ce1992053f 100644 --- a/docs/README.plugins.md +++ b/docs/README.plugins.md @@ -87,6 +87,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-plugins) for guidelines on how t | [rust-mcp-development](../plugins/rust-mcp-development/README.md) | Build high-performance Model Context Protocol servers in Rust using the official rmcp SDK with async/await, procedural macros, and type-safe implementations. | 2 items | rust, mcp, model-context-protocol, server-development, sdk, tokio, async, macros, rmcp | | [salesforce-development](../plugins/salesforce-development/README.md) | Complete Salesforce agentic development environment covering Apex & Triggers, Flow automation, Lightning Web Components, Aura components, and Visualforce pages. | 7 items | salesforce, apex, triggers, lwc, aura, flow, visualforce, crm, salesforce-dx | | [security-best-practices](../plugins/security-best-practices/README.md) | Security frameworks, accessibility guidelines, performance optimization, and code quality best practices for building secure, maintainable, and high-performance applications. | 1 items | security, accessibility, performance, code-quality, owasp, a11y, optimization, best-practices | +| [shepherd-task](../plugins/shepherd-task/README.md) | End-to-end task shepherding system that automates the full lifecycle of GitHub issues through Copilot coding agent assignment, CI approval, code review resolution, and merge. Includes orchestration scripts for both Bash and PowerShell. | 3 items | shepherd, task-automation, copilot-coding-agent, ci-cd, code-review, github-actions, workflow-approval, yolo | | [skill-image-gen](../plugins/skill-image-gen/README.md) | Generate images using AI directly from your coding workflow. Supports OpenAI (gpt-image-2) and Google Gemini. BYO API key — the skill guides you through setup on first use. | 1 items | image-generation, openai, gemini, ai, art, sprites, textures, icons | | [software-engineering-team](../plugins/software-engineering-team/README.md) | 7 specialized agents covering the full software development lifecycle from UX design and architecture to security and DevOps. | 7 items | team, enterprise, security, devops, ux, architecture, product, ai-ethics | | [structured-autonomy](../plugins/structured-autonomy/README.md) | Premium planning, thrifty implementation | 3 items | | diff --git a/docs/README.skills.md b/docs/README.skills.md index 46d2bb38bf..7aa289cf1a 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -369,6 +369,12 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [security-review](../skills/security-review/SKILL.md)
`gh skills install github/awesome-copilot security-review` | AI-powered codebase security scanner that reasons about code like a security researcher — tracing data flows, understanding component interactions, and catching vulnerabilities that pattern-matching tools miss. Use this skill when asked to scan code for security vulnerabilities, find bugs, check for SQL injection, XSS, command injection, exposed API keys, hardcoded secrets, insecure dependencies, access control issues, or any request like "is my code secure?", "review for security issues", "audit this codebase", or "check for vulnerabilities". Covers injection flaws, authentication and access control bugs, secrets exposure, weak cryptography, insecure dependencies, and business logic issues across JavaScript, TypeScript, Python, Java, PHP, Go, Ruby, and Rust. | `references/language-patterns.md`
`references/report-format.md`
`references/secret-patterns.md`
`references/vuln-categories.md`
`references/vulnerable-packages.md` | | [semantic-kernel](../skills/semantic-kernel/SKILL.md)
`gh skills install github/awesome-copilot semantic-kernel` | Create, update, refactor, explain, or review Semantic Kernel solutions using shared guidance plus language-specific references for .NET and Python. | `references/dotnet.md`
`references/python.md` | | [setup-my-iq](../skills/setup-my-iq/SKILL.md)
`gh skills install github/awesome-copilot setup-my-iq` | Create, set up, or update the personal context portfolio: structured markdown files describing
who you are, how you work, your teams, and your tool/ADO configuration. Runs the interview
workflow for first-time setup and targeted edits for updates.

Trigger this skill when the user asks to: set up their context, create or update their context
portfolio, "create my IQ", "set up my IQ", edit their profile, add/remove a stakeholder,
update ADO config, change team info, update pillars, or set up any plugin configuration.
Trigger when another skill fails to find context (missing files or TODO markers) and needs
context populated. Also trigger when the user mentions a context change in passing
(e.g., "my manager changed", "we added someone to the team") to offer a context file update.

Do NOT trigger for read-only questions like "who's on my team?" or "what's my ADO config?".
Those are answered directly from the context files referenced in the loaded custom
instructions; no skill is needed. | `assets/templates` | +| [shepherd-task-approve-workflows-and-wait-for-completion](../skills/shepherd-task-approve-workflows-and-wait-for-completion/SKILL.md)
`gh skills install github/awesome-copilot shepherd-task-approve-workflows-and-wait-for-completion` | Use this skill to approve pending workflow runs on a PR branch and wait for them to complete. | None | +| [shepherd-task-create-ignorance-reduction-plan](../skills/shepherd-task-create-ignorance-reduction-plan/SKILL.md)
`gh skills install github/awesome-copilot shepherd-task-create-ignorance-reduction-plan` | Use this skill when creating a new ignorance reduction plan — a structured document that maps unknowns, spikes, and phased implementation steps for a multi-day engineering campaign. | None | +| [shepherd-task-create-issues-from-plan](../skills/shepherd-task-create-issues-from-plan/SKILL.md)
`gh skills install github/awesome-copilot shepherd-task-create-issues-from-plan` | Use this skill to turn the ordered implementation section of an ignorance reduction plan into detailed, serial child Task issues under an existing GitHub parent issue, incorporating resolved research, spike artifacts, concrete example-issue style, branch instructions, gating tests, persistent run artifacts, and verified sub-issue ordering. All 12 inputs are required. | `examples` | +| [shepherd-task-create-post-mortem](../skills/shepherd-task-create-post-mortem/SKILL.md)
`gh skills install github/awesome-copilot shepherd-task-create-post-mortem` | Create a structured post-mortem report from shepherd-task run artifacts, including metrics, timeline, failures, and actionable recommendations. | `examples` | +| [shepherd-task-from-assignment-to-ready](../skills/shepherd-task-from-assignment-to-ready/SKILL.md)
`gh skills install github/awesome-copilot shepherd-task-from-assignment-to-ready` | Use this skill to shepherd a child Task issue from 'assigned to Copilot' through CI approval and review-agent feedback resolution, stopping just before marking the PR as **Ready for review**. | None | +| [shepherd-task-from-ready-to-merged-to-base](../skills/shepherd-task-from-ready-to-merged-to-base/SKILL.md)
`gh skills install github/awesome-copilot shepherd-task-from-ready-to-merged-to-base` | Use this skill to shepherd a task PR from 'Ready for review' through Copilot code review, local comment resolution, and merge to the specified base branch. | None | | [shuffle-json-data](../skills/shuffle-json-data/SKILL.md)
`gh skills install github/awesome-copilot shuffle-json-data` | Shuffle repetitive JSON objects safely by validating schema consistency before randomising entries. | None | | [signal-write](../skills/signal-write/SKILL.md)
`gh skills install github/awesome-copilot signal-write` | Emit structured agent signals — hands-up, blocked, done, checkpoint, partnership. Signals are written as JSON to .signals/ for dashboard consumption and noted in the journal for persistence. | None | | [slang-shader-engineer](../skills/slang-shader-engineer/SKILL.md)
`gh skills install github/awesome-copilot slang-shader-engineer` | Use when working with Slang shaders, shader modules, HLSL-compatible GPU code, graphics pipelines, compute shaders, tessellation, ray tracing, parameter blocks, generics, interfaces, capabilities, cross-compilation, shader optimization, shader review, or C++ engine integration for Slang. Trigger on any mention of Slang, .slang files, slangc, SPIR-V from Slang, Slang modules, [shader("compute")], [shader("vertex")], or requests to write/review/refactor shader code with modern language features. Also trigger for Slang-to-HLSL/GLSL/Metal/CUDA cross-compile questions, or when the user says "shader" alongside "generics", "interfaces", "parameter blocks", "autodiff", or "capabilities". | `references/language-reference.md`
`references/rules-and-patterns.md`
`references/slang-documentation-full.md` | diff --git a/plugins/shepherd-task/.github/plugin/plugin.json b/plugins/shepherd-task/.github/plugin/plugin.json new file mode 100644 index 0000000000..9f6747dfb8 --- /dev/null +++ b/plugins/shepherd-task/.github/plugin/plugin.json @@ -0,0 +1,25 @@ +{ + "name": "shepherd-task", + "description": "End-to-end task shepherding system that automates the full lifecycle of GitHub issues through Copilot coding agent assignment, CI approval, code review resolution, and merge. Includes orchestration scripts for both Bash and PowerShell.", + "version": "1.0.0", + "author": { + "name": "Ed Burns" + }, + "repository": "https://github.com/github/awesome-copilot", + "license": "MIT", + "keywords": [ + "shepherd", + "task-automation", + "copilot-coding-agent", + "ci-cd", + "code-review", + "github-actions", + "workflow-approval", + "yolo" + ], + "skills": [ + "./skills/shepherd-task-approve-workflows-and-wait-for-completion/", + "./skills/shepherd-task-from-assignment-to-ready/", + "./skills/shepherd-task-from-ready-to-merged-to-base/" + ] +} diff --git a/plugins/shepherd-task/README.md b/plugins/shepherd-task/README.md new file mode 100644 index 0000000000..3940d829f0 --- /dev/null +++ b/plugins/shepherd-task/README.md @@ -0,0 +1,267 @@ +## Summary + +**shepherd-task** is a collection of skills and scripts that invoke GitHub and GitHub Copilot agents and CLIs to drive a sufficiently detailed set of GitHub issues from assignment to merged. + +**shepherd-task** uses this opinionated choice of the following agents and tools. + +1. Copilot Coding Agent (CCA). +2. Copilot Code Review Agent (CCRA). +3. GitHub CLI `gh`. +4. GitHub Copilot CLI `copilot`. + +It shepherds tasks from assignment through CI approval, code review resolution, and merge — all orchestrated via `copilot --yolo` sessions driven by shell scripts. + +## Enabling assumptions + +### Preconditions + +1. The work will happen within a single GitHub repository. +1. The repository has GitHub Copilot Coding Agent and Code Review Agent installed and properly configured. +1. User has `gh` CLI installed at version 2.45.0 or later and is signed in. +1. USer has permissions on the repository to: + 1. Assign issues to `Copilot`. + 1. Push to the repository. +1. User has `copilot` CLI installed at version `1.0.72-0` or later and is signed in. +1. User has a local development environment suitable for using Copilot CLI and `git worktree` to do development work, including running tests. +1. User accepts that all of the work is to be done on a non-`main` base branch in the repository. +1. User accepts this plugin will invoke `copilot --yolo`. +1. The specification for the job to be done is encoded in an ordered set of GitHub issues in the issue tracker of the GitHub repository. +1. User has `jq` installed and accessible to Copilot CLI. + +### Postconditions + +The user of `shepherd-task` is responsible for creating a PR from the specified non-`main base branch to main. + +## Motivation + +When using the Copilot coding agent to implement tasks from a backlog, there is a significant amount of repetitive supervisory work: assigning the issue, waiting for the PR, approving workflow runs, interpreting CI failures, requesting changes, waiting for fixes, resolving code review comments, and finally merging. This system automates that entire loop, allowing a developer to hand off a list of issue numbers and walk away. + +A successful run of the system will result in the work being merged to the specified non-`main` base branch, so that a further PR can be made from that branch with greater human oversight. + +This plugin was developed and battle-tested in the `github/copilot-sdk` repository, where it has successfully shepherded dozens of tasks through the full lifecycle. It is being contributed here so the broader community can adopt, adapt, and improve it. + +## Architecture + +The system has two layers: + +1. **Skills** (Copilot-invocable instructions) — three SKILL.md files that Copilot reads during `--yolo` sessions to know what to do at each phase: + - `shepherd-task-from-assignment-to-ready`: Assigns an issue to @copilot with a specific base branch, waits for PR creation, iterates through CI approval and review-agent feedback (up to 20 iterations). + - `shepherd-task-from-ready-to-merged-to-base`: Marks the PR as ready, waits for Copilot code review, resolves comments locally in a git worktree, pushes fixes, and merges. + - `shepherd-task-approve-workflows-and-wait-for-completion`: Reusable sub-skill that approves pending `action_required` workflow runs and blocks until completion. + +2. **Orchestration scripts** (user-facing entry points) — shell scripts that launch `copilot --yolo` sessions, verify outcomes with `gh` CLI between phases, and provide idempotent retry semantics. Both Bash and PowerShell variants are included for cross-platform support. + +The plugin manifest (`plugin.json`) ties these together as a discoverable unit in the marketplace. + +### Invocation flow + +1. User invokes `shepherd-task-given-list`. See [figure 01: script shepherd-task-given-list](figure-01-shepherd-task-given-list.md). + +2. For each task, `shepherd-task-given-list` invokes `shepherd-task`. See [figure 02: script shepherd-task](figure-02-shepherd-task.md). + + a. Phase 1: Assign task to Copilot Coding Agent and make it ready for review. See [figure 03: skill: from-assigned-to-ready](figure-03-from-assigned-to-ready.md). + + b. Phase 2: Mark ready for review and resolve all Copilot Code Review Agent comments that have merit. See [figure 04: skill: from ready-to-merged-to-base](figure-04-from-ready-to-merged.md). + +3. Perform post-mortem. See [figure 05: skill: create post-mortem](figure-05-post-mortem.md). + + +## Sample invocation + +Bash + +```bash +. ./plugins/shepherd-task/scripts/shepherd-task-given-list.sh 13,4,5,6,7,8,9,10,11 edburns/2-build-out-demo edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk +``` + +PowerShell + +```powershell +. .\plugins\shepherd-task\scripts\shepherd-task-given-list.ps1 13,4,5,6,7,8,9,10,11 edburns/2-build-out-demo edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk +``` + +This invocation was used to implement the issues in this Epic: https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/2 . + +## Installation + +### Step 1: Install skills (via `gh skill`) + +```bash +gh skill install github/awesome-copilot shepherd-task-from-assignment-to-ready +gh skill install github/awesome-copilot shepherd-task-from-ready-to-merged-to-base +gh skill install github/awesome-copilot shepherd-task-approve-workflows-and-wait-for-completion +``` + +Requires GitHub CLI v2.90.0+. + +### Step 2: Install orchestration scripts + +The scripts are not installable via `gh skill`. Either: +- Clone this repository and run from `plugins/shepherd-task/scripts/`, or +- Use the bundled installer to copy only the scripts into your repo: + + Bash + + ```bash + . ./plugins/shepherd-task/scripts/install-task-shepherd.sh /path/to/your/repo + ``` + + PowerShell + + ```powershell + . .\plugins\shepherd-task\scripts\install-task-shepherd.ps1 \path\to\your\repo + ``` + +The installer copies **only** the orchestration scripts — it does not duplicate the skills (those are managed by `gh skill install` above). + +## Uninstallation + +There is no `gh skill uninstall` command. To remove installed skills, manually delete the skill directories from your agent's skills location. + +- Use the bundled uninstaller to remove the scripts from your repo. + + Bash + + ```bash + . ./plugins/shepherd-task/scripts/uninstall-task-shepherd.sh /path/to/your/repo + ``` + + PowerShell + + ```powershell + . .\plugins\shepherd-task\scripts\uninstall-task-shepherd.ps1 \path\to\your\repo + ``` + + +## How it works + +``` +User runs: shepherd-task-given-list.sh "1841,1842,1843" feature-branch owner/repo + └─ For each issue, calls shepherd-task.sh + ├─ Phase 1: echo prompt | copilot --yolo (invokes shepherd-task-from-assignment-to-ready) + │ └─ skill loops: assign → wait for PR → approve workflows → fix CI → repeat + ├─ Verify: gh pr checks, no unresolved reviews + ├─ Phase 2: echo prompt | copilot --yolo (invokes shepherd-task-from-ready-to-merged-to-base) + │ └─ skill loops: mark ready → resolve review comments locally → push → merge + └─ Verify: PR merged to non-main base branch, issue closed +``` + +### Redacting log files + +Before committing shepherd-task logs, redact secret-bearing fields from a log +directory. Run the script from the directory containing the log directory: + +```bash +./redact-secrets.sh shepherd-tasks-20260803-1550 +``` + +The Bash script requires `jq`; the PowerShell script uses built-in JSON +support. Both scripts process `.json*` files recursively and replace +credential fields and content-bearing event fields with `[REDACTED]`. + +## Key design decisions + +- **Scripts verify state independently** — they don't trust Copilot exit codes; they use `gh` CLI to confirm PR existence, CI status, and merge state between phases. +- **Idempotent** — if a PR already exists for the issue, Phase 1 is skipped. If already merged, Phase 2 is skipped. Safe to re-run after failures. +- **Local review resolution** — Phase 2 resolves Copilot code review comments in a local git worktree rather than asking the remote agent to fix itself, giving more reliable results. +- **Cross-platform** — every script has both `.sh` and `.ps1` variants. + +## Per-file manifest + +### Plugin + +| File | Purpose | +|------|---------| +| `plugins/shepherd-task/.github/plugin/plugin.json` | Plugin manifest grouping the three skills; includes metadata, keywords, and version | +| `plugins/shepherd-task/README.md` | User-facing documentation for the plugin | + +### Orchestration scripts + +| File | Purpose | +|------|---------| +| `plugins/shepherd-task/scripts/shepherd-task-given-list.sh` | Bash: iterates a comma-separated list of issue numbers, invoking shepherd-task.sh for each | +| `plugins/shepherd-task/scripts/shepherd-task-given-list.ps1` | PowerShell equivalent | +| `plugins/shepherd-task/scripts/shepherd-task.sh` | Bash: orchestrates Phase 1 + Phase 2 for a single task issue with state verification | +| `plugins/shepherd-task/scripts/shepherd-task.ps1` | PowerShell equivalent | +| `plugins/shepherd-task/scripts/shepherd-task-inspect-json.sh` | Bash: debug utility to inspect copilot JSON session logs | +| `plugins/shepherd-task/scripts/shepherd-task-inspect-json.ps1` | PowerShell equivalent | +| `plugins/shepherd-task/scripts/redact-secrets.sh` | Bash: redact secret-bearing fields from `.json*` shepherd logs | +| `plugins/shepherd-task/scripts/redact-secrets.ps1` | PowerShell equivalent | +| `plugins/shepherd-task/scripts/install-task-shepherd.sh` | Bash: copies the full shepherd system into another repository | +| `plugins/shepherd-task/scripts/install-task-shepherd.ps1` | PowerShell equivalent | + +### Skills + +| File | Purpose | +|------|---------| +| `skills/shepherd-task-from-assignment-to-ready/SKILL.md` | Skill instructions for Phase 1: assign to Copilot → PR created → CI passing → no unresolved reviews | +| `skills/shepherd-task-from-ready-to-merged-to-base/SKILL.md` | Skill instructions for Phase 2: mark ready → resolve code review → merge to base branch | +| `skills/shepherd-task-approve-workflows-and-wait-for-completion/SKILL.md` | Reusable sub-skill: approve `action_required` workflow runs and wait for completion | + +### Generated files (updated by `npm run build`) + +| File | Purpose | +|------|---------| +| `.github/plugin/marketplace.json` | Marketplace registry updated with new shepherd-task plugin entry | +| `docs/README.plugins.md` | Plugin listing updated to include shepherd-task | +| `docs/README.skills.md` | Skills listing updated with the three new shepherd-task skills | + +## Validation + +``` +$ npm run skill:validate +✅ shepherd-task-approve-workflows-and-wait-for-completion is valid +✅ shepherd-task-from-assignment-to-ready is valid +✅ shepherd-task-from-ready-to-merged-to-base is valid + +$ npm run plugin:validate +✅ shepherd-task is valid +``` + +## Testing + +This system has been used in production on `github/copilot-sdk` to shepherd implementation tasks for Java SDK development. It handles the common failure modes: workflow approval gates, CI flakiness, Copilot code review comment iteration, merge conflicts, and base branch drift. + +## Pull Request Checklist + +- [x] I have read and followed the [CONTRIBUTING.md](https://github.com/github/awesome-copilot/blob/main/CONTRIBUTING.md) guidelines. +- [x] I have read and followed the [Guidance for submissions involving paid services](https://github.com/github/awesome-copilot/discussions/968). +- [x] My contribution adds a new instruction, prompt, agent, skill, workflow, or canvas extension file in the correct directory. +- [x] The file follows the required naming convention. +- [x] The content is clearly structured and follows the example format. +- [x] I have tested my instructions, prompt, agent, skill, workflow, or canvas extension with GitHub Copilot. +- [x] I have run `npm start` and verified that `README.md` is up to date. +- [ ] I am targeting the `main` branch for this pull request. + +--- + +## Description + + + +--- + +## Type of Contribution + +- [ ] New instruction file. +- [ ] New prompt file. +- [ ] New agent file. +- [x] New plugin. +- [x] New skill file. +- [ ] New agentic workflow. +- [ ] New canvas extension. +- [ ] Update to existing instruction, prompt, agent, plugin, skill, workflow, or canvas extension. +- [ ] Other (please specify): + +--- + +## Additional Notes + + + +--- + +By submitting this pull request, I confirm that my contribution abides by the [Code of Conduct](../CODE_OF_CONDUCT.md) and will be licensed under the MIT License. + + + diff --git a/plugins/shepherd-task/figure-01-shepherd-task-given-list.md b/plugins/shepherd-task/figure-01-shepherd-task-given-list.md new file mode 100644 index 0000000000..f75bcadc02 --- /dev/null +++ b/plugins/shepherd-task/figure-01-shepherd-task-given-list.md @@ -0,0 +1,55 @@ +# Figure 01: shepherd-task-given-list — Batch Dispatch + +This diagram shows the highest-level orchestration: how `shepherd-task-given-list.ps1` takes a comma-separated list of issue numbers and dispatches them serially. + +## Example Invocation + +```powershell +shepherd-task-given-list.ps1 "51,52,53,54" edburns/dd-3034809-test-01 edburns/Build26-BRK206-your-agent +``` + +## Sequence Diagram + +```mermaid +sequenceDiagram + actor User + participant STGL as shepherd-task-given-list.ps1 + participant ST as shepherd-task.ps1 + participant Copilot as copilot --yolo + participant PostMortem as shepherd-task-create-post-mortem + + User->>STGL: "51,52,53,54"
edburns/dd-3034809-test-01
edburns/Build26-BRK206-your-agent + + Note over STGL: Create timestamped log directory
shepherd-tasks-YYYYMMDD-HHMM/ + + Note over STGL: Parse comma-separated list
into [51, 52, 53, 54] + + rect rgb(220, 240, 255) + Note over STGL,ST: Serial loop — one issue at a time + STGL->>ST: shepherd-task.ps1 -TaskIssue 51
-BaseBranch edburns/dd-3034809-test-01
-Repo edburns/Build26-BRK206-your-agent
-LogDir shepherd-tasks-YYYYMMDD-HHMM/ + ST-->>STGL: exit 0 (success) + + STGL->>ST: shepherd-task.ps1 -TaskIssue 52
-BaseBranch edburns/dd-3034809-test-01
-Repo edburns/Build26-BRK206-your-agent
-LogDir shepherd-tasks-YYYYMMDD-HHMM/ + ST-->>STGL: exit 0 (success) + + STGL->>ST: shepherd-task.ps1 -TaskIssue 53
-BaseBranch edburns/dd-3034809-test-01
-Repo edburns/Build26-BRK206-your-agent
-LogDir shepherd-tasks-YYYYMMDD-HHMM/ + ST-->>STGL: exit 0 (success) + + STGL->>ST: shepherd-task.ps1 -TaskIssue 54
-BaseBranch edburns/dd-3034809-test-01
-Repo edburns/Build26-BRK206-your-agent
-LogDir shepherd-tasks-YYYYMMDD-HHMM/ + ST-->>STGL: exit 0 (success) + end + + rect rgb(255, 245, 220) + Note over STGL,PostMortem: finally block — runs on success OR failure + STGL->>Copilot: echo prompt | copilot --yolo
"Invoke skill shepherd-task-create-post-mortem" + Copilot->>PostMortem: Invoke with SHEPHERD_LOG_DIR,
SCRIPT_EXIT_CODE, TASK_ISSUES,
BASE_BRANCH, REPO + PostMortem-->>Copilot: Write YYYYMMDD-HHMM-post-mortem.md
to log directory + Copilot-->>STGL: Session complete + end + + STGL-->>User: exit 0
"All tasks shepherded successfully" +``` + +## Failure Behavior + +If any `shepherd-task.ps1` invocation fails (non-zero exit), the loop stops immediately at that issue. The `finally` block still runs, invoking the post-mortem skill with the non-zero exit code so that a failure report is always generated. diff --git a/plugins/shepherd-task/figure-02-shepherd-task.md b/plugins/shepherd-task/figure-02-shepherd-task.md new file mode 100644 index 0000000000..f56c8f4d4e --- /dev/null +++ b/plugins/shepherd-task/figure-02-shepherd-task.md @@ -0,0 +1,80 @@ +# Figure 02: shepherd-task — Single Issue Orchestration + +This diagram drills down into what happens inside `shepherd-task.ps1` for a single issue. It shows the two phases, the `copilot --yolo` sessions that invoke skills, and the independent state verification between phases. + +## Sequence Diagram + +```mermaid +sequenceDiagram + participant ST as shepherd-task.ps1 + participant GH as gh CLI + participant Copilot as copilot --yolo + participant Skill1 as shepherd-task-from-
assignment-to-ready + participant Skill2 as shepherd-task-from-
ready-to-merged-to-base + + Note over ST: Receive TASK_ISSUE, BASE_BRANCH, REPO, LogDir + + rect rgb(230, 245, 230) + Note over ST,GH: Idempotency check — skip Phase 1 if PR exists + ST->>GH: Find-LinkedPR (timeline, body search, title/branch match) + GH-->>ST: PR exists? → skip to verification
No PR? → proceed to Phase 1 + end + + rect rgb(220, 240, 255) + Note over ST,Skill1: Phase 1: Assignment to Ready for Review + ST->>Copilot: echo phase1Prompt | copilot --yolo
--output-format json --share phase1.md > phase1.json + Note over Copilot: COPILOT_OTEL_FILE_EXPORTER_PATH set
for telemetry capture + Copilot->>Skill1: "Invoke skill
shepherd-task-from-assignment-to-ready
with TASK_ISSUE, BASE_BRANCH, REPO" + Note over Skill1: Assign → wait for PR → approve workflows
→ fix CI → iterate (max 20 rounds)
(details in Figure 03) + Skill1-->>Copilot: Skill complete + Copilot-->>ST: Session exits + end + + rect rgb(255, 240, 230) + Note over ST,GH: Phase 1 Verification (script verifies independently via gh) + ST->>GH: Find-LinkedPR → PR_NUMBER + GH-->>ST: PR #N found + ST->>GH: gh pr view — verify base branch = BASE_BRANCH + GH-->>ST: Base branch OK (or fix it) + ST->>GH: Test-CIPassing — all checks pass? + GH-->>ST: CI passing ✓ + ST->>GH: Test-NoUnresolvedReviews — GraphQL review threads + GH-->>ST: No unresolved comments ✓ + Note over ST: "Phase 1 VERIFIED: PR #N is ready" + end + + rect rgb(230, 245, 230) + Note over ST,GH: Idempotency check — skip Phase 2 if already merged + ST->>GH: gh pr view — state == MERGED? + GH-->>ST: Not merged → proceed to Phase 2 + end + + rect rgb(220, 230, 255) + Note over ST,Skill2: Phase 2: Ready for Review to Merged + ST->>Copilot: echo phase2Prompt | copilot --yolo
--output-format json --share phase2.md > phase2.json + Note over Copilot: COPILOT_OTEL_FILE_EXPORTER_PATH set
for telemetry capture + Copilot->>Skill2: "Invoke skill
shepherd-task-from-ready-to-merged-to-base
with TASK_ISSUE, BASE_BRANCH,
REPO, PR_NUMBER" + Note over Skill2: Mark ready → request review → resolve
comments locally → push → merge
(details in Figure 04) + Skill2-->>Copilot: Skill complete + Copilot-->>ST: Session exits + end + + rect rgb(255, 240, 230) + Note over ST,GH: Phase 2 Verification (script verifies independently via gh) + ST->>GH: gh pr view — state == MERGED? + GH-->>ST: MERGED ✓ + ST->>GH: gh pr view — merged into BASE_BRANCH? + GH-->>ST: Correct base ✓ + ST->>GH: gh issue view — state == CLOSED? + GH-->>ST: CLOSED ✓ (or close it) + end + + Note over ST: "SHEPHERD TASK COMPLETE:
PR #N merged to BASE_BRANCH" + ST-->>ST: exit 0 +``` + +## Key Design Points + +- **Scripts verify state independently** — they don't trust `copilot` exit codes; they use `gh` CLI to confirm PR existence, CI status, and merge state between phases. +- **Idempotent** — if a PR already exists for the issue, Phase 1 is skipped. If already merged, Phase 2 is skipped. Safe to re-run after failures. +- **Each phase runs in a separate `copilot --yolo` session** — the skill instructions are read fresh each time. diff --git a/plugins/shepherd-task/figure-03-from-assigned-to-ready.md b/plugins/shepherd-task/figure-03-from-assigned-to-ready.md new file mode 100644 index 0000000000..d542593632 --- /dev/null +++ b/plugins/shepherd-task/figure-03-from-assigned-to-ready.md @@ -0,0 +1,119 @@ +# Figure 03: From Assignment to Ready for Review + +This diagram shows the detail of the `shepherd-task-from-assignment-to-ready` skill, including its nested invocation of `shepherd-task-approve-workflows-and-wait-for-completion`. All of this runs inside a single `copilot --yolo` session (the Local Copilot CLI), which orchestrates interaction with the remote Copilot Coding Agent (CCA) via the GitHub API. + +## Sequence Diagram + +```mermaid +sequenceDiagram + participant CLI as Local Copilot CLI
(copilot --yolo) + participant GitHub as GitHub API / gh CLI + participant CCA as Copilot Coding Agent
(remote) + participant ApproveWF as shepherd-task-approve-
workflows-and-wait + + Note over CLI: Skill inputs: TASK_ISSUE, BASE_BRANCH, REPO + + rect rgb(220, 240, 255) + Note over CLI,CCA: Step 1: Assign issue to @Copilot with base branch + CLI->>GitHub: POST /repos/REPO/issues/TASK_ISSUE/assignees
{ assignees: ["copilot-swe-agent[bot]"],
agent_assignment: { base_branch: BASE_BRANCH } } + GitHub->>CCA: Trigger: issue assigned to Copilot + Note over CCA: CCA creates topic branch from BASE_BRANCH,
opens draft PR, may push an empty
Initial plan commit, then starts work + end + + rect rgb(230, 245, 230) + Note over CLI,GitHub: Step 2: Poll for PR creation (up to 15 min, every 30s) + loop Strategy A → B → C each iteration + CLI->>GitHub: A: Issue timeline (cross-referenced PRs) + CLI->>GitHub: B: PR body search for #TASK_ISSUE + CLI->>GitHub: C: Title/branch name regex match + end + GitHub-->>CLI: PR #N found + CLI->>GitHub: Verify base branch == BASE_BRANCH + CLI->>GitHub: Verify closingIssuesReferences
contains TASK_ISSUE + Note over CLI: PR discovery is a startup signal,
not evidence of completed work + end + + rect rgb(255, 245, 220) + Note over CLI,CCA: Step 3: Wait for completed CCA implementation + loop Poll up to 2 hours + CLI->>GitHub: Query PR state, draft state,
and CCA timeline events + GitHub-->>CLI: latest copilot_work_started /
copilot_work_finished + end + Note over CLI: Require latest finish >= latest start + CLI->>GitHub: Query PR metadata, files,
base commit tree, head commit tree + Note over CLI: Require changed_files > 0,
nonempty files API, and differing trees + end + + rect rgb(235, 245, 255) + Note over CLI,GitHub: Step 4: Verify issue requirements + CLI->>GitHub: Read complete issue body and PR patches + Note over CLI: Map every deliverable and acceptance
criterion to concrete PASS evidence + Note over CLI: Record HEAD_SHA as validation candidate + end + + rect rgb(255, 245, 220) + Note over CLI,ApproveWF: Steps 5–6: Approve workflows (nested skill) + CLI->>GitHub: Check for action_required workflow runs + CLI->>ApproveWF: Invoke shepherd-task-approve-
workflows-and-wait-for-completion
with REPO, JTBDTASK_BRANCH, PR_NUMBER + ApproveWF->>GitHub: gh run rerun (for each pending run) + ApproveWF->>GitHub: gh pr checks --watch --fail-fast + GitHub-->>ApproveWF: All runs complete + ApproveWF-->>CLI: Workflows done + Note over CLI: Require relevant substantive checks
for current HEAD_SHA; selector-only
success cannot pass + CLI->>GitHub: Query check runs for exact HEAD_SHA + Note over CLI: Run every issue-specified gating
command against HEAD_SHA + end + + rect rgb(240, 230, 255) + Note over CLI,GitHub: Step 6: Evaluate CI results + CLI->>GitHub: gh pr checks PR_NUMBER
--jq (exclude "No remove-before-merge directories") + GitHub-->>CLI: Real failures? or All pass? + end + + alt All CI checks pass and no unresolved reviews + Note over CLI: Skip to Step 8–9 + else CI failures or unresolved review comments + rect rgb(255, 230, 230) + Note over CLI,CCA: Step 7: Iteration loop (max 20 rounds) + loop Up to 20 iterations + CLI->>GitHub: 7.1: Gather failed run logs
gh run view RUN_ID --log-failed + CLI->>GitHub: 7.2: Gather bot review comments
from PR comments and reviews + Note over CLI: 7.3: Compose "Request changes" review
with log excerpts + fix instructions + CLI->>GitHub: gh pr review PR_NUMBER --request-changes
--body "@copilot Please fix..." + GitHub->>CCA: Review notification: changes requested + Note over CCA: CCA reads review, pushes fix commits + CLI->>GitHub: 7.4: Poll for new commits
(compare HEAD SHA, up to 10 min) + GitHub-->>CLI: New SHA detected + Note over CLI: Discard all stale evidence;
return to Step 3 and repeat
every gate for the new SHA + end + end + end + + rect rgb(230, 255, 230) + Note over CLI,GitHub: Step 8: Check for unresolved pre-ready review comments + CLI->>GitHub: GraphQL: paginated reviewThreads.isResolved
and reviewDecision + CLI->>GitHub: Query bot comments on PR + Note over CLI: If unresolved comments exist,
iterate using same Step 7 pattern
(shares the 20-iteration budget) + end + + rect rgb(220, 255, 220) + Note over CLI,GitHub: Step 9: Atomic final readiness gate + CLI->>GitHub: Re-query PR, CCA lifecycle, diff,
checks, workflows, and reviews + CLI->>GitHub: Query HEAD SHA again after all gates + alt HEAD changed or any gate failed + Note over CLI: Discard results and return to Step 3,
or fail closed for manual intervention + else Same HEAD and every gate passed + Note over CLI: "SHEPHERD COMPLETE: PR #N for task
#TASK_ISSUE is ready for marking
as Ready for Review." + end + end +``` + +## Notes + +- The Local Copilot CLI acts as the **orchestrator** — it never modifies code itself in this phase. All code changes are made by the **remote Copilot Coding Agent (CCA)**. +- A linked draft PR, an `Initial plan` commit, passing selector checks, or an absence of comments is not evidence that CCA implemented the issue. +- The phase fails closed unless one unchanged HEAD SHA satisfies the CCA lifecycle, effective-diff, issue-requirement, gating-command, relevant-CI, and review gates. +- Any new HEAD SHA invalidates prior evidence and restarts validation at Step 3. +- The `agent_assignment.base_branch` API parameter is the only reliable way to set the base branch. The simpler `gh issue edit --add-assignee` does not support this parameter, causing CCA to default to `main`. +- The "No remove-before-merge directories" CI failure is always ignored — it is expected on feature branches. +- The 20-iteration budget is shared between CI fix iterations (Step 7) and pre-ready review comment iterations (Step 8). diff --git a/plugins/shepherd-task/figure-04-from-ready-to-merged.md b/plugins/shepherd-task/figure-04-from-ready-to-merged.md new file mode 100644 index 0000000000..756d975d8c --- /dev/null +++ b/plugins/shepherd-task/figure-04-from-ready-to-merged.md @@ -0,0 +1,125 @@ +# Figure 04: From Ready for Review to Merged + +This diagram shows the detail of the `shepherd-task-from-ready-to-merged-to-base` skill. Unlike Phase 1 (which relies on the remote Copilot Coding Agent), this phase resolves review comments **locally** using a git worktree, with dialog between the Copilot Code Review Agent (CCRA) and the local Copilot CLI session. + +## Sequence Diagram + +```mermaid +sequenceDiagram + participant CLI as Local Copilot CLI
(copilot --yolo) + participant GitHub as GitHub API / gh CLI + participant CCRA as Copilot Code Review
Agent (remote) + participant Worktree as Git Worktree
(../review-copilot-pr-N) + participant ApproveWF as shepherd-task-approve-
workflows-and-wait + + Note over CLI: Skill inputs: TASK_ISSUE, BASE_BRANCH, REPO, PR_NUMBER + + rect rgb(220, 240, 255) + Note over CLI,CCRA: Step 0–1: Find PR, mark ready, request review + CLI->>GitHub: Find PR via timeline/body/title strategies + CLI->>GitHub: gh pr ready PR_NUMBER + CLI->>GitHub: gh pr edit --add-reviewer
copilot-pull-request-reviewer + Note over CCRA: CCRA begins analyzing the PR diff + end + + rect rgb(230, 245, 230) + Note over CLI,CCRA: Step 2–3: Wait for CCRA review and count comments + CLI->>GitHub: Poll for review (4 detection strategies,
every 30s, up to 10 min) + Note over CLI: Strategy A: body matches "Copilot's findings"
Strategy B: body matches "Pull request overview"
Strategy C: user login contains copilot-pull-request-reviewer
Strategy D: line-level comments from user "Copilot" + CCRA-->>GitHub: Post review with N comments + GitHub-->>CLI: Review detected, N comments found + end + + alt N == 0 (no comments) + Note over CLI: Skip to Step 15 (final checks) + else N > 0 (comments to address) + rect rgb(255, 245, 220) + Note over CLI,Worktree: Step 4: Set up local worktree (sibling directory) + CLI->>GitHub: git fetch upstream + CLI->>Worktree: git worktree add
"../review-copilot-pr-N"
"upstream/JTBDTASK_BRANCH" + end + + rect rgb(240, 240, 255) + Note over CLI,ApproveWF: Step 5: Approve workflows before gathering comments + CLI->>ApproveWF: Invoke approve-workflows skill
(REPO, JTBDTASK_BRANCH, PR_NUMBER) + Note over ApproveWF: (See Figure 03 for details) + ApproveWF-->>CLI: Workflows complete + end + + rect rgb(255, 230, 230) + Note over CLI,GitHub: Review resolution loop (max 8 rounds) + loop Up to 8 iterations + CLI->>GitHub: Step 6: Gather all review comments
from copilot-pull-request-reviewer / Copilot + + loop For each review comment + Note over CLI,Worktree: Step 7: Address each comment locally + Note over CLI: 7.1: Evaluate comment merit + alt Comment has merit + CLI->>Worktree: 7.2: Implement fix in worktree
Run targeted tests + CLI->>Worktree: 7.3: git commit (do NOT push yet)
Record commit hash for reply + else Comment has no merit + Note over CLI: Note: will resolve with explanation in Step 9 + end + end + + CLI->>Worktree: Step 8: Push all fixes at once
git push REMOTE HEAD:JTBDTASK_BRANCH + + loop For each addressed comment + Note over CLI,GitHub: Step 9: Reply and resolve threads + CLI->>GitHub: POST .../comments/COMMENT_ID/replies
body: "Fixed in abc1234. [explanation]" + CLI->>GitHub: GraphQL resolveReviewThread
mutation { resolveReviewThread(...) } + end + + rect rgb(255, 245, 220) + Note over CLI,ApproveWF: Steps 10–11: Wait for CI, approve workflows + CLI->>GitHub: gh pr checks --watch + CLI->>ApproveWF: Invoke approve-workflows skill + ApproveWF-->>CLI: Workflows complete + end + + CLI->>GitHub: Step 12: Re-request Copilot review
gh pr edit --add-reviewer
copilot-pull-request-reviewer + Note over CCRA: CCRA reviews the updated diff + CLI->>GitHub: Step 13: Poll for new review findings + CCRA-->>GitHub: Post new review + GitHub-->>CLI: New review detected + + Note over CLI: If 0 new comments → break loop
If comments remain → next iteration + end + end + end + + rect rgb(230, 255, 230) + Note over CLI,ApproveWF: Step 14: Final workflow approval + CLI->>ApproveWF: Invoke approve-workflows skill + ApproveWF-->>CLI: All runs complete + end + + rect rgb(220, 255, 220) + Note over CLI,GitHub: Steps 15–21: Final checks, cleanup, and merge + CLI->>GitHub: Step 15: Verify only expected CI failure remains + CLI->>Worktree: Step 16: git worktree remove
"../review-copilot-pr-N" + CLI->>GitHub: Step 17: Verify base branch ≠ main + CLI->>GitHub: Step 18: Handle merge conflicts
(rebase + force-with-lease if needed) + CLI->>GitHub: Step 19: gh pr merge --merge --delete-branch + CLI->>GitHub: Step 20: gh issue close TASK_ISSUE + Note over CLI: Step 21: "SHEPHERD COMPLETE:
PR #N merged to BASE_BRANCH" + end +``` + +## Key Design Points + +### Local Worktree Resolution + +All review comment fixes happen in a **sibling git worktree** (`../review-copilot-pr-N`), not in the main working tree and not via the remote Copilot Coding Agent. This gives more reliable results because the local Copilot CLI has full context of the codebase and can run tests before committing. + +### "Fixed in \" Pattern + +When replying to each review comment, the reply includes the commit hash that addresses it: `"Fixed in abc1234. [explanation of the fix]"`. This creates a traceable link between the review feedback and the specific commit that resolved it. The thread is then resolved via the GraphQL `resolveReviewThread` mutation. + +### Max Review Rounds + +The review resolution loop runs for a **maximum of 8 iterations**. If the CCRA continues to generate new comments after 8 rounds, the skill reports failure and stops, requiring manual intervention. This prevents infinite loops when the review agent and the local fixer disagree. + +### Workflow Approval + +The `shepherd-task-approve-workflows-and-wait-for-completion` sub-skill is invoked at multiple points: before gathering comments (Step 5), after pushing fixes (Step 11), and before the final merge (Step 14). Each push to the PR branch triggers new workflow runs that require approval. diff --git a/plugins/shepherd-task/figure-05-post-mortem.md b/plugins/shepherd-task/figure-05-post-mortem.md new file mode 100644 index 0000000000..372c245b3d --- /dev/null +++ b/plugins/shepherd-task/figure-05-post-mortem.md @@ -0,0 +1,75 @@ +# Figure 05: Post-Mortem Report Generation + +This diagram shows the detail of the `shepherd-task-create-post-mortem` skill. It is invoked from the `finally` block of `shepherd-task-given-list.ps1` (see Figure 01) so that a report is always generated, regardless of whether the run succeeded or failed. + +## Sequence Diagram + +```mermaid +sequenceDiagram + participant STGL as shepherd-task-given-list.ps1
(finally block) + participant Copilot as copilot --yolo + participant Skill as shepherd-task-create-post-mortem + participant LogDir as SHEPHERD_LOG_DIR
(shepherd-tasks-YYYYMMDD-HHMM/) + participant ParentDir as Parent Directory
(campaign context) + + rect rgb(255, 245, 220) + Note over STGL,Copilot: Invocation (runs on success OR failure) + STGL->>Copilot: echo prompt | copilot --yolo
"Invoke skill shepherd-task-create-post-mortem"
with SHEPHERD_LOG_DIR, SCRIPT_EXIT_CODE,
TASK_ISSUES, BASE_BRANCH, REPO + end + + rect rgb(220, 240, 255) + Note over Skill,LogDir: Step 1: Validate & collect run artifacts + Copilot->>Skill: Invoke skill + Skill->>LogDir: Validate directory exists + Skill->>LogDir: Collect phase artifacts:
• phase1-task-*.json (session data)
• phase2-task-*.json (session data)
• phase1-task-*.md (session shares)
• phase2-task-*.md (session shares) + end + + rect rgb(230, 245, 230) + Note over Skill,ParentDir: Step 2: Collect campaign context from parent directory + Skill->>ParentDir: Scan for supplementary files:
• *memory*.md
• *prompts.md
• *job-logs.txt + end + + rect rgb(240, 230, 255) + Note over Skill: Step 3: Extract quantitative metrics from artifacts + Note over Skill: Per-task metrics:
• Issues and PRs touched
• Phase 1 & Phase 2 durations
• Review rounds (Comments generated: N)
• Success/failure and failure signatures
• Idle/timeout markers
• Token usage (inputTokens / outputTokens) + end + + rect rgb(255, 240, 230) + Note over Skill: Step 4: Compose report (8 required sections) + Note over Skill: §1 Executive Summary
— outcome, completion rate, elapsed time + Note over Skill: §2 System Architecture
— CCA, CCRA, Local CLI responsibilities + Note over Skill: §3 Per-Task Metrics
— table: issue, PR, timings, rounds, result + Note over Skill: §4 Aggregate Statistics
— totals, averages, convergence signals + Note over Skill: §5 AI Credits and Token Usage
— measured values or "data unavailable" + Note over Skill: §6 Wall-Clock Timeline
— batch windows and notable events + Note over Skill: §7 Failure Analysis (if any)
— root causes, evidence, fixes + Note over Skill: §8 Observations & Recommendations
— what worked, what failed, improvements + end + + rect rgb(220, 255, 220) + Note over Skill,LogDir: Step 5: Write report + Skill->>LogDir: Write YYYYMMDD-HHMM-post-mortem.md + Note over Skill: Links use full URLs:
[#123](https://github.com/REPO/issues/123)
[#456](https://github.com/REPO/pull/456) + end + + Skill-->>Copilot: Report written + Copilot-->>STGL: Session complete +``` + +## Key Design Points + +### Always Runs + +The post-mortem skill is invoked from a `finally` block (PowerShell) or `trap EXIT` (Bash), so it executes for **all outcomes** — full success, partial failure, or early abort. The `SCRIPT_EXIT_CODE` input distinguishes these cases. + +### Evidence-Based + +The report is built entirely from local run artifacts (JSON session logs, markdown shares, OTEL traces). It does not require GitHub API calls unless local artifacts are insufficient, keeping it reliable even when network access is degraded. + +### Structured for Both Outcomes + +The 8-section structure serves both **successful runs** (throughput, convergence, quality metrics) and **failed runs** (root cause analysis, failure signatures, corrective actions). Sections are populated or marked "N/A" based on available data. + +### Link Formatting + +All issue and PR references are rendered as full Markdown hyperlinks using the `REPO` input — never plain-text `#123` references. Table-of-contents entries use plain text to avoid nested link issues. diff --git a/plugins/shepherd-task/scripts/install-task-shepherd.ps1 b/plugins/shepherd-task/scripts/install-task-shepherd.ps1 new file mode 100644 index 0000000000..5de5ad1a71 --- /dev/null +++ b/plugins/shepherd-task/scripts/install-task-shepherd.ps1 @@ -0,0 +1,82 @@ +<# +.SYNOPSIS + Installs the shepherd-task plugin and skills into the user's Copilot home directory. + +.DESCRIPTION + Installs to: + ~/.copilot/plugins/shepherd-task/ (plugin with orchestration scripts) + ~/.copilot/skills/shepherd-task-* (skills, only if not already present) + +.EXAMPLE + ./install-task-shepherd.ps1 +#> + +$ErrorActionPreference = 'Stop' + +$CopilotHome = if ($env:COPILOT_HOME) { $env:COPILOT_HOME } else { Join-Path $HOME '.copilot' } + +$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Definition +$SourceRepo = (Resolve-Path (Join-Path $ScriptDir '..\..\..')).Path +$PluginSrc = (Resolve-Path (Join-Path $ScriptDir '..')).Path + +# Install plugin. +$pluginDest = Join-Path $CopilotHome 'plugins' 'shepherd-task' +if (-not (Test-Path $pluginDest)) { + New-Item -ItemType Directory -Path $pluginDest -Force | Out-Null +} +Copy-Item -Path (Join-Path $PluginSrc '*') -Destination $pluginDest -Recurse -Force +Write-Host "Installed plugin to $pluginDest" + +# Verify the log redaction utilities were included in the plugin installation. +$redactionScripts = @( + 'redact-secrets.sh' + 'redact-secrets.ps1' +) +foreach ($script in $redactionScripts) { + $installedScript = Join-Path $pluginDest 'scripts' $script + if (-not (Test-Path $installedScript -PathType Leaf)) { + throw "Redaction script was not installed: $installedScript" + } +} + +# Install skills (only if not already present). +$skills = @( + 'shepherd-task-from-assignment-to-ready' + 'shepherd-task-from-ready-to-merged-to-base' + 'shepherd-task-approve-workflows-and-wait-for-completion' + 'shepherd-task-create-ignorance-reduction-plan' + 'shepherd-task-create-post-mortem' + 'shepherd-task-create-issues-from-plan' +) + +$skillsInstalled = 0 +$skillsSkipped = 0 +foreach ($skill in $skills) { + $skillSrc = Join-Path $SourceRepo 'skills' $skill + $skillDest = Join-Path $CopilotHome 'skills' $skill + + if (-not (Test-Path $skillSrc -PathType Container)) { + Write-Warning "Source skill not found: $skillSrc" + continue + } + + if (Test-Path $skillDest -PathType Container) { + Write-Host "Skipped ~/.copilot/skills/$skill (already exists)" + $skillsSkipped++ + } else { + New-Item -ItemType Directory -Path $skillDest -Force | Out-Null + Copy-Item -Path (Join-Path $skillSrc '*') -Destination $skillDest -Recurse -Force + Write-Host "Installed ~/.copilot/skills/$skill" + $skillsInstalled++ + } +} + +Write-Host "" +Write-Host "Installation complete." +Write-Host " Plugin: $pluginDest" +Write-Host " Skills: $skillsInstalled installed, $skillsSkipped already present" +Write-Host "" +Write-Host "Interview script available at:" +Write-Host " $(Join-Path $pluginDest 'scripts' 'shepherd-task-interview-user-to-create-issues.ps1')" +Write-Host "" +Write-Host "Verify with: copilot skill list" diff --git a/plugins/shepherd-task/scripts/install-task-shepherd.sh b/plugins/shepherd-task/scripts/install-task-shepherd.sh new file mode 100755 index 0000000000..5bca34d9a0 --- /dev/null +++ b/plugins/shepherd-task/scripts/install-task-shepherd.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# +# install-task-shepherd.sh — Installs the shepherd-task plugin and skills +# into the user's Copilot home directory. +# +# Installs to: +# ~/.copilot/plugins/shepherd-task/ (plugin with orchestration scripts) +# ~/.copilot/skills/shepherd-task-* (skills, only if not already present) +# +# Usage: ./install-task-shepherd.sh + +set -euo pipefail + +COPILOT_HOME="${COPILOT_HOME:-$HOME/.copilot}" + +# Resolve the source repo root (three levels up from plugins/shepherd-task/scripts/). +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +SOURCE_REPO="$(cd "$SCRIPT_DIR/../../.." && pwd)" +PLUGIN_SRC="$SCRIPT_DIR/.." + +# Install plugin. +plugin_dest="$COPILOT_HOME/plugins/shepherd-task" +mkdir -p "$plugin_dest" +cp -R "$PLUGIN_SRC/." "$plugin_dest/" +echo "Installed plugin to $plugin_dest" + +# Verify the log redaction utilities were included in the plugin installation. +REDACTION_SCRIPTS=( + "redact-secrets.sh" + "redact-secrets.ps1" +) +for script in "${REDACTION_SCRIPTS[@]}"; do + if [ ! -f "$plugin_dest/scripts/$script" ]; then + echo "ERROR: Redaction script was not installed: $plugin_dest/scripts/$script" >&2 + exit 1 + fi +done + +# Install skills (only if not already present). +SKILLS=( + "shepherd-task-from-assignment-to-ready" + "shepherd-task-from-ready-to-merged-to-base" + "shepherd-task-approve-workflows-and-wait-for-completion" + "shepherd-task-create-ignorance-reduction-plan" + "shepherd-task-create-post-mortem" + "shepherd-task-create-issues-from-plan" +) + +skills_installed=0 +skills_skipped=0 +for skill in "${SKILLS[@]}"; do + skill_src="$SOURCE_REPO/skills/$skill" + skill_dest="$COPILOT_HOME/skills/$skill" + + if [ ! -d "$skill_src" ]; then + echo "WARNING: Source skill not found: $skill_src" >&2 + continue + fi + + if [ -d "$skill_dest" ]; then + echo "Skipped ~/.copilot/skills/$skill (already exists)" + skills_skipped=$((skills_skipped + 1)) + else + mkdir -p "$skill_dest" + cp -R "$skill_src/." "$skill_dest/" + echo "Installed ~/.copilot/skills/$skill" + skills_installed=$((skills_installed + 1)) + fi +done + +echo "" +echo "Installation complete." +echo " Plugin: $plugin_dest" +echo " Skills: $skills_installed installed, $skills_skipped already present" +echo "" +echo "Interview script available at:" +echo " $plugin_dest/scripts/shepherd-task-interview-user-to-create-issues.sh" +echo "" +echo "Verify with: copilot skill list" diff --git a/plugins/shepherd-task/scripts/redact-secrets.ps1 b/plugins/shepherd-task/scripts/redact-secrets.ps1 new file mode 100644 index 0000000000..5fad437228 --- /dev/null +++ b/plugins/shepherd-task/scripts/redact-secrets.ps1 @@ -0,0 +1,107 @@ +<# +.SYNOPSIS + Redact secret-bearing fields from shepherd JSONL logs. + +.PARAMETER LogDirectory + A path relative to the current working directory containing .json* files. + +.EXAMPLE + ./redact-secrets.ps1 shepherd-tasks-20260803-1550 +#> + +param( + [Parameter(Mandatory = $true, Position = 0)] + [string]$LogDirectory +) + +$ErrorActionPreference = "Stop" + +if (-not (Test-Path -LiteralPath $LogDirectory -PathType Container)) { + throw "Log directory not found: $LogDirectory" +} + +$sensitiveKeyPattern = '(?i)(password|passwd|secret|token|api[-_]?key|authorization|credential|private[-_]?key|access[-_]?key|client[-_]?secret|connection[-_]?string)' +$contentKeyPattern = '(?i)^(content|encryptedContent|reasoningOpaque|arguments|result|error|prompt|toolRequests|userContent|assistantContent|toolCompleteResultContent)$' + +function Redact-String { + param([string]$Value) + + $result = $Value -replace '(?i)bearer\s+[A-Za-z0-9._~+/-]+', 'Bearer [REDACTED]' + $result -replace 'gh[opsu]_[A-Za-z0-9_]+|sk-[A-Za-z0-9_-]+|xox[baprs]-[A-Za-z0-9-]+|AIza[0-9A-Za-z_-]+', '[REDACTED]' +} + +function Redact-JsonValue { + param( + [AllowNull()] + [object]$Value, + [string]$Key + ) + + if ($null -eq $Value) { + return $null + } + if ($Key -match $sensitiveKeyPattern -or $Key -match $contentKeyPattern) { + return '[REDACTED]' + } + if ($Value -is [string]) { + return Redact-String $Value + } + if ($Value -is [System.Collections.IDictionary]) { + $result = [ordered]@{} + foreach ($entry in $Value.GetEnumerator()) { + $result[$entry.Key] = Redact-JsonValue $entry.Value ([string]$entry.Key) + } + return [pscustomobject]$result + } + if ($Value -is [System.Collections.IEnumerable] -and $Value -isnot [string]) { + return @($Value | ForEach-Object { Redact-JsonValue $_ $Key }) + } + if ($Value -is [pscustomobject]) { + $result = [ordered]@{} + foreach ($property in $Value.PSObject.Properties) { + $result[$property.Name] = Redact-JsonValue $property.Value $property.Name + } + return [pscustomobject]$result + } + return $Value +} + +function Redact-File { + param([System.IO.FileInfo]$File) + + $temporaryPath = "$($File.FullName).redact.$([guid]::NewGuid().ToString('N'))" + try { + $output = [System.Collections.Generic.List[string]]::new() + foreach ($line in [System.IO.File]::ReadLines($File.FullName)) { + if ([string]::IsNullOrWhiteSpace($line)) { + $output.Add('') + continue + } + try { + $json = $line | ConvertFrom-Json -Depth 100 + } catch { + throw "Invalid JSONL in $($File.FullName)" + } + $output.Add((Redact-JsonValue $json '' | ConvertTo-Json -Compress -Depth 100)) + } + [System.IO.File]::WriteAllLines($temporaryPath, $output) + $originalMode = $File.Mode + Move-Item -LiteralPath $temporaryPath -Destination $File.FullName -Force + $replacement = Get-Item -LiteralPath $File.FullName + $replacement.Mode = $originalMode + Write-Output "Redacted $($File.FullName)" + } catch { + Remove-Item -LiteralPath $temporaryPath -Force -ErrorAction SilentlyContinue + throw + } +} + +$files = Get-ChildItem -LiteralPath $LogDirectory -File -Recurse | + Where-Object { $_.Name -like '*.json*' } +if ($files.Count -eq 0) { + throw "No .json* files found in $LogDirectory" +} + +foreach ($file in $files) { + Redact-File $file +} diff --git a/plugins/shepherd-task/scripts/redact-secrets.sh b/plugins/shepherd-task/scripts/redact-secrets.sh new file mode 100755 index 0000000000..a3bdbf2cfa --- /dev/null +++ b/plugins/shepherd-task/scripts/redact-secrets.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +# +# redact-secrets.sh — Redact secret-bearing fields from shepherd JSONL logs. +# +# Usage: ./redact-secrets.sh +# log-directory is relative to the current working directory and contains +# .json* files produced by shepherd-task. + +set -euo pipefail + +LOG_DIR="${1:?Usage: $0 }" +if [[ ! -d "$LOG_DIR" ]]; then + echo "Log directory not found: $LOG_DIR" >&2 + exit 1 +fi + +JQ_FILTER=' + def sensitive_key: + test("(?i)(password|passwd|secret|token|api[-_]?key|authorization|credential|private[-_]?key|access[-_]?key|client[-_]?secret|connection[-_]?string)"); + def content_key: + test("(?i)^(content|encryptedContent|reasoningOpaque|arguments|result|error|prompt|toolRequests|userContent|assistantContent|toolCompleteResultContent)$"); + def scrub_string: + gsub("(?i)bearer[[:space:]]+[A-Za-z0-9._~+/-]+"; "Bearer [REDACTED]") + | gsub("gh[opsu]_[A-Za-z0-9_]+|sk-[A-Za-z0-9_-]+|xox[baprs]-[A-Za-z0-9-]+|AIza[0-9A-Za-z_-]+"; "[REDACTED]"); + def scrub: + if type == "object" then + with_entries( + if ((.key | sensitive_key) or (.key | content_key)) then + .value = "[REDACTED]" + else + .value |= scrub + end + ) + elif type == "array" then + map(scrub) + elif type == "string" then + scrub_string + else + . + end; + scrub +' + +redact_file() { + local file="$1" + local temp + temp=$(mktemp "${file}.redact.XXXXXX") + + if ! while IFS= read -r line || [[ -n "$line" ]]; do + if [[ -z "$line" ]]; then + printf '\n' + else + printf '%s\n' "$line" | jq -c "$JQ_FILTER" + fi + done <"$file" >"$temp"; then + rm -f "$temp" + echo "Invalid JSONL; left unchanged: $file" >&2 + exit 1 + fi + + chmod --reference="$file" "$temp" + mv "$temp" "$file" + echo "Redacted $file" +} + +mapfile -d '' files < <(find "$LOG_DIR" -type f -name '*.json*' -print0) +if [[ ${#files[@]} -eq 0 ]]; then + echo "No .json* files found in $LOG_DIR" >&2 + exit 1 +fi + +for file in "${files[@]}"; do + redact_file "$file" +done diff --git a/plugins/shepherd-task/scripts/shepherd-task-given-list.ps1 b/plugins/shepherd-task/scripts/shepherd-task-given-list.ps1 new file mode 100644 index 0000000000..943742e114 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-given-list.ps1 @@ -0,0 +1,127 @@ +<# +.SYNOPSIS + Shepherds a list of child Task issues end-to-end by invoking shepherd-task.ps1 for each. + +.DESCRIPTION + Takes a comma-separated list of issue numbers and invokes shepherd-task.ps1 + sequentially for each one. + +.PARAMETER TaskIssues + Comma-separated list of issue numbers (e.g., "1841,1842,1843"). + +.PARAMETER BaseBranch + The base branch the task PRs should target. This is never main. + +.PARAMETER Repo + Repository in OWNER/REPO format. +#> + +param( + [Parameter(Mandatory = $true, Position = 0)] + [string]$TaskIssues, + + [Parameter(Mandatory = $true, Position = 1)] + [string]$BaseBranch, + + [Parameter(Mandatory = $true, Position = 2)] + [string]$Repo +) + +$ErrorActionPreference = 'Stop' + +$usage = 'Usage: shepherd-task-given-list.ps1 ' + +if ($TaskIssues -notmatch '^[1-9][0-9]*(,[1-9][0-9]*)*$') { + throw "TASK_ISSUES must be a comma-separated list of positive issue numbers (for example: 2167,2168); received '$TaskIssues'. $usage" +} + +if ($BaseBranch -eq 'main') { + throw "BASE_BRANCH must not be 'main'. $usage" +} + +& git check-ref-format --branch $BaseBranch *> $null +if ($LASTEXITCODE -ne 0) { + throw "BASE_BRANCH is not a valid Git branch name; received '$BaseBranch'. $usage" +} + +if ($Repo -notmatch '^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$') { + throw "REPO must be in OWNER/REPO format; received '$Repo'. $usage" +} + +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$shepherdScript = Join-Path $scriptDir 'shepherd-task.ps1' +$scriptExitCode = 0 +$postMortemInvoked = $false + +$logDir = "shepherd-tasks-$(Get-Date -Format 'yyyyMMdd-HHmm')" +if (-not (Test-Path $logDir)) { + New-Item -ItemType Directory -Path $logDir | Out-Null +} +$logDirFull = (Resolve-Path $logDir).Path +Write-Output "Logging shepherd task files to $logDirFull" + +$issues = $TaskIssues -split ',' | ForEach-Object { $_.Trim() } | Where-Object { $_ -ne '' } + +function Invoke-PostMortemSkill { + param( + [Parameter(Mandatory = $true)] + [int]$ScriptExitCode + ) + + if ($postMortemInvoked) { return } + $script:postMortemInvoked = $true + + try { + $timestamp = Get-Date -Format 'yyyyMMdd-HHmm' + $postMortemPath = Join-Path $logDirFull "$timestamp-post-mortem.md" + $sessionSharePath = Join-Path $logDirFull "post-mortem-session-$timestamp.md" + $sessionJsonPath = Join-Path $logDirFull "post-mortem-session-$timestamp.json" + + $prompt = @" +Invoke skill ``shepherd-task-create-post-mortem`` with these inputs: + +- SHEPHERD_LOG_DIR: $logDirFull +- SCRIPT_EXIT_CODE: $ScriptExitCode +- TASK_ISSUES: $TaskIssues +- BASE_BRANCH: $BaseBranch +- REPO: $Repo + +Write the report to: +- OUTPUT_FILE: $postMortemPath +"@ + + Write-Output "[shepherd-task] Generating post-mortem report at: $postMortemPath" + $prompt | copilot --yolo --output-format json --share $sessionSharePath > $sessionJsonPath + if ($LASTEXITCODE -ne 0) { + Write-Warning "[shepherd-task] Post-mortem skill invocation exited with code $LASTEXITCODE." + } + } + catch { + Write-Warning "[shepherd-task] Post-mortem skill invocation failed: $($_.Exception.Message)" + } +} + +try { + foreach ($issue in $issues) { + Write-Output "=== Shepherding task issue #$issue ===" + & $shepherdScript -TaskIssue $issue -BaseBranch $BaseBranch -Repo $Repo -LogDir $logDir + $issueExitCode = $LASTEXITCODE + if ($issueExitCode -ne 0) { + $script:scriptExitCode = $issueExitCode + throw "shepherd-task.ps1 failed for issue #$issue (exit code $issueExitCode)" + } + } + + Write-Output "=== All tasks shepherded successfully ===" +} +catch { + if ($script:scriptExitCode -eq 0) { + $script:scriptExitCode = 1 + } + Write-Error $_ +} +finally { + Invoke-PostMortemSkill -ScriptExitCode $scriptExitCode +} + +exit $scriptExitCode diff --git a/plugins/shepherd-task/scripts/shepherd-task-given-list.sh b/plugins/shepherd-task/scripts/shepherd-task-given-list.sh new file mode 100755 index 0000000000..c0f5484cd5 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-given-list.sh @@ -0,0 +1,108 @@ +#!/usr/bin/env bash +# +# shepherd-task-given-list.sh — Shepherds a list of child Task issues end-to-end +# by invoking shepherd-task.sh sequentially for each one. +# +# Usage: ./shepherd-task-given-list.sh +# TASK_ISSUES: comma-separated list of issue numbers (e.g., "1841,1842,1843") +# BASE_BRANCH: the base branch the task PRs should target (never main) +# REPO: repository in OWNER/REPO format + +set -euo pipefail + +usage() { + echo "Usage: $0 " >&2 +} + +fail_input() { + echo "Error: $1" >&2 + usage + exit 1 +} + +if [[ $# -ne 3 ]]; then + fail_input "Expected exactly 3 arguments; received $#. TASK_ISSUES must be the first argument." +fi + +TASK_ISSUES="$1" +BASE_BRANCH="$2" +REPO="$3" + +if [[ ! "$TASK_ISSUES" =~ ^[1-9][0-9]*(,[1-9][0-9]*)*$ ]]; then + fail_input "TASK_ISSUES must be a comma-separated list of positive issue numbers (for example: 2167,2168); received '$TASK_ISSUES'." +fi + +if [[ "$BASE_BRANCH" == "main" ]]; then + fail_input "BASE_BRANCH must not be 'main'." +fi + +if ! git check-ref-format --branch "$BASE_BRANCH" >/dev/null 2>&1; then + fail_input "BASE_BRANCH is not a valid Git branch name; received '$BASE_BRANCH'." +fi + +if [[ ! "$REPO" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]]; then + fail_input "REPO must be in OWNER/REPO format; received '$REPO'." +fi + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" + +LOG_DIR="shepherd-tasks-$(date +%Y%m%d-%H%M)" +mkdir -p "$LOG_DIR" +LOG_DIR_FULL="$(cd "$LOG_DIR" && pwd)" +echo "Logging shepherd task files to $LOG_DIR_FULL" + +IFS=',' read -ra ISSUES <<< "$TASK_ISSUES" + +invoke_post_mortem_on_exit() { + local script_exit=$? + local timestamp post_mortem_path share_path json_path prompt pm_exit + + # Prevent duplicate invocation if EXIT trap is triggered more than once. + if [[ "${POST_MORTEM_INVOKED:-0}" == "1" ]]; then + return + fi + POST_MORTEM_INVOKED=1 + + # Always attempt post-mortem generation once shepherding log directory exists. + if [[ -z "${LOG_DIR_FULL:-}" ]]; then + return + fi + + timestamp="$(date +%Y%m%d-%H%M)" + post_mortem_path="$LOG_DIR_FULL/${timestamp}-post-mortem.md" + share_path="$LOG_DIR_FULL/post-mortem-session-${timestamp}.md" + json_path="$LOG_DIR_FULL/post-mortem-session-${timestamp}.json" + + prompt="Invoke skill \`shepherd-task-create-post-mortem\` with these inputs: + +- SHEPHERD_LOG_DIR: $LOG_DIR_FULL +- SCRIPT_EXIT_CODE: $script_exit +- TASK_ISSUES: $TASK_ISSUES +- BASE_BRANCH: $BASE_BRANCH +- REPO: $REPO + +Write the report to: +- OUTPUT_FILE: $post_mortem_path" + + echo "[shepherd-task] Generating post-mortem report at: $post_mortem_path" + set +e + printf '%s' "$prompt" | copilot --yolo --output-format json --share "$share_path" > "$json_path" + pm_exit=$? + set -e + + if [[ $pm_exit -ne 0 ]]; then + echo "[shepherd-task] WARNING: post-mortem skill invocation exited with code $pm_exit" >&2 + fi +} + +trap 'invoke_post_mortem_on_exit' EXIT + +for issue in "${ISSUES[@]}"; do + issue="$(echo "$issue" | tr -d '[:space:]')" + [[ -z "$issue" ]] && continue + echo "=== Shepherding task issue #${issue} ===" + "$SCRIPT_DIR/shepherd-task.sh" "$issue" "$BASE_BRANCH" "$REPO" "$LOG_DIR" +done + +echo "=== All tasks shepherded successfully ===" +exit 0 diff --git a/plugins/shepherd-task/scripts/shepherd-task-inspect-json.ps1 b/plugins/shepherd-task/scripts/shepherd-task-inspect-json.ps1 new file mode 100644 index 0000000000..5ad36ebb14 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-inspect-json.ps1 @@ -0,0 +1,53 @@ +<# +.SYNOPSIS + Inspect a copilot JSON log file, showing the last N meaningful events. + +.PARAMETER JsonFile + Path to the JSON log file (relative or absolute). + +.PARAMETER Count + Number of messages to show (default: 20). +#> + +param( + [Parameter(Mandatory = $true, Position = 0)] + [string]$JsonFile, + + [Parameter(Mandatory = $false, Position = 1)] + [int]$Count = 20 +) + +$events = Get-Content $JsonFile | + Where-Object { $_ -notmatch '"ephemeral":true' -and $_ -notmatch '"tool.execution_partial_result"' } | + ForEach-Object { + try { $_ | ConvertFrom-Json } catch { $null } + } | + Where-Object { $_ -ne $null } | + Select-Object -Last $Count + +$events | ForEach-Object { + $evt = $_ + $ts = if ($evt.timestamp) { ([datetime]$evt.timestamp).ToString("HH:mm:ss") } else { "--------" } + $summary = switch -Wildcard ($evt.type) { + "user.message" { $txt = $evt.data.content; "USER: " + $txt.Substring(0, [Math]::Min(120, $txt.Length)) } + "assistant.message" { + if ($evt.data.content) { + $txt = $evt.data.content; "ASST: " + $txt.Substring(0, [Math]::Min(120, $txt.Length)) + } else { + $names = ($evt.data.toolRequests | ForEach-Object { $_.name }) -join ", " + "ASST: [tool calls: $names]" + } + } + "tool.execution_start" { + $args_summary = ($evt.data.arguments | ConvertTo-Json -Compress -Depth 1) -replace '[\r\n]','' + if ($args_summary.Length -gt 80) { $args_summary = $args_summary.Substring(0,77) + "..." } + "TOOL> " + $evt.data.toolName + " :: " + $args_summary + } + "tool.execution_complete" { "TOOL< " + $(if ($evt.data.success) {"OK"} else {"FAIL"}) } + "assistant.reasoning" { $txt = $evt.data.content; "THINK: " + $txt.Substring(0, [Math]::Min(120, $txt.Length)) } + "assistant.turn_start" { $null } + "assistant.turn_end" { $null } + default { $evt.type } + } + if ($summary) { "$ts | $summary" } + } diff --git a/plugins/shepherd-task/scripts/shepherd-task-inspect-json.sh b/plugins/shepherd-task/scripts/shepherd-task-inspect-json.sh new file mode 100755 index 0000000000..59b1036730 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-inspect-json.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# +# shepherd-task-inspect-json.sh — Show the last N meaningful events from a copilot JSON log. +# +# Usage: ./shepherd-task-inspect-json.sh [count] +# json-file: path to the JSON log file +# count: number of messages to show (default: 20) + +set -euo pipefail + +JSON_FILE="${1:?Usage: $0 [count]}" +COUNT="${2:-20}" + +jq -Rrc ' + select(contains("\"ephemeral\":true") | not) + | select(contains("\"tool.execution_partial_result\"") | not) + | fromjson? +' "$JSON_FILE" | tail -n "$COUNT" | while IFS= read -r line; do + ts=$(echo "$line" | jq -r 'if .timestamp then .timestamp[11:19] else "--------" end') + type=$(echo "$line" | jq -r '.type') + + case "$type" in + user.message) + content=$(echo "$line" | jq -r '(.data.content // "")[0:120]') + echo "$ts | USER: $content" + ;; + assistant.message) + content=$(echo "$line" | jq -r 'if (.data.content // "") != "" then .data.content[0:120] else "[tool calls: " + ([.data.toolRequests[]?.name] | join(", ")) + "]" end') + echo "$ts | ASST: $content" + ;; + tool.execution_start) + tool=$(echo "$line" | jq -r '.data.toolName') + arguments=$(echo "$line" | jq -c '.data.arguments' | tr -d '\r\n') + if [[ ${#arguments} -gt 80 ]]; then + arguments="${arguments:0:77}..." + fi + echo "$ts | TOOL> $tool :: $arguments" + ;; + tool.execution_complete) + status=$(echo "$line" | jq -r 'if .data.success then "OK" else "FAIL" end') + echo "$ts | TOOL< $status" + ;; + assistant.reasoning) + content=$(echo "$line" | jq -r '(.data.content // "")[0:120]') + echo "$ts | THINK: $content" + ;; + assistant.turn_start|assistant.turn_end) + ;; + *) + echo "$ts | $type" + ;; + esac +done diff --git a/plugins/shepherd-task/scripts/shepherd-task-inspect-otel-token-summary.ps1 b/plugins/shepherd-task/scripts/shepherd-task-inspect-otel-token-summary.ps1 new file mode 100644 index 0000000000..bd46d2419c --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-inspect-otel-token-summary.ps1 @@ -0,0 +1,96 @@ +<# +.SYNOPSIS + Summarize token usage from OTel JSONL exports produced by shepherd-task. + +.PARAMETER Target + Path to a single OTel JSONL file or a directory containing *.jsonl files. + +.EXAMPLE + ./shepherd-task-inspect-otel-token-summary.ps1 ./shepherd-tasks-20260708-1244 + ./shepherd-task-inspect-otel-token-summary.ps1 ./phase1-otel-20260708-1244-4.jsonl +#> + +param( + [Parameter(Mandatory = $true, Position = 0)] + [string]$Target +) + +$ErrorActionPreference = "Stop" + +function Get-TokenSummary { + param([string]$FilePath) + + $content = Get-Content $FilePath -Raw + $lines = Get-Content $FilePath | Where-Object { $_.Trim() -ne "" } + + $inputTokens = 0 + $outputTokens = 0 + $llmCalls = 0 + + foreach ($line in $lines) { + try { + $obj = $line | ConvertFrom-Json -Depth 20 + } catch { + continue + } + + # Walk resourceSpans -> scopeSpans -> spans + if ($obj.resourceSpans) { + foreach ($rs in $obj.resourceSpans) { + foreach ($ss in $rs.scopeSpans) { + foreach ($span in $ss.spans) { + if ($span.name -match "^chat ") { + $llmCalls++ + } + foreach ($attr in $span.attributes) { + if ($attr.key -eq "gen_ai.usage.input_tokens") { + $val = if ($attr.value.intValue) { $attr.value.intValue } elseif ($attr.value.stringValue) { [int]$attr.value.stringValue } else { 0 } + $inputTokens += $val + } + if ($attr.key -eq "gen_ai.usage.output_tokens") { + $val = if ($attr.value.intValue) { $attr.value.intValue } elseif ($attr.value.stringValue) { [int]$attr.value.stringValue } else { 0 } + $outputTokens += $val + } + } + } + } + } + } + } + + return @{ + InputTokens = $inputTokens + OutputTokens = $outputTokens + LLMCalls = $llmCalls + } +} + +Write-Output "=== OTel Token Usage Summary ===" +Write-Output "" + +$totalInput = 0 +$totalOutput = 0 +$totalCalls = 0 + +if (Test-Path $Target -PathType Container) { + $files = Get-ChildItem -Path $Target -Filter "*.jsonl" + if ($files.Count -eq 0) { + Write-Output "No .jsonl files found in $Target" + exit 1 + } + foreach ($f in $files) { + $summary = Get-TokenSummary -FilePath $f.FullName + $name = $f.Name + Write-Output ("{0,-50} {1,8} input {2,8} output {3,4} calls" -f $name, $summary.InputTokens, $summary.OutputTokens, $summary.LLMCalls) + $totalInput += $summary.InputTokens + $totalOutput += $summary.OutputTokens + $totalCalls += $summary.LLMCalls + } + Write-Output "" + Write-Output "--- TOTALS ---" + Write-Output ("{0,-50} {1,8} input {2,8} output {3,4} calls" -f "ALL FILES", $totalInput, $totalOutput, $totalCalls) +} else { + $summary = Get-TokenSummary -FilePath $Target + $name = Split-Path $Target -Leaf + Write-Output ("{0,-50} {1,8} input {2,8} output {3,4} calls" -f $name, $summary.InputTokens, $summary.OutputTokens, $summary.LLMCalls) +} diff --git a/plugins/shepherd-task/scripts/shepherd-task-inspect-otel-token-summary.sh b/plugins/shepherd-task/scripts/shepherd-task-inspect-otel-token-summary.sh new file mode 100755 index 0000000000..97d09f54d3 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-inspect-otel-token-summary.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# +# shepherd-task-inspect-otel-token-summary.sh — Summarize token usage from OTel JSONL exports. +# +# Usage: ./shepherd-task-inspect-otel-token-summary.sh +# otel-jsonl-file: path to a single OTel JSONL file +# directory: path to a log directory; processes all *.jsonl files within + +set -euo pipefail + +TARGET="${1:?Usage: $0 }" + +summarize_file() { + local file="$1" + local basename + basename=$(basename "$file") + + local input_tokens output_tokens llm_calls + input_tokens=$(jq -r ' + [.. | objects | select(.key == "gen_ai.usage.input_tokens") | .value.intValue // .value.stringValue // 0 | tonumber] | add // 0 + ' "$file" 2>/dev/null || echo 0) + + output_tokens=$(jq -r ' + [.. | objects | select(.key == "gen_ai.usage.output_tokens") | .value.intValue // .value.stringValue // 0 | tonumber] | add // 0 + ' "$file" 2>/dev/null || echo 0) + + llm_calls=$(jq -r ' + [.. | objects | select(.name? // "" | test("^chat ")) ] | length + ' "$file" 2>/dev/null || echo 0) + + printf "%-50s %8s input %8s output %4s calls\n" "$basename" "$input_tokens" "$output_tokens" "$llm_calls" +} + +echo "=== OTel Token Usage Summary ===" +echo "" + +total_input=0 +total_output=0 +total_calls=0 + +if [[ -d "$TARGET" ]]; then + files=("$TARGET"/*.jsonl) + if [[ ${#files[@]} -eq 0 || ! -e "${files[0]}" ]]; then + echo "No .jsonl files found in $TARGET" + exit 1 + fi + for f in "${files[@]}"; do + summarize_file "$f" + input=$(jq -r '[.. | objects | select(.key == "gen_ai.usage.input_tokens") | .value.intValue // .value.stringValue // 0 | tonumber] | add // 0' "$f" 2>/dev/null || echo 0) + output=$(jq -r '[.. | objects | select(.key == "gen_ai.usage.output_tokens") | .value.intValue // .value.stringValue // 0 | tonumber] | add // 0' "$f" 2>/dev/null || echo 0) + calls=$(jq -r '[.. | objects | select(.name? // "" | test("^chat ")) ] | length' "$f" 2>/dev/null || echo 0) + total_input=$((total_input + input)) + total_output=$((total_output + output)) + total_calls=$((total_calls + calls)) + done + echo "" + echo "--- TOTALS ---" + printf "%-50s %8s input %8s output %4s calls\n" "ALL FILES" "$total_input" "$total_output" "$total_calls" +else + summarize_file "$TARGET" +fi diff --git a/plugins/shepherd-task/scripts/shepherd-task-interview-user-to-create-issues.ps1 b/plugins/shepherd-task/scripts/shepherd-task-interview-user-to-create-issues.ps1 new file mode 100644 index 0000000000..7574662f58 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-interview-user-to-create-issues.ps1 @@ -0,0 +1,128 @@ +<# +.SYNOPSIS + Interviews the user for 11 inputs to the shepherd-task-create-issues-from-plan + skill and writes timestamped prompt and invocation artifacts. + +.DESCRIPTION + Asks the user each required input interactively, then writes a prompt file named + YYYYMMDD-HHMM-invoke-shepherd-task-create-issues-from-plan-skill.md inside a + persistent log directory. It also writes a PowerShell script that invokes the prompt + with JSON, session-share, and OTel logging enabled. + +.EXAMPLE + .\shepherd-task-interview-user-to-create-issues.ps1 +#> + +$ErrorActionPreference = 'Stop' + +function Read-Required { + param( + [string]$Prompt, + [string]$Default = '' + ) + $displayPrompt = if ($Default) { "$Prompt [$Default]" } else { $Prompt } + do { + $value = Read-Host $displayPrompt + if ([string]::IsNullOrWhiteSpace($value) -and $Default) { + $value = $Default + } + if ([string]::IsNullOrWhiteSpace($value)) { + Write-Host " This input is required." -ForegroundColor Yellow + } + } while ([string]::IsNullOrWhiteSpace($value)) + return $value.Trim() +} + +Write-Host "=== shepherd-task-create-issues-from-plan — Input Interview ===" -ForegroundColor Cyan +Write-Host "" + +$REPO = Read-Required "1/11 REPO (OWNER/REPO format, e.g. github/copilot-sdk)" + +$BASE_BRANCH = Read-Required "2/11 BASE_BRANCH (non-main topic branch, e.g. edburns/1917-java-embed-cli)" + +$PARENT_ISSUE = Read-Required "3/11 PARENT_ISSUE (positive integer issue number only, e.g. 123)" + +$PLAN_DIRECTORY = Read-Required "4/11 PLAN_DIRECTORY (repo-relative path to directory containing plan and spikes)" + +$PLAN_FILE_NAME = Read-Required "5/11 PLAN_FILE_NAME (name of the plan file within that directory)" + +Write-Host "" +Write-Host " Hint: copy the exact markdown heading from the plan." -ForegroundColor DarkGray +$QUESTIONS_SECTION = Read-Required "6/11 QUESTIONS_SECTION (exact heading of the resolved questions section)" + +$IMPLEMENTATION_SECTION = Read-Required "7/11 IMPLEMENTATION_SECTION (exact heading of the implementation/build-order section)" + +Write-Host "" +Write-Host " Hint: provide full GitHub issue URLs separated by commas." -ForegroundColor DarkGray +$EXAMPLE_ISSUES = Read-Required "8/11 EXAMPLE_ISSUES (full GitHub issue URLs whose style to follow)" + +$BASE_REMOTE = Read-Required "9/11 BASE_REMOTE (git remote name, e.g. upstream or origin)" "upstream" + +$ISSUE_TYPE = Read-Required "10/11 ISSUE_TYPE (GitHub issue type for children)" "Task" + +Write-Host "" +Write-Host " Hint: repo-relative paths or constraints; comma-separated." -ForegroundColor DarkGray +$SUPPORTING_ARTIFACTS = Read-Required "11/11 SUPPORTING_ARTIFACTS (paths to spikes, screenshots, etc.)" "$PLAN_DIRECTORY" + +# Build the prompt file. +$timestamp = Get-Date -Format 'yyyyMMdd-HHmm' +$logDir = Join-Path (Get-Location) "shepherd-task-$timestamp" +New-Item -ItemType Directory -Path $logDir -Force | Out-Null +$logDirFull = (Resolve-Path $logDir).Path +$outFile = Join-Path $logDirFull "$timestamp-invoke-shepherd-task-create-issues-from-plan-skill.md" +$invocationFile = Join-Path $logDirFull "$timestamp-invoke-shepherd-task-create-issues-from-plan-skill.ps1" + +$body = @" +Invoke skill ``shepherd-task-create-issues-from-plan`` with these inputs: + +- REPO: $REPO +- BASE_BRANCH: $BASE_BRANCH +- PARENT_ISSUE: $PARENT_ISSUE +- PLAN_DIRECTORY: $PLAN_DIRECTORY +- PLAN_FILE_NAME: $PLAN_FILE_NAME +- QUESTIONS_SECTION: $QUESTIONS_SECTION +- IMPLEMENTATION_SECTION: $IMPLEMENTATION_SECTION +- EXAMPLE_ISSUES: $EXAMPLE_ISSUES +- BASE_REMOTE: $BASE_REMOTE +- ISSUE_TYPE: $ISSUE_TYPE +- SUPPORTING_ARTIFACTS: $SUPPORTING_ARTIFACTS +- LOG_DIRECTORY: $logDirFull +"@ + +Set-Content -Path $outFile -Value $body -Encoding utf8NoBOM + +$escapedOutFile = $outFile.Replace("'", "''") +$escapedLogDir = $logDirFull.Replace("'", "''") +$command = @' +$timestamp = '__TIMESTAMP__' +$logDirFull = '__LOG_DIRECTORY__' +New-Item -ItemType Directory -Path $logDirFull -Force | Out-Null +$sessionSharePath = Join-Path $logDirFull "create-issues-session-$timestamp.md" +$sessionJsonPath = Join-Path $logDirFull "create-issues-session-$timestamp.json" +$sessionOtelPath = Join-Path $logDirFull "create-issues-otel-$timestamp.jsonl" +$promptPath = '__PROMPT_PATH__' +$prompt = Get-Content $promptPath -Raw +Write-Output "[shepherd-task] Logging create-issues run to: $logDirFull" +$env:COPILOT_OTEL_FILE_EXPORTER_PATH = $sessionOtelPath +$copilotExit = 0 +try { + $prompt | copilot --yolo --output-format json --share $sessionSharePath > $sessionJsonPath + $copilotExit = $LASTEXITCODE +} +finally { + Remove-Item Env:\COPILOT_OTEL_FILE_EXPORTER_PATH -ErrorAction SilentlyContinue +} +if ($copilotExit -ne 0) { + Write-Error "[shepherd-task] FAILED: copilot exited with code $copilotExit" +} +else { + Write-Output "[shepherd-task] Create-issues session complete." +} +'@.Replace('__TIMESTAMP__', $timestamp).Replace('__LOG_DIRECTORY__', $escapedLogDir).Replace('__PROMPT_PATH__', $escapedOutFile) + +Set-Content -Path $invocationFile -Value $command -Encoding utf8NoBOM + +Write-Host "" +Write-Host "Artifacts written:" -ForegroundColor Green +Write-Host " Prompt: $outFile" +Write-Host " Script: $invocationFile" diff --git a/plugins/shepherd-task/scripts/shepherd-task-interview-user-to-create-issues.sh b/plugins/shepherd-task/scripts/shepherd-task-interview-user-to-create-issues.sh new file mode 100755 index 0000000000..5c1ee10d43 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-interview-user-to-create-issues.sh @@ -0,0 +1,113 @@ +#!/usr/bin/env bash +# +# shepherd-task-interview-user-to-create-issues.sh — Interviews the user for 11 inputs +# to the shepherd-task-create-issues-from-plan skill and writes a timestamped prompt +# and invocation script inside a persistent log directory. +# +# Usage: ./shepherd-task-interview-user-to-create-issues.sh + +set -euo pipefail + +read_required() { + local prompt="$1" + local default="${2:-}" + local value="" + + if [[ -n "$default" ]]; then + prompt="$prompt [$default]" + fi + + while true; do + printf '%s: ' "$prompt" >&2 + IFS= read -r value + value="${value#"${value%%[![:space:]]*}"}" + value="${value%"${value##*[![:space:]]}"}" + if [[ -z "$value" && -n "$default" ]]; then + value="$default" + fi + if [[ -n "$value" ]]; then + echo "$value" + return + fi + echo " This input is required." >&2 + done +} + +echo "=== shepherd-task-create-issues-from-plan — Input Interview ===" +echo "" + +REPO=$(read_required "1/11 REPO (OWNER/REPO format, e.g. github/copilot-sdk)") + +BASE_BRANCH=$(read_required "2/11 BASE_BRANCH (non-main topic branch, e.g. edburns/1917-java-embed-cli)") + +PARENT_ISSUE=$(read_required "3/11 PARENT_ISSUE (positive integer issue number only, e.g. 123)") + +PLAN_DIRECTORY=$(read_required "4/11 PLAN_DIRECTORY (repo-relative path to directory containing plan and spikes)") + +PLAN_FILE_NAME=$(read_required "5/11 PLAN_FILE_NAME (name of the plan file within that directory)") + +echo "" +echo " Hint: copy the exact markdown heading from the plan." +QUESTIONS_SECTION=$(read_required "6/11 QUESTIONS_SECTION (exact heading of the resolved questions section)") + +IMPLEMENTATION_SECTION=$(read_required "7/11 IMPLEMENTATION_SECTION (exact heading of the implementation/build-order section)") + +echo "" +echo " Hint: provide full GitHub issue URLs separated by commas." +EXAMPLE_ISSUES=$(read_required "8/11 EXAMPLE_ISSUES (full GitHub issue URLs whose style to follow)") + +BASE_REMOTE=$(read_required "9/11 BASE_REMOTE (git remote name, e.g. upstream or origin)" "upstream") + +ISSUE_TYPE=$(read_required "10/11 ISSUE_TYPE (GitHub issue type for children)" "Task") + +echo "" +echo " Hint: repo-relative paths or constraints; comma-separated." +SUPPORTING_ARTIFACTS=$(read_required "11/11 SUPPORTING_ARTIFACTS (paths to spikes, screenshots, etc.)" "$PLAN_DIRECTORY") + +# Build the prompt file. +timestamp=$(date +%Y%m%d-%H%M) +log_dir="$(pwd)/shepherd-task-${timestamp}" +mkdir -p "$log_dir" +log_dir_full="$(cd "$log_dir" && pwd)" +out_file="$log_dir_full/${timestamp}-invoke-shepherd-task-create-issues-from-plan-skill.md" +invocation_file="$log_dir_full/${timestamp}-invoke-shepherd-task-create-issues-from-plan-skill.sh" + +cat > "$out_file" < "$session_json_path"\n' + printf 'copilot_exit=$?\n' + printf 'unset COPILOT_OTEL_FILE_EXPORTER_PATH\n' + printf 'if [[ $copilot_exit -ne 0 ]]; then echo "[shepherd-task] FAILED: copilot exited with code $copilot_exit" >&2; else echo "[shepherd-task] Create-issues session complete."; fi\n' +} > "$invocation_file" +chmod +x "$invocation_file" + +echo "" +echo "Artifacts written:" +echo " Prompt: $out_file" +echo " Script: $invocation_file" diff --git a/plugins/shepherd-task/scripts/shepherd-task-monitor.ps1 b/plugins/shepherd-task/scripts/shepherd-task-monitor.ps1 new file mode 100644 index 0000000000..ec56cdae90 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-monitor.ps1 @@ -0,0 +1,301 @@ +<# +.SYNOPSIS + Monitors an ongoing shepherd-task run by watching its log directory and polling GitHub. + +.DESCRIPTION + Watches a shepherd-task log directory for new files and polls the GitHub API + for real-time PR/review/CI status. Alerts on failures, stalls, and completion. + + Run this in a SEPARATE terminal while shepherd-task-given-list.ps1 is running. + +.PARAMETER LogDir + Path to the shepherd-tasks log directory (e.g., shepherd-tasks-20260718-1648). + +.PARAMETER Repo + Repository in OWNER/REPO format. + +.PARAMETER PollInterval + Seconds between polls (default: 30). + +.EXAMPLE + .\shepherd-task-monitor.ps1 ..\shepherd-tasks-20260718-1827 edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk +#> + +param( + [Parameter(Mandatory = $true, Position = 0)] + [string]$LogDir, + + [Parameter(Mandatory = $true, Position = 1)] + [string]$Repo, + + [Parameter(Mandatory = $false, Position = 2)] + [int]$PollInterval = 30 +) + +$ErrorActionPreference = "Continue" + +if (-not (Test-Path $LogDir)) { + Write-Error "Log directory not found: $LogDir" + exit 1 +} + +$LogDirFull = (Resolve-Path $LogDir).Path +Write-Host "=== Shepherd Task Monitor ===" -ForegroundColor Cyan +Write-Host "Log directory: $LogDirFull" +Write-Host "Repository: $Repo" +Write-Host "Poll interval: ${PollInterval}s" +Write-Host "Press Ctrl+C to stop." +Write-Host "" + +# --- State tracking --- +$knownFiles = @{} +$issueState = @{} # issue# -> { phase1: done/running/none, phase2: done/running/none, pr: #, status: ... } +$lastActivity = Get-Date +$staleMinutes = 20 + +function Get-Timestamp { + return (Get-Date -Format 'HH:mm:ss') +} + +function Write-Monitor { + param([string]$Message, [string]$Color = "White") + Write-Host "[$(Get-Timestamp)] $Message" -ForegroundColor $Color +} + +function Write-Alert { + param([string]$Message) + Write-Host "[$(Get-Timestamp)] ⚠️ $Message" -ForegroundColor Yellow +} + +function Write-Success { + param([string]$Message) + Write-Host "[$(Get-Timestamp)] ✅ $Message" -ForegroundColor Green +} + +function Write-Failure { + param([string]$Message) + Write-Host "[$(Get-Timestamp)] ❌ $Message" -ForegroundColor Red +} + +# Extract issue number from a log filename like "phase1-task-20260718-1650-33.md" +function Get-IssueFromFilename { + param([string]$Name) + if ($Name -match '-(\d+)\.(md|json|jsonl)$') { + return $Matches[1] + } + return $null +} + +# Extract phase from a log filename +function Get-PhaseFromFilename { + param([string]$Name) + if ($Name -match '^(phase[12])') { + return $Matches[1] + } + return $null +} + +# Find PR linked to an issue +function Find-PR { + param([string]$Issue) + + # Strategy A: issue timeline + $pr = gh api "/repos/$Repo/issues/$Issue/timeline" ` + --jq '.[] | select(.event == "cross-referenced") | select(.source.issue.pull_request != null) | select(.source.issue.state == "open") | .source.issue.number' 2>$null | + Select-Object -First 1 + + if ($pr) { return $pr.Trim() } + + # Strategy B: PR body search + $pr = gh pr list -R $Repo --state all --json number,body ` + --jq ".[] | select(.body | test(`"#$Issue`")) | .number" 2>$null | + Select-Object -First 1 + + if ($pr) { return $pr.Trim() } + + return $null +} + +# Get PR status summary +function Get-PRStatus { + param([string]$PRNumber) + + $info = gh pr view $PRNumber -R $Repo --json state,isDraft,baseRefName,headRefName,mergeable,reviews,reviewRequests 2>$null | ConvertFrom-Json + if (-not $info) { return $null } + + $reviewCount = 0 + $copilotReviews = @() + if ($info.reviews) { + $copilotReviews = @($info.reviews | Where-Object { + $_.author.login -match 'copilot|copilot-pull-request-reviewer' + }) + $reviewCount = $copilotReviews.Count + } + + # Check for unresolved review comments + $comments = gh api "/repos/$Repo/pulls/$PRNumber/comments" ` + --jq '.[] | select(.user.login | test("copilot-pull-request-reviewer|Copilot")) | .id' 2>$null + $commentCount = if ($comments) { @($comments).Count } else { 0 } + + # Check CI + $ciFailures = gh pr checks $PRNumber -R $Repo --json name,state,bucket ` + --jq '.[] | select(.bucket == "fail") | select(.name != "No remove-before-merge directories") | .name' 2>$null + $ciStatus = if ([string]::IsNullOrWhiteSpace($ciFailures)) { "passing" } else { "failing" } + + return @{ + State = $info.state + IsDraft = $info.isDraft + Base = $info.baseRefName + Mergeable = $info.mergeable + ReviewRounds = $reviewCount + Comments = $commentCount + CI = $ciStatus + } +} + +# --- Main monitor loop --- +$iteration = 0 +while ($true) { + $iteration++ + $now = Get-Date + $newFiles = @() + + # Scan log directory for new or changed files + $currentFiles = Get-ChildItem $LogDirFull -File -ErrorAction SilentlyContinue + foreach ($file in $currentFiles) { + $key = $file.Name + if (-not $knownFiles.ContainsKey($key)) { + $knownFiles[$key] = $file.Length + $newFiles += $file + $lastActivity = $now + } + elseif ($knownFiles[$key] -ne $file.Length) { + $knownFiles[$key] = $file.Length + $lastActivity = $now + } + } + + # Process new files + foreach ($file in $newFiles) { + $issue = Get-IssueFromFilename $file.Name + $phase = Get-PhaseFromFilename $file.Name + $ext = [System.IO.Path]::GetExtension($file.Name) + + if ($issue -and $phase -and $ext -eq '.md') { + # .md means the session exported — phase complete + if (-not $issueState.ContainsKey($issue)) { + $issueState[$issue] = @{ phase1 = 'none'; phase2 = 'none'; pr = $null; status = 'unknown' } + } + + if ($phase -eq 'phase1') { + $issueState[$issue].phase1 = 'done' + Write-Monitor "Issue #$($issue): Phase 1 session exported ($($file.Name))" + + # Try to find the PR + $pr = Find-PR $issue + if ($pr) { + $issueState[$issue].pr = $pr + Write-Monitor "Issue #$($issue): Linked to PR #$($pr)" + } + } + elseif ($phase -eq 'phase2') { + $issueState[$issue].phase2 = 'done' + Write-Monitor "Issue #$($issue): Phase 2 session exported ($($file.Name))" + } + } + elseif ($issue -and $phase -and $ext -eq '.json') { + # .json appearing means the copilot process exited (stdout captured) + if (-not $issueState.ContainsKey($issue)) { + $issueState[$issue] = @{ phase1 = 'none'; phase2 = 'none'; pr = $null; status = 'unknown' } + } + } + } + + # For issues with known PRs, poll GitHub for status + $activeIssue = $null + foreach ($issue in $issueState.Keys) { + $state = $issueState[$issue] + $pr = $state.pr + + # Detect the currently active issue (has a PR but phase2 not done) + if ($pr -and $state.phase2 -ne 'done') { + $activeIssue = $issue + } + + # If phase2 just completed, check final PR state + if ($pr -and $state.phase2 -eq 'done' -and $state.status -ne 'merged' -and $state.status -ne 'failed') { + $prStatus = Get-PRStatus $pr + if ($prStatus) { + if ($prStatus.State -eq 'MERGED') { + $state.status = 'merged' + Write-Success "Issue #$($issue): PR #$($pr) MERGED ($($prStatus.ReviewRounds) review rounds)" + } + elseif ($prStatus.State -eq 'CLOSED') { + $state.status = 'failed' + Write-Failure "Issue #$($issue): PR #$($pr) CLOSED (not merged)" + } + else { + $state.status = 'open' + Write-Failure "Issue #$($issue): PR #$($pr) still OPEN after Phase 2 exited" + } + } + } + } + + # Poll active PR for real-time status + if ($activeIssue) { + $pr = $issueState[$activeIssue].pr + if ($pr) { + $prStatus = Get-PRStatus $pr + if ($prStatus) { + $statusLine = "Issue #$($activeIssue) PR #$($pr): state=$($prStatus.State) draft=$($prStatus.IsDraft) " + + "reviews=$($prStatus.ReviewRounds) comments=$($prStatus.Comments) CI=$($prStatus.CI)" + + if ($prStatus.State -eq 'MERGED') { + Write-Success $statusLine + $issueState[$activeIssue].status = 'merged' + } + elseif ($prStatus.CI -eq 'failing') { + Write-Alert $statusLine + } + else { + Write-Monitor $statusLine -Color "Gray" + } + } + } + } + # If no active issue detected yet but files exist, try to find one + elseif ($iteration -eq 1 -or $newFiles.Count -gt 0) { + foreach ($issue in $issueState.Keys) { + $state = $issueState[$issue] + if (-not $state.pr) { + $pr = Find-PR $issue + if ($pr) { + $state.pr = $pr + Write-Monitor "Issue #$($issue): Found PR #$($pr)" + } + } + } + } + + # Stale detection + $staleDuration = ($now - $lastActivity).TotalMinutes + if ($staleDuration -gt $staleMinutes) { + Write-Alert "No activity for $([Math]::Round($staleDuration, 0)) minutes — shepherd may be stalled or waiting for CCRA" + } + + # Periodic summary (every 5 iterations) + if ($iteration % 5 -eq 0 -and $issueState.Count -gt 0) { + Write-Host "" + Write-Host "[$(Get-Timestamp)] === Summary ===" -ForegroundColor Cyan + foreach ($issue in $issueState.Keys | Sort-Object) { + $s = $issueState[$issue] + $prLabel = if ($s.pr) { "PR #$($s.pr)" } else { "no PR" } + $statusLabel = $s.status + Write-Host " Issue #$($issue) : P1=$($s.phase1) P2=$($s.phase2) $prLabel status=$statusLabel" + } + Write-Host "" + } + + Start-Sleep -Seconds $PollInterval +} diff --git a/plugins/shepherd-task/scripts/shepherd-task-monitor.sh b/plugins/shepherd-task/scripts/shepherd-task-monitor.sh new file mode 100755 index 0000000000..2fedf8649e --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task-monitor.sh @@ -0,0 +1,249 @@ +#!/usr/bin/env bash +# +# shepherd-task-monitor.sh — Monitors an ongoing shepherd-task run. +# +# Watches a shepherd-task log directory for new files and polls GitHub +# for real-time PR/review/CI status. Alerts on failures, stalls, and completion. +# +# Run this in a SEPARATE terminal while shepherd-task-given-list.sh is running. +# +# Usage: ./shepherd-task-monitor.sh [POLL_INTERVAL] + +set -uo pipefail + +LOG_DIR="${1:?Usage: $0 [POLL_INTERVAL]}" +REPO="${2:?Usage: $0 [POLL_INTERVAL]}" +POLL_INTERVAL="${3:-30}" +STALE_MINUTES=20 + +if [[ ! -d "$LOG_DIR" ]]; then + echo "Error: Log directory not found: $LOG_DIR" >&2 + exit 1 +fi + +LOG_DIR_FULL="$(cd "$LOG_DIR" && pwd)" + +echo "=== Shepherd Task Monitor ===" +echo "Log directory: $LOG_DIR_FULL" +echo "Repository: $REPO" +echo "Poll interval: ${POLL_INTERVAL}s" +echo "Press Ctrl+C to stop." +echo "" + +# --- State tracking --- +declare -A KNOWN_FILES # filename -> size +declare -A ISSUE_PHASE1 # issue# -> done|none +declare -A ISSUE_PHASE2 # issue# -> done|none +declare -A ISSUE_PR # issue# -> PR number +declare -A ISSUE_STATUS # issue# -> merged|failed|open|unknown +LAST_ACTIVITY=$(date +%s) + +timestamp() { date +%H:%M:%S; } + +monitor() { echo "[$(timestamp)] $*"; } +alert() { echo "[$(timestamp)] ⚠️ $*"; } +success() { echo "[$(timestamp)] ✅ $*"; } +failure() { echo "[$(timestamp)] ❌ $*"; } + +# Extract issue number from filename like "phase1-task-20260718-1650-33.md" +get_issue() { + local name="$1" + if [[ "$name" =~ -([0-9]+)\.(md|json|jsonl)$ ]]; then + echo "${BASH_REMATCH[1]}" + fi +} + +# Extract phase from filename +get_phase() { + local name="$1" + if [[ "$name" =~ ^(phase[12]) ]]; then + echo "${BASH_REMATCH[1]}" + fi +} + +# Find PR linked to an issue +find_pr() { + local issue="$1" + local pr="" + + # Strategy A: issue timeline + pr=$(gh api "/repos/$REPO/issues/$issue/timeline" \ + --jq '.[] | select(.event == "cross-referenced") | select(.source.issue.pull_request != null) | select(.source.issue.state == "open") | .source.issue.number' 2>/dev/null | head -1) + if [[ -n "$pr" ]]; then echo "$pr"; return 0; fi + + # Strategy B: PR body search + pr=$(gh pr list -R "$REPO" --state all --json number,body \ + --jq ".[] | select(.body | test(\"#$issue\")) | .number" 2>/dev/null | head -1) + if [[ -n "$pr" ]]; then echo "$pr"; return 0; fi + + return 1 +} + +# Get PR status as a one-line summary +get_pr_status() { + local pr_number="$1" + + local state draft base mergeable + local info + info=$(gh pr view "$pr_number" -R "$REPO" --json state,isDraft,baseRefName,mergeable 2>/dev/null) + if [[ -z "$info" ]]; then return 1; fi + + state=$(echo "$info" | jq -r '.state') + draft=$(echo "$info" | jq -r '.isDraft') + mergeable=$(echo "$info" | jq -r '.mergeable') + + # Count Copilot review rounds + local review_count + review_count=$(gh api "/repos/$REPO/pulls/$pr_number/reviews" \ + --jq '[.[] | select(.user.login | test("copilot-pull-request-reviewer|Copilot"))] | length' 2>/dev/null || echo "0") + + # Count open review comments from Copilot + local comment_count + comment_count=$(gh api "/repos/$REPO/pulls/$pr_number/comments" \ + --jq '[.[] | select(.user.login | test("copilot-pull-request-reviewer|Copilot"))] | length' 2>/dev/null || echo "0") + + # Check CI + local ci_failures ci_status + ci_failures=$(gh pr checks "$pr_number" -R "$REPO" --json name,state,bucket \ + --jq '.[] | select(.bucket == "fail") | select(.name != "No remove-before-merge directories") | .name' 2>/dev/null) + if [[ -z "$ci_failures" ]]; then ci_status="passing"; else ci_status="failing"; fi + + echo "state=$state draft=$draft reviews=$review_count comments=$comment_count CI=$ci_status" +} + +# --- Main monitor loop --- +ITERATION=0 + +while true; do + ITERATION=$((ITERATION + 1)) + NOW=$(date +%s) + NEW_FILES=() + + # Scan log directory for new or changed files + while IFS= read -r line; do + filename=$(basename "$line") + size=$(stat -f%z "$line" 2>/dev/null || stat -c%s "$line" 2>/dev/null || echo "0") + + if [[ -z "${KNOWN_FILES[$filename]+x}" ]]; then + KNOWN_FILES[$filename]="$size" + NEW_FILES+=("$filename") + LAST_ACTIVITY=$NOW + elif [[ "${KNOWN_FILES[$filename]}" != "$size" ]]; then + KNOWN_FILES[$filename]="$size" + LAST_ACTIVITY=$NOW + fi + done < <(find "$LOG_DIR_FULL" -maxdepth 1 -type f 2>/dev/null) + + # Process new files + for filename in "${NEW_FILES[@]}"; do + issue=$(get_issue "$filename") + phase=$(get_phase "$filename") + ext="${filename##*.}" + + [[ -z "$issue" || -z "$phase" ]] && continue + + # Initialize issue state if needed + : "${ISSUE_PHASE1[$issue]:=none}" + : "${ISSUE_PHASE2[$issue]:=none}" + : "${ISSUE_STATUS[$issue]:=unknown}" + + if [[ "$ext" == "md" ]]; then + if [[ "$phase" == "phase1" ]]; then + ISSUE_PHASE1[$issue]="done" + monitor "Issue #$issue: Phase 1 session exported ($filename)" + + # Try to find the PR + pr=$(find_pr "$issue") || true + if [[ -n "$pr" ]]; then + ISSUE_PR[$issue]="$pr" + monitor "Issue #$issue: Linked to PR #$pr" + fi + elif [[ "$phase" == "phase2" ]]; then + ISSUE_PHASE2[$issue]="done" + monitor "Issue #$issue: Phase 2 session exported ($filename)" + fi + fi + done + + # For issues with known PRs, check status + ACTIVE_ISSUE="" + for issue in "${!ISSUE_PR[@]}"; do + pr="${ISSUE_PR[$issue]}" + + # Detect active issue (has PR, phase2 not done) + if [[ "${ISSUE_PHASE2[$issue]}" != "done" ]]; then + ACTIVE_ISSUE="$issue" + fi + + # If phase2 just completed, check final state + if [[ "${ISSUE_PHASE2[$issue]}" == "done" && "${ISSUE_STATUS[$issue]}" != "merged" && "${ISSUE_STATUS[$issue]}" != "failed" ]]; then + pr_state=$(gh pr view "$pr" -R "$REPO" --json state --jq '.state' 2>/dev/null) + review_count=$(gh api "/repos/$REPO/pulls/$pr/reviews" \ + --jq '[.[] | select(.user.login | test("copilot-pull-request-reviewer|Copilot"))] | length' 2>/dev/null || echo "?") + if [[ "$pr_state" == "MERGED" ]]; then + ISSUE_STATUS[$issue]="merged" + success "Issue #$issue: PR #$pr MERGED ($review_count review rounds)" + elif [[ "$pr_state" == "CLOSED" ]]; then + ISSUE_STATUS[$issue]="failed" + failure "Issue #$issue: PR #$pr CLOSED (not merged)" + else + ISSUE_STATUS[$issue]="open" + failure "Issue #$issue: PR #$pr still OPEN after Phase 2 exited" + fi + fi + done + + # Poll active PR for real-time status + if [[ -n "$ACTIVE_ISSUE" && -n "${ISSUE_PR[$ACTIVE_ISSUE]+x}" ]]; then + pr="${ISSUE_PR[$ACTIVE_ISSUE]}" + status_line=$(get_pr_status "$pr") + if [[ -n "$status_line" ]]; then + full_line="Issue #$ACTIVE_ISSUE PR #$pr: $status_line" + if [[ "$status_line" == *"state=MERGED"* ]]; then + success "$full_line" + ISSUE_STATUS[$ACTIVE_ISSUE]="merged" + elif [[ "$status_line" == *"CI=failing"* ]]; then + alert "$full_line" + else + monitor "$full_line" + fi + fi + fi + + # Try to find PRs for issues that don't have one yet + if [[ $ITERATION -eq 1 ]] || [[ ${#NEW_FILES[@]} -gt 0 ]]; then + for issue in "${!ISSUE_PHASE1[@]}"; do + if [[ -z "${ISSUE_PR[$issue]+x}" ]]; then + pr=$(find_pr "$issue") || true + if [[ -n "$pr" ]]; then + ISSUE_PR[$issue]="$pr" + monitor "Issue #$issue: Found PR #$pr" + fi + fi + done + fi + + # Stale detection + STALE_SECONDS=$(( NOW - LAST_ACTIVITY )) + STALE_MINS=$(( STALE_SECONDS / 60 )) + if [[ $STALE_MINS -ge $STALE_MINUTES ]]; then + alert "No activity for ${STALE_MINS} minutes — shepherd may be stalled or waiting for CCRA" + fi + + # Periodic summary (every 5 iterations) + if [[ $(( ITERATION % 5 )) -eq 0 ]] && [[ ${#ISSUE_PHASE1[@]} -gt 0 ]]; then + echo "" + echo "[$(timestamp)] === Summary ===" + for issue in $(echo "${!ISSUE_PHASE1[@]}" | tr ' ' '\n' | sort -n); do + p1="${ISSUE_PHASE1[$issue]}" + p2="${ISSUE_PHASE2[$issue]:-none}" + pr="${ISSUE_PR[$issue]:-none}" + status="${ISSUE_STATUS[$issue]:-unknown}" + [[ "$pr" != "none" ]] && pr="PR #$pr" + echo " Issue #$issue : P1=$p1 P2=$p2 $pr status=$status" + done + echo "" + fi + + sleep "$POLL_INTERVAL" +done diff --git a/plugins/shepherd-task/scripts/shepherd-task.ps1 b/plugins/shepherd-task/scripts/shepherd-task.ps1 new file mode 100644 index 0000000000..fe9b6c0c00 --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task.ps1 @@ -0,0 +1,224 @@ +<# +.SYNOPSIS + Shepherds a child Task issue end-to-end: from Copilot assignment through merge. + +.DESCRIPTION + Orchestrates two phases by launching separate `copilot --yolo` sessions: + Phase 1: Assignment to Ready for Review + Phase 2: Ready for Review to Merged + + Between phases, the script verifies state using gh CLI (not copilot exit codes). + +.PARAMETER TaskIssue + The issue number (e.g., 1841) or URL of the child task to shepherd. + +.PARAMETER BaseBranch + The base branch the task PR should target. + +.PARAMETER Repo + Repository in OWNER/REPO format. +#> + +param( + [Parameter(Mandatory = $true, Position = 0)] + [string]$TaskIssue, + + [Parameter(Mandatory = $true, Position = 1)] + [string]$BaseBranch, + + [Parameter(Mandatory = $true, Position = 2)] + [string]$Repo, + + [Parameter(Mandatory = $false, Position = 3)] + [string]$LogDir = "shepherd-tasks-$(Get-Date -Format 'yyyyMMdd-HHmm')" +) + +$ErrorActionPreference = "Stop" + +if (-not (Test-Path $LogDir)) { + New-Item -ItemType Directory -Path $LogDir | Out-Null +} + +function Write-Status($msg) { + Write-Output "[shepherd-task] $msg" +} + +function Write-Fail($msg) { + Write-Output "[shepherd-task] FAILED: $msg" +} + +function Write-Ok($msg) { + Write-Output "[shepherd-task] $msg" +} + +# --- Helper: Find the PR linked to the task issue --- +function Find-LinkedPR { + # Strategy A: Issue timeline for cross-referenced PRs + $prNumber = gh api "/repos/$Repo/issues/$TaskIssue/timeline" ` + --jq '.[] | select(.event == "cross-referenced") | select(.source.issue.pull_request != null) | select(.source.issue.state == "open") | .source.issue.number' 2>$null | + Select-Object -First 1 + + if ($prNumber) { return $prNumber.Trim() } + + # Strategy B: Search PR bodies for the issue number + $prNumber = gh pr list -R $Repo --state open --json number,body ` + --jq ".[] | select(.body | test(`"#$TaskIssue`")) | .number" 2>$null | + Select-Object -First 1 + + if ($prNumber) { return $prNumber.Trim() } + + # Strategy C: Title or branch name match + $prNumber = gh pr list -R $Repo --state open --json number,title,headRefName ` + --jq ".[] | select((.title | test(`"$TaskIssue`"; `"i`")) or (.headRefName | test(`"$TaskIssue`"))) | .number" 2>$null | + Select-Object -First 1 + + if ($prNumber) { return $prNumber.Trim() } + + return $null +} + +# --- Helper: Verify all CI checks pass (excluding expected failure) --- +function Test-CIPassing { + param([string]$PRNumber) + + $failures = gh pr checks $PRNumber -R $Repo --json name,state,bucket ` + --jq '.[] | select(.bucket == "fail") | select(.name != "No remove-before-merge directories") | .name' 2>$null + + return [string]::IsNullOrWhiteSpace($failures) +} + +# --- Helper: Check for unresolved bot review comments --- +function Test-NoUnresolvedReviews { + param([string]$PRNumber) + + $repoOwner = ($Repo -split '/')[0] + $repoName = ($Repo -split '/')[1] + + $unresolved = gh api graphql -F owner=$repoOwner -F name=$repoName -F number=$PRNumber -f query=' + query($owner: String!, $name: String!, $number: Int!) { + repository(owner: $owner, name: $name) { + pullRequest(number: $number) { + reviewThreads(first: 100) { + nodes { isResolved comments(first: 1) { nodes { author { login } } } } + } + } + } + }' --jq '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false) | .comments.nodes[0].author.login' 2>$null + + return [string]::IsNullOrWhiteSpace($unresolved) +} + +# ============================================================================= +# PHASE 1: Assignment to Ready for Review +# ============================================================================= + +# Idempotency: skip Phase 1 if a PR already exists for this issue +$prNumber = Find-LinkedPR +if ($prNumber) { + Write-Status "PR #$prNumber already exists for issue #$TaskIssue — skipping Phase 1." +} else { + Write-Status "Phase 1: Launching copilot --yolo for task #$TaskIssue" + + $phase1Prompt = @" +Invoke skill ``shepherd-task-from-assignment-to-ready`` with these inputs: + +- TASK_ISSUE: $TaskIssue +- BASE_BRANCH: $BaseBranch +- REPO: $Repo +"@ + + Write-Status "Phase 1 prompt: $phase1Prompt" + $phase1Share = Join-Path $LogDir "phase1-task-$(Get-Date -Format 'yyyyMMdd-HHmm')-$TaskIssue.md" + $phase1Json = Join-Path $LogDir "phase1-task-$(Get-Date -Format 'yyyyMMdd-HHmm')-$TaskIssue.json" + $phase1Otel = Join-Path (Resolve-Path $LogDir) "phase1-otel-$(Get-Date -Format 'yyyyMMdd-HHmm')-$TaskIssue.jsonl" + $env:COPILOT_OTEL_FILE_EXPORTER_PATH = $phase1Otel + $phase1Prompt | copilot --yolo --output-format json --share $phase1Share > $phase1Json + Remove-Item Env:\COPILOT_OTEL_FILE_EXPORTER_PATH -ErrorAction SilentlyContinue + + Write-Status "Phase 1: copilot exited. Verifying state..." + + # --- Verify Phase 1 outcome --- + $prNumber = Find-LinkedPR + if (-not $prNumber) { + Write-Fail "No open PR found linked to issue #$TaskIssue after Phase 1." + exit 1 + } +} +Write-Status "Found PR #$prNumber" + +# Verify base branch +$actualBase = gh pr view $prNumber -R $Repo --json baseRefName --jq '.baseRefName' +if ($actualBase -ne $BaseBranch) { + Write-Status "PR base is '$actualBase', fixing to '$BaseBranch'..." + gh pr edit $prNumber -R $Repo --base $BaseBranch +} + +# Verify CI passing +if (-not (Test-CIPassing $prNumber)) { + Write-Fail "CI checks not passing on PR #$prNumber after Phase 1." + exit 1 +} + +# Verify no unresolved reviews +if (-not (Test-NoUnresolvedReviews $prNumber)) { + Write-Fail "Unresolved review comments remain on PR #$prNumber after Phase 1." + exit 1 +} + +Write-Ok "Phase 1 VERIFIED: PR #$prNumber is ready. CI passing, no unresolved comments." + +# ============================================================================= +# PHASE 2: Ready for Review to Merged +# ============================================================================= + +# Idempotency: skip Phase 2 if PR is already merged +$prState = gh pr view $prNumber -R $Repo --json state --jq '.state' +if ($prState -eq "MERGED") { + Write-Ok "PR #$prNumber already merged — skipping Phase 2." +} else { + Write-Status "Phase 2: Launching copilot --yolo for PR #$prNumber" + + $phase2Prompt = @" +Invoke skill ``shepherd-task-from-ready-to-merged-to-base`` with these inputs: + +- TASK_ISSUE: $TaskIssue +- BASE_BRANCH: $BaseBranch +- REPO: $Repo +- PR_NUMBER: $prNumber +"@ + + Write-Status "Phase 2 prompt: $phase2Prompt" + $phase2Share = Join-Path $LogDir "phase2-task-$(Get-Date -Format 'yyyyMMdd-HHmm')-$TaskIssue.md" + $phase2Json = Join-Path $LogDir "phase2-task-$(Get-Date -Format 'yyyyMMdd-HHmm')-$TaskIssue.json" + $phase2Otel = Join-Path (Resolve-Path $LogDir) "phase2-otel-$(Get-Date -Format 'yyyyMMdd-HHmm')-$TaskIssue.jsonl" + $env:COPILOT_OTEL_FILE_EXPORTER_PATH = $phase2Otel + $phase2Prompt | copilot --yolo --output-format json --share $phase2Share > $phase2Json + Remove-Item Env:\COPILOT_OTEL_FILE_EXPORTER_PATH -ErrorAction SilentlyContinue + + Write-Status "Phase 2: copilot exited. Verifying state..." + + # --- Verify Phase 2 outcome --- + $prState = gh pr view $prNumber -R $Repo --json state --jq '.state' + if ($prState -ne "MERGED") { + Write-Fail "PR #$prNumber is in state '$prState', expected MERGED." + exit 1 + } +} + +# Verify merged into correct branch +$mergedBase = gh pr view $prNumber -R $Repo --json baseRefName --jq '.baseRefName' +if ($mergedBase -ne $BaseBranch) { + Write-Fail "PR #$prNumber was merged into '$mergedBase', expected '$BaseBranch'." + exit 1 +} + +# Verify issue is closed +$issueState = gh issue view $TaskIssue -R $Repo --json state --jq '.state' +if ($issueState -ne "CLOSED") { + Write-Status "Issue #$TaskIssue still open, closing..." + gh issue close $TaskIssue -R $Repo +} + +Write-Ok "SHEPHERD TASK COMPLETE: Task #$TaskIssue has been fully shepherded." +Write-Ok "PR #$prNumber merged to $BaseBranch." +exit 0 diff --git a/plugins/shepherd-task/scripts/shepherd-task.sh b/plugins/shepherd-task/scripts/shepherd-task.sh new file mode 100755 index 0000000000..bc4b9df3ea --- /dev/null +++ b/plugins/shepherd-task/scripts/shepherd-task.sh @@ -0,0 +1,180 @@ +#!/usr/bin/env bash +# +# shepherd-task.sh — Shepherds a child Task issue end-to-end: +# from Copilot assignment through merge. +# +# Orchestrates two phases by launching separate `copilot --yolo` sessions. +# Between phases, verifies state using gh CLI (not copilot exit codes). +# +# Usage: ./shepherd-task.sh + +set -euo pipefail + +TASK_ISSUE="${1:?Usage: $0 [LOG_DIR]}" +BASE_BRANCH="${2:?Usage: $0 [LOG_DIR]}" +REPO="${3:?Usage: $0 [LOG_DIR]}" +LOG_DIR="${4:-shepherd-tasks-$(date +%Y%m%d-%H%M)}" + +mkdir -p "$LOG_DIR" + +# --- Helpers --- + +status() { echo "[shepherd-task] $*"; } +fail() { echo "[shepherd-task] FAILED: $*"; exit 1; } +ok() { echo "[shepherd-task] $*"; } + +# Find the PR linked to the task issue using three strategies. +find_linked_pr() { + local pr_number="" + + # Strategy A: Issue timeline for cross-referenced PRs + pr_number=$(gh api "/repos/$REPO/issues/$TASK_ISSUE/timeline" \ + --jq '.[] | select(.event == "cross-referenced") | select(.source.issue.pull_request != null) | select(.source.issue.state == "open") | .source.issue.number' 2>/dev/null | head -1) + + if [[ -n "$pr_number" ]]; then echo "$pr_number"; return 0; fi + + # Strategy B: Search PR bodies for the issue number + pr_number=$(gh pr list -R "$REPO" --state open --json number,body \ + --jq ".[] | select(.body | test(\"#$TASK_ISSUE\")) | .number" 2>/dev/null | head -1) + + if [[ -n "$pr_number" ]]; then echo "$pr_number"; return 0; fi + + # Strategy C: Title or branch name match + pr_number=$(gh pr list -R "$REPO" --state open --json number,title,headRefName \ + --jq ".[] | select((.title | test(\"$TASK_ISSUE\"; \"i\")) or (.headRefName | test(\"$TASK_ISSUE\"))) | .number" 2>/dev/null | head -1) + + if [[ -n "$pr_number" ]]; then echo "$pr_number"; return 0; fi + + return 1 +} + +# Verify all CI checks pass (excluding expected failure). +ci_passing() { + local pr_number="$1" + local failures + failures=$(gh pr checks "$pr_number" -R "$REPO" --json name,state,bucket \ + --jq '.[] | select(.bucket == "fail") | select(.name != "No remove-before-merge directories") | .name' 2>/dev/null) + + [[ -z "$failures" ]] +} + +# Check for unresolved bot review comments. +no_unresolved_reviews() { + local pr_number="$1" + local repo_owner="${REPO%%/*}" + local repo_name="${REPO##*/}" + local unresolved + unresolved=$(gh api graphql -F owner="$repo_owner" -F name="$repo_name" -F number="$pr_number" -f query=' + query($owner: String!, $name: String!, $number: Int!) { + repository(owner: $owner, name: $name) { + pullRequest(number: $number) { + reviewThreads(first: 100) { + nodes { isResolved comments(first: 1) { nodes { author { login } } } } + } + } + } + }' --jq '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false) | .comments.nodes[0].author.login' 2>/dev/null) + + [[ -z "$unresolved" ]] +} + +# ============================================================================= +# PHASE 1: Assignment to Ready for Review +# ============================================================================= + +# Idempotency: skip Phase 1 if a PR already exists for this issue +PR_NUMBER=$(find_linked_pr) || true +if [[ -n "$PR_NUMBER" ]]; then + status "PR #$PR_NUMBER already exists for issue #$TASK_ISSUE — skipping Phase 1." +else + status "Phase 1: Launching copilot --yolo for task #$TASK_ISSUE" + + PHASE1_PROMPT="Invoke skill \`shepherd-task-from-assignment-to-ready\` with these inputs: + +- TASK_ISSUE: $TASK_ISSUE +- BASE_BRANCH: $BASE_BRANCH +- REPO: $REPO" + + status "Phase 1 prompt:" + echo "$PHASE1_PROMPT" + PHASE1_SHARE="$LOG_DIR/phase1-task-$(date +%Y%m%d-%H%M)-$TASK_ISSUE.md" + PHASE1_JSON="$LOG_DIR/phase1-task-$(date +%Y%m%d-%H%M)-$TASK_ISSUE.json" + PHASE1_OTEL="$(cd "$LOG_DIR" && pwd)/phase1-otel-$(date +%Y%m%d-%H%M)-$TASK_ISSUE.jsonl" + export COPILOT_OTEL_FILE_EXPORTER_PATH="$PHASE1_OTEL" + echo "$PHASE1_PROMPT" | copilot --yolo --output-format json --share "$PHASE1_SHARE" > "$PHASE1_JSON" + unset COPILOT_OTEL_FILE_EXPORTER_PATH + + status "Phase 1: copilot exited. Verifying state..." + + # --- Verify Phase 1 outcome --- + PR_NUMBER=$(find_linked_pr) || fail "No open PR found linked to issue #$TASK_ISSUE after Phase 1." +fi +status "Found PR #$PR_NUMBER" + +# Verify base branch +ACTUAL_BASE=$(gh pr view "$PR_NUMBER" -R "$REPO" --json baseRefName --jq '.baseRefName') +if [[ "$ACTUAL_BASE" != "$BASE_BRANCH" ]]; then + status "PR base is '$ACTUAL_BASE', fixing to '$BASE_BRANCH'..." + gh pr edit "$PR_NUMBER" -R "$REPO" --base "$BASE_BRANCH" +fi + +# Verify CI passing +ci_passing "$PR_NUMBER" || fail "CI checks not passing on PR #$PR_NUMBER after Phase 1." + +# Verify no unresolved reviews +no_unresolved_reviews "$PR_NUMBER" || fail "Unresolved review comments remain on PR #$PR_NUMBER after Phase 1." + +ok "Phase 1 VERIFIED: PR #$PR_NUMBER is ready. CI passing, no unresolved comments." + +# ============================================================================= +# PHASE 2: Ready for Review to Merged +# ============================================================================= + +# Idempotency: skip Phase 2 if PR is already merged +PR_STATE=$(gh pr view "$PR_NUMBER" -R "$REPO" --json state --jq '.state') +if [[ "$PR_STATE" == "MERGED" ]]; then + ok "PR #$PR_NUMBER already merged — skipping Phase 2." +else + status "Phase 2: Launching copilot --yolo for PR #$PR_NUMBER" + + PHASE2_PROMPT="Invoke skill \`shepherd-task-from-ready-to-merged-to-base\` with these inputs: + +- TASK_ISSUE: $TASK_ISSUE +- BASE_BRANCH: $BASE_BRANCH +- REPO: $REPO +- PR_NUMBER: $PR_NUMBER" + + status "Phase 2 prompt:" + echo "$PHASE2_PROMPT" + PHASE2_SHARE="$LOG_DIR/phase2-task-$(date +%Y%m%d-%H%M)-$TASK_ISSUE.md" + PHASE2_JSON="$LOG_DIR/phase2-task-$(date +%Y%m%d-%H%M)-$TASK_ISSUE.json" + PHASE2_OTEL="$(cd "$LOG_DIR" && pwd)/phase2-otel-$(date +%Y%m%d-%H%M)-$TASK_ISSUE.jsonl" + export COPILOT_OTEL_FILE_EXPORTER_PATH="$PHASE2_OTEL" + echo "$PHASE2_PROMPT" | copilot --yolo --output-format json --share "$PHASE2_SHARE" > "$PHASE2_JSON" + unset COPILOT_OTEL_FILE_EXPORTER_PATH + + status "Phase 2: copilot exited. Verifying state..." + + # --- Verify Phase 2 outcome --- + PR_STATE=$(gh pr view "$PR_NUMBER" -R "$REPO" --json state --jq '.state') + if [[ "$PR_STATE" != "MERGED" ]]; then + fail "PR #$PR_NUMBER is in state '$PR_STATE', expected MERGED." + fi +fi + +# Verify merged into correct branch +MERGED_BASE=$(gh pr view "$PR_NUMBER" -R "$REPO" --json baseRefName --jq '.baseRefName') +if [[ "$MERGED_BASE" != "$BASE_BRANCH" ]]; then + fail "PR #$PR_NUMBER was merged into '$MERGED_BASE', expected '$BASE_BRANCH'." +fi + +# Verify issue is closed +ISSUE_STATE=$(gh issue view "$TASK_ISSUE" -R "$REPO" --json state --jq '.state') +if [[ "$ISSUE_STATE" != "CLOSED" ]]; then + status "Issue #$TASK_ISSUE still open, closing..." + gh issue close "$TASK_ISSUE" -R "$REPO" +fi + +ok "SHEPHERD TASK COMPLETE: Task #$TASK_ISSUE has been fully shepherded." +ok "PR #$PR_NUMBER merged to $BASE_BRANCH." +exit 0 diff --git a/plugins/shepherd-task/scripts/uninstall-task-shepherd.ps1 b/plugins/shepherd-task/scripts/uninstall-task-shepherd.ps1 new file mode 100644 index 0000000000..3a975b6ebf --- /dev/null +++ b/plugins/shepherd-task/scripts/uninstall-task-shepherd.ps1 @@ -0,0 +1,60 @@ +<# +.SYNOPSIS + Removes the shepherd-task plugin and skills from the user's Copilot home directory. + +.DESCRIPTION + Removes: + ~/.copilot/plugins/shepherd-task/ + ~/.copilot/skills/shepherd-task-* + +.EXAMPLE + ./uninstall-task-shepherd.ps1 +#> + +$ErrorActionPreference = 'Stop' + +$CopilotHome = if ($env:COPILOT_HOME) { $env:COPILOT_HOME } else { Join-Path $HOME '.copilot' } + +# Remove plugin. +$pluginDir = Join-Path $CopilotHome 'plugins' 'shepherd-task' +if (Test-Path $pluginDir -PathType Container) { + $redactionScripts = @( + 'redact-secrets.sh' + 'redact-secrets.ps1' + ) + foreach ($script in $redactionScripts) { + $installedScript = Join-Path $pluginDir 'scripts' $script + if (Test-Path $installedScript -PathType Leaf) { + Remove-Item -Path $installedScript -Force + Write-Host "Removed $installedScript" + } + } + Remove-Item -Path $pluginDir -Recurse -Force + Write-Host "Removed $pluginDir" +} else { + Write-Host "Plugin not found: $pluginDir (skipped)" +} + +# Remove skills. +$skills = @( + 'shepherd-task-from-assignment-to-ready' + 'shepherd-task-from-ready-to-merged-to-base' + 'shepherd-task-approve-workflows-and-wait-for-completion' + 'shepherd-task-create-ignorance-reduction-plan' + 'shepherd-task-create-post-mortem' + 'shepherd-task-create-issues-from-plan' +) +foreach ($skill in $skills) { + $skillDir = Join-Path $CopilotHome 'skills' $skill + if (Test-Path $skillDir -PathType Container) { + Remove-Item -Path $skillDir -Recurse -Force + Write-Host "Removed ~/.copilot/skills/$skill" + } +} + +# Remove any generated interview prompt files from the plugin scripts directory. +$interviewPrompts = Join-Path $pluginDir 'scripts' '*invoke-shepherd-task-create-issues-from-plan-skill.md' +Remove-Item -Path $interviewPrompts -Force -ErrorAction SilentlyContinue + +Write-Host "" +Write-Host "Shepherd-task fully uninstalled." diff --git a/plugins/shepherd-task/scripts/uninstall-task-shepherd.sh b/plugins/shepherd-task/scripts/uninstall-task-shepherd.sh new file mode 100755 index 0000000000..71b4f5a75a --- /dev/null +++ b/plugins/shepherd-task/scripts/uninstall-task-shepherd.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# +# uninstall-task-shepherd.sh — Removes the shepherd-task plugin and skills +# from the user's Copilot home directory. +# +# Removes: +# ~/.copilot/plugins/shepherd-task/ +# ~/.copilot/skills/shepherd-task-* +# +# Usage: ./uninstall-task-shepherd.sh + +set -euo pipefail + +COPILOT_HOME="${COPILOT_HOME:-$HOME/.copilot}" + +# Remove plugin. +PLUGIN_DIR="$COPILOT_HOME/plugins/shepherd-task" +if [ -d "$PLUGIN_DIR" ]; then + REDACTION_SCRIPTS=( + "redact-secrets.sh" + "redact-secrets.ps1" + ) + for script in "${REDACTION_SCRIPTS[@]}"; do + if [ -f "$PLUGIN_DIR/scripts/$script" ]; then + rm -f "$PLUGIN_DIR/scripts/$script" + echo "Removed $PLUGIN_DIR/scripts/$script" + fi + done + rm -rf "$PLUGIN_DIR" + echo "Removed $PLUGIN_DIR" +else + echo "Plugin not found: $PLUGIN_DIR (skipped)" +fi + +# Remove skills. +SKILLS=( + "shepherd-task-from-assignment-to-ready" + "shepherd-task-from-ready-to-merged-to-base" + "shepherd-task-approve-workflows-and-wait-for-completion" + "shepherd-task-create-ignorance-reduction-plan" + "shepherd-task-create-post-mortem" + "shepherd-task-create-issues-from-plan" +) +for skill in "${SKILLS[@]}"; do + skill_dir="$COPILOT_HOME/skills/$skill" + if [ -d "$skill_dir" ]; then + rm -rf "$skill_dir" + echo "Removed ~/.copilot/skills/$skill" + fi +done + +# Remove any generated interview prompt files from the plugin scripts directory. +rm -f "$PLUGIN_DIR/scripts/"*invoke-shepherd-task-create-issues-from-plan-skill.md 2>/dev/null || true + +echo "" +echo "Shepherd-task fully uninstalled." diff --git a/plugins/shepherd-task/test/01-prepare-base-branch.ps1 b/plugins/shepherd-task/test/01-prepare-base-branch.ps1 new file mode 100644 index 0000000000..00e0c476df --- /dev/null +++ b/plugins/shepherd-task/test/01-prepare-base-branch.ps1 @@ -0,0 +1,286 @@ +<# +.SYNOPSIS + Prepares the base branch for a shepherd-task test run by creating + the remove-before-merge directory with an ignorance-reduction plan. + +.DESCRIPTION + This script: + 1. Creates a new branch from the currently checked-out commit. + 2. Creates the -remove-before-merge/ directory. + 3. Writes the ignorance-reduction plan for the math-tool scenario. + 4. Commits and pushes the branch to origin. + + Enabling assumptions from plugins/shepherd-task/README.md must already + be satisfied before running this script. + +.PARAMETER Repo + GitHub repository in owner/repo format (e.g. "edburns/my-test-repo"). + +.PARAMETER BaseBranch + The non-main base branch name (e.g. "edburns/dd-3034809-test-01"). + The slug after the last "/" is used for directory and file naming. + +.EXAMPLE + .\01-prepare-base-branch.ps1 -Repo edburns/my-test-repo -BaseBranch edburns/dd-3034809-test-01 +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory)] + [ValidatePattern('^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$')] + [string]$Repo, + + [Parameter(Mandatory)] + [string]$BaseBranch +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +# ── Derive names from base branch ──────────────────────────────────────── + +$slug = $BaseBranch +if ($slug -match '/') { + $slug = $slug.Substring($slug.LastIndexOf('/') + 1) +} + +$rbmDir = "$slug-remove-before-merge" +$planFile = "$slug-ignorance-reduction-plan.md" + +Write-Host "Repo: $Repo" +Write-Host "Base branch: $BaseBranch" +Write-Host "Slug: $slug" +Write-Host "RBM directory: $rbmDir" +Write-Host "Plan file: $rbmDir/$planFile" +Write-Host "" + +# ── Verify git state ──────────────────────────────────────────────────── + +$gitStatus = git status --porcelain 2>&1 +if ($gitStatus) { + throw "Working tree is not clean. Commit or stash changes before running this script." +} + +# ── Create and switch to the base branch ───────────────────────────────── + +Write-Host "Creating branch '$BaseBranch' from current HEAD..." +git checkout -b $BaseBranch 2>&1 | Write-Host +if ($LASTEXITCODE -ne 0) { + throw "Failed to create branch '$BaseBranch'. Does it already exist?" +} + +# ── Create remove-before-merge directory ───────────────────────────────── + +Write-Host "Creating directory '$rbmDir'..." +New-Item -ItemType Directory -Path $rbmDir -Force | Out-Null + +# ── Write ignorance-reduction plan ─────────────────────────────────────── + +$planContent = @" +# Implementation plan: PowerShell math-tool ($slug) + +Human DRI: (test harness — automated) +Project root: repository root +Test framework: Pester 5 + +--- + +## Goal + +Build a PowerShell math-tool script (``math-tool.ps1``) incrementally across +four sequential tasks. Each task is a separate GitHub issue assigned to the +Copilot coding agent. The purpose is to exercise the **shepherd-task** +pipeline end-to-end: issue assignment → PR creation → CI → code review → +merge to base branch. + +The domain is intentionally trivial (Fibonacci and factorial computation) so +that token cost is low and failures are easy to diagnose. + +### Technology stack + +| Concern | Technology | +|---------|-----------| +| Runtime | PowerShell 7 (``pwsh``) | +| Test framework | Pester 5 | +| CI validation | Script exit codes + Pester ``-PassThru`` | + +### Deliverables + +| File | Purpose | +|------|---------| +| ``math-tool.ps1`` | Main script with math functions and CLI interface | +| ``math-tool.Tests.ps1`` | Pester 5 test suite | + +--- + +## Phase 1 — Architecture + +### 1.1 — Script structure + +The script follows standard PowerShell conventions: + +| Element | Design | +|---------|--------| +| Functions | ``Get-Fibonacci``, ``Get-Factorial`` — pure functions, no side effects | +| Parameters | Script-level ``param()`` block with ``-N``, ``-Operation``, ``-Verbose`` | +| Output | Single line to stdout: ``() = `` | +| Types | ``[int]`` for ``-N``, ``[bigint]`` for factorial results | +| Validation | ``[ValidateRange(0, 100)]`` on ``-N``, ``[ValidateSet()]`` on ``-Operation`` | + +### 1.2 — Test structure + +| Pattern | Approach | +|---------|----------| +| Unit tests | Dot-source ``math-tool.ps1``, test ``Get-Fibonacci`` and ``Get-Factorial`` directly | +| Integration tests | Invoke ``pwsh -File math-tool.ps1 `` and assert stdout content | +| Error tests | Invoke with invalid args, assert non-zero exit or error output | + +### 1.3 — Final file layout + +``` +(repo root) +├── math-tool.ps1 # Main script +├── math-tool.Tests.ps1 # Pester 5 test suite +└── $rbmDir/ + └── $planFile # This file +``` + +--- + +## Phase 2 — Ignorance reduction + +### 2.1 — Pester availability + +**Question:** Is Pester 5 available in the Copilot coding agent's GitHub Actions +runner environment? + +**Resolution:** Yes. GitHub-hosted ``ubuntu-latest`` runners include PowerShell 7 +with Pester 5 pre-installed. If not present, ``Install-Module Pester -Force +-Scope CurrentUser`` is a one-line fix that the agent can use. + +### 2.2 — BigInt support in PowerShell + +**Question:** Does PowerShell support ``[bigint]`` for large factorial values? + +**Resolution:** Yes. PowerShell 7 supports ``[bigint]`` (``System.Numerics.BigInteger``) +natively. ``Get-Factorial -N 100`` produces the correct 158-digit result without +overflow. + +### 2.3 — Script-level param() and dot-sourcing interaction + +**Question:** When a Pester test dot-sources ``math-tool.ps1``, does the +script-level ``param()`` block interfere? + +**Resolution:** Yes, this is a known pattern. The script must guard its +main execution logic so that dot-sourcing only defines the functions. +Standard pattern: + +```powershell +# Functions defined here are available after dot-sourcing +function Get-Fibonacci { ... } +function Get-Factorial { ... } + +# Main execution — only runs when script is invoked directly +if (`$MyInvocation.InvocationName -ne '.') { + # param() values are used here + ... +} +``` + +Alternatively, put the main logic after the functions and use ``param()`` at +the top — Pester tests can dot-source and then call functions directly, +ignoring the main output. + +--- + +## Phase 3 — Implementation (build order) + +Each step is a separate GitHub issue assigned to the Copilot coding agent. + +### 3.1 — Hardcoded Fibonacci(10) with Pester tests + +**What:** Create ``math-tool.ps1`` with a ``Get-Fibonacci`` function and a +hardcoded call to ``Get-Fibonacci -N 10``. Create ``math-tool.Tests.ps1`` +with Pester 5 tests. + +**Key files:** +- ``math-tool.ps1`` — ``Get-Fibonacci`` function (iterative), hardcoded output +- ``math-tool.Tests.ps1`` — tests for N=0, 1, 10, 20 + +**Gating criteria:** +1. ``pwsh -File math-tool.ps1`` prints ``Fibonacci(10) = 55`` +2. ``Invoke-Pester -Path math-tool.Tests.ps1 -PassThru`` — all tests pass +3. Exit code 0 + +### 3.2 — Parameterize N from command line + +**What:** Add a script-level ``param()`` block with ``[int]`$N = 10``. +Update tests to cover parameterized invocation. + +**Key files:** +- ``math-tool.ps1`` — add ``param()`` block, dynamic output +- ``math-tool.Tests.ps1`` — add script invocation tests + +**Gating criteria:** +1. ``pwsh -File math-tool.ps1`` prints ``Fibonacci(10) = 55`` (default) +2. ``pwsh -File math-tool.ps1 -N 15`` prints ``Fibonacci(15) = 610`` +3. ``pwsh -File math-tool.ps1 -N 0`` prints ``Fibonacci(0) = 0`` +4. All Pester tests pass + +### 3.3 — Add factorial operation + +**What:** Add ``Get-Factorial`` function and ``-Operation`` parameter with +``[ValidateSet('fibonacci','factorial')]``. + +**Key files:** +- ``math-tool.ps1`` — add ``Get-Factorial``, ``-Operation`` parameter, dispatch logic +- ``math-tool.Tests.ps1`` — add factorial unit tests and script invocation tests + +**Gating criteria:** +1. ``pwsh -File math-tool.ps1 -Operation factorial -N 5`` prints ``Factorial(5) = 120`` +2. ``pwsh -File math-tool.ps1 -Operation fibonacci -N 10`` prints ``Fibonacci(10) = 55`` +3. Default (no ``-Operation``) still prints ``Fibonacci(10) = 55`` +4. All Pester tests pass + +### 3.4 — Input validation, error handling, and help + +**What:** Add ``[ValidateRange(0, 100)]``, comment-based help, and ``-Verbose`` +output. + +**Key files:** +- ``math-tool.ps1`` — validation attributes, help block, verbose messages +- ``math-tool.Tests.ps1`` — validation error tests, verbose output test, help test + +**Gating criteria:** +1. ``pwsh -File math-tool.ps1 -N -1`` produces a validation error +2. ``pwsh -File math-tool.ps1 -N 50 -Operation factorial`` succeeds (large factorial) +3. ``pwsh -File math-tool.ps1 -Verbose -N 5`` includes verbose text +4. ``Get-Help .\math-tool.ps1`` shows SYNOPSIS +5. All Pester tests pass +"@ + +$planPath = Join-Path $rbmDir $planFile +Set-Content -Path $planPath -Value $planContent -Encoding utf8NoBOM +Write-Host "Wrote ignorance-reduction plan to '$planPath'" + +# ── Commit and push ────────────────────────────────────────────────────── + +Write-Host "" +Write-Host "Committing and pushing..." +git add $rbmDir +git commit -m "chore: add $rbmDir with ignorance-reduction plan for math-tool test" +if ($LASTEXITCODE -ne 0) { + throw "git commit failed" +} + +git push -u origin $BaseBranch +if ($LASTEXITCODE -ne 0) { + throw "git push failed" +} + +Write-Host "" +Write-Host "Done. Base branch '$BaseBranch' is ready." +Write-Host " Remote: https://github.com/$Repo/tree/$BaseBranch" +Write-Host " Plan: $rbmDir/$planFile" +Write-Host "" +Write-Host "Next step: run 02-create-issues.ps1 -Repo $Repo -BaseBranch $BaseBranch" diff --git a/plugins/shepherd-task/test/02-create-issues.ps1 b/plugins/shepherd-task/test/02-create-issues.ps1 new file mode 100644 index 0000000000..e89132f392 --- /dev/null +++ b/plugins/shepherd-task/test/02-create-issues.ps1 @@ -0,0 +1,372 @@ +<# +.SYNOPSIS + Creates a parent epic and four sequential child issues for the + shepherd-task math-tool test scenario. + +.DESCRIPTION + This script creates a deterministic set of GitHub issues that exercise + the shepherd-task pipeline end-to-end. Each child issue is a small, + well-scoped coding task for a PowerShell math-tool script. The issues + build on one another sequentially, exactly as shepherd-task expects. + + Enabling assumptions from plugins/shepherd-task/README.md must already + be satisfied before running this script. + +.PARAMETER Repo + GitHub repository in owner/repo format (e.g. "edburns/my-test-repo"). + +.PARAMETER BaseBranch + The non-main base branch on which work will happen + (e.g. "edburns/math-tool-test"). + +.EXAMPLE + .\02-create-issues.ps1 -Repo edburns/my-test-repo -BaseBranch edburns/math-tool-test +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory)] + [ValidatePattern('^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$')] + [string]$Repo, + + [Parameter(Mandatory)] + [string]$BaseBranch +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +# ── Derive names from base branch ──────────────────────────────────────── + +$slug = $BaseBranch +if ($slug -match '/') { + $slug = $slug.Substring($slug.LastIndexOf('/') + 1) +} + +$rbmDir = "$slug-remove-before-merge" +$planFile = "$slug-ignorance-reduction-plan.md" + +# ── Helpers ────────────────────────────────────────────────────────────── + +function New-Issue { + param( + [string]$Title, + [string]$Body + ) + $url = gh issue create ` + --repo $Repo ` + --title $Title ` + --body $Body 2>&1 + + if ($LASTEXITCODE -ne 0) { + throw "Failed to create issue '$Title': $url" + } + # gh issue create prints the URL, e.g. https://github.com/owner/repo/issues/42 + $url = ($url | Select-Object -Last 1).Trim() + $number = [int]($url -split '/')[-1] + $parsed = [PSCustomObject]@{ number = $number; url = $url } + Write-Host " Created issue #$number`: $Title" + return $parsed +} + +function Add-SubIssue { + param( + [int]$ParentNumber, + [int]$ChildNumber + ) + $null = gh api graphql -f query="mutation { + addSubIssue(input: {issueId: `"$($parentNodeId)`", subIssueId: `"$($childNodeIds[$ChildNumber])`"}) { + issue { id } + } + }" 2>&1 +} + +# ── Issue body templates ───────────────────────────────────────────────── + +$issue1Body = @" +## Task + +Create a PowerShell script ``math-tool.ps1`` that computes the 10th Fibonacci number (hardcoded) and prints the result, plus a Pester test that validates it. + +## Pre-reading (REQUIRED) + +Before starting, read the implementation plan on the ``$BaseBranch`` branch: + +`````` +$rbmDir/$planFile +`````` + +Then carefully re-read these sections: +- **Phase 1, Section 1.1 — Script structure** (function and parameter conventions) +- **Phase 1, Section 1.2 — Test structure** (Pester patterns) +- **Phase 3, Section 3.1 — Hardcoded Fibonacci** (this task's specification) + +## Branch + +Work on branch: ``$BaseBranch`` (push to ``origin``). + +## Specification + +Create the following files in the repository root: + +1. **``math-tool.ps1``**: + - Implement a function ``Get-Fibonacci`` that computes the Nth Fibonacci number iteratively (not recursively) + - The script calls ``Get-Fibonacci -N 10`` and writes the result to stdout: ``Fibonacci(10) = 55`` + - The function must be defined with ``function Get-Fibonacci { [CmdletBinding()] param(...) ... }`` + - Use ``[int]`` parameter type for N + +2. **``math-tool.Tests.ps1``** — Pester 5 test file: + - Import ``math-tool.ps1`` via dot-sourcing + - Test that ``Get-Fibonacci -N 0`` returns ``0`` + - Test that ``Get-Fibonacci -N 1`` returns ``1`` + - Test that ``Get-Fibonacci -N 10`` returns ``55`` + - Test that ``Get-Fibonacci -N 20`` returns ``6765`` + +## Gating criteria (ALL must pass before closing) + +1. Running ``pwsh -File math-tool.ps1`` prints ``Fibonacci(10) = 55`` +2. Running ``Invoke-Pester -Path math-tool.Tests.ps1 -PassThru`` shows all tests passing +3. The script exits with code 0 +"@ + +$issue2Body = @" +## Task + +Update ``math-tool.ps1`` to accept the Fibonacci index as a command-line parameter instead of using a hardcoded value. + +## Pre-reading (REQUIRED) + +Before starting, read the implementation plan on the ``$BaseBranch`` branch: + +`````` +$rbmDir/$planFile +`````` + +Then carefully re-read these sections: +- **Phase 2, Section 2.3 — Script-level param() and dot-sourcing** (interaction pattern) +- **Phase 3, Section 3.2 — Parameterize N** (this task's specification) + +## Branch + +Work on branch: ``$BaseBranch`` (push to ``origin``). + +## Prerequisite + +Issue 1 (Fibonacci hardcoded) must be complete. ``math-tool.ps1`` exists with a working ``Get-Fibonacci`` function. + +## Specification + +Modify the following files: + +1. **``math-tool.ps1``**: + - Add a ``param()`` block at script level with parameter ``[int]`$N = 10`` + - The script calls ``Get-Fibonacci -N `$N`` and prints ``Fibonacci() = `` + - Default behavior (no argument) must still print ``Fibonacci(10) = 55`` + +2. **``math-tool.Tests.ps1``** — update Pester tests: + - Keep all existing ``Get-Fibonacci`` unit tests + - Add a test that invokes the script with ``-N 15`` and verifies stdout contains ``Fibonacci(15) = 610`` + - Add a test that invokes the script with no arguments and verifies stdout contains ``Fibonacci(10) = 55`` + +## Gating criteria (ALL must pass before closing) + +1. ``pwsh -File math-tool.ps1`` prints ``Fibonacci(10) = 55`` (default) +2. ``pwsh -File math-tool.ps1 -N 15`` prints ``Fibonacci(15) = 610`` +3. ``pwsh -File math-tool.ps1 -N 0`` prints ``Fibonacci(0) = 0`` +4. All Pester tests pass +"@ + +$issue3Body = @" +## Task + +Add a ``-Operation`` parameter to ``math-tool.ps1`` that supports both ``fibonacci`` and ``factorial`` operations. + +## Pre-reading (REQUIRED) + +Before starting, read the implementation plan on the ``$BaseBranch`` branch: + +`````` +$rbmDir/$planFile +`````` + +Then carefully re-read these sections: +- **Phase 2, Section 2.2 — BigInt support** (``[bigint]`` for large factorials) +- **Phase 3, Section 3.3 — Add factorial operation** (this task's specification) + +## Branch + +Work on branch: ``$BaseBranch`` (push to ``origin``). + +## Prerequisite + +Issue 2 (parameterized N) must be complete. ``math-tool.ps1`` accepts ``-N`` as a parameter. + +## Specification + +Modify the following files: + +1. **``math-tool.ps1``**: + - Add a new function ``Get-Factorial`` that computes N! iteratively using ``[bigint]`` to handle large values + - Add parameter ``[ValidateSet('fibonacci','factorial')][string]`$Operation = 'fibonacci'`` + - The script dispatches to the correct function based on ``-Operation`` + - Output format: ``Fibonacci() = `` or ``Factorial() = `` + - Default behavior (no arguments) must still print ``Fibonacci(10) = 55`` + +2. **``math-tool.Tests.ps1``** — update Pester tests: + - Keep all existing tests + - Add ``Describe 'Get-Factorial'`` block with tests: + - ``Get-Factorial -N 0`` returns ``1`` + - ``Get-Factorial -N 1`` returns ``1`` + - ``Get-Factorial -N 5`` returns ``120`` + - ``Get-Factorial -N 20`` returns ``2432902008176640000`` + - Add script-level tests: + - ``-Operation factorial -N 5`` produces ``Factorial(5) = 120`` + - ``-Operation fibonacci -N 10`` produces ``Fibonacci(10) = 55`` + - Default (no ``-Operation``) still produces ``Fibonacci(10) = 55`` + +## Gating criteria (ALL must pass before closing) + +1. ``pwsh -File math-tool.ps1 -Operation factorial -N 5`` prints ``Factorial(5) = 120`` +2. ``pwsh -File math-tool.ps1 -Operation fibonacci -N 10`` prints ``Fibonacci(10) = 55`` +3. ``pwsh -File math-tool.ps1`` prints ``Fibonacci(10) = 55`` (default unchanged) +4. All Pester tests pass (both Fibonacci and Factorial) +"@ + +$issue4Body = @" +## Task + +Add input validation, error handling, and built-in help to ``math-tool.ps1``. + +## Pre-reading (REQUIRED) + +Before starting, read the implementation plan on the ``$BaseBranch`` branch: + +`````` +$rbmDir/$planFile +`````` + +Then carefully re-read these sections: +- **Phase 1, Section 1.1 — Script structure** (validation attributes) +- **Phase 2, Section 2.3 — Script-level param() and dot-sourcing** (dot-source safety for tests) +- **Phase 3, Section 3.4 — Input validation, error handling, and help** (this task's specification) + +## Branch + +Work on branch: ``$BaseBranch`` (push to ``origin``). + +## Prerequisite + +Issue 3 (multi-operation) must be complete. ``math-tool.ps1`` supports ``-Operation fibonacci`` and ``-Operation factorial``. + +## Specification + +Modify the following files: + +1. **``math-tool.ps1``**: + - Add ``[ValidateRange(0, 100)]`` to the ``-N`` parameter + - Add comment-based help (``<# .SYNOPSIS ... .EXAMPLE ... #>``) at the top of the script + - If ``-N`` is out of range, PowerShell's built-in validation should produce a clear error + - Add a ``-Verbose`` switch that, when present, prints the computation method (e.g., ``Computing fibonacci(10) iteratively...``) before the result + +2. **``math-tool.Tests.ps1``** — update Pester tests: + - Keep all existing tests + - Add tests for validation: + - Invoking with ``-N -1`` should throw a validation error + - Invoking with ``-N 101`` should throw a validation error + - Add test for verbose output: + - ``-Verbose -N 5`` should include verbose stream text containing ``Computing`` + - Add test for help: + - ``Get-Help .\math-tool.ps1`` should return help text containing ``SYNOPSIS`` + +## Gating criteria (ALL must pass before closing) + +1. ``pwsh -File math-tool.ps1 -N -1`` produces a validation error (non-zero exit) +2. ``pwsh -File math-tool.ps1 -N 50 -Operation factorial`` succeeds (large factorial works with ``[bigint]``) +3. ``pwsh -File math-tool.ps1 -Verbose -N 5`` prints verbose text plus ``Fibonacci(5) = 5`` +4. ``Get-Help .\math-tool.ps1`` shows help with SYNOPSIS +5. All Pester tests pass +"@ + +# ── Create the parent epic ─────────────────────────────────────────────── + +Write-Host "Creating issues in $Repo on branch $BaseBranch..." +Write-Host "" + +$epicBody = @" +## shepherd-task test scenario: math-tool.ps1 + +This epic contains four sequential tasks that build a PowerShell math-tool +script incrementally. The tasks are designed to exercise the shepherd-task +pipeline end-to-end. + +**Base branch:** ``$BaseBranch`` + +### Tasks + +1. Hardcoded Fibonacci(10) with Pester tests +2. Parameterize N from command line +3. Add factorial operation (multi-operation dispatch) +4. Input validation, error handling, and help + +Each task builds on the previous one. Run them in order via +``shepherd-task-given-list.ps1``. +"@ + +Write-Host "[1/5] Creating parent epic..." +$epic = New-Issue -Title "[Epic] shepherd-task test: math-tool.ps1" -Body $epicBody + +# ── Create child issues ────────────────────────────────────────────────── + +Write-Host "[2/5] Creating task 1..." +$t1 = New-Issue ` + -Title "1 — Hardcoded Fibonacci: create math-tool.ps1 with Pester tests" ` + -Body $issue1Body + +Write-Host "[3/5] Creating task 2..." +$t2 = New-Issue ` + -Title "2 — Parameterize N: accept -N from command line" ` + -Body $issue2Body + +Write-Host "[4/5] Creating task 3..." +$t3 = New-Issue ` + -Title "3 — Multi-operation: add -Operation (fibonacci | factorial)" ` + -Body $issue3Body + +Write-Host "[5/5] Creating task 4..." +$t4 = New-Issue ` + -Title "4 — Validation and help: input guards, verbose output, comment-based help" ` + -Body $issue4Body + +# ── Link child issues to the parent epic via sub-issues API ────────────── + +Write-Host "" +Write-Host "Linking child issues to parent epic #$($epic.number)..." + +$epicNodeId = gh api "repos/$Repo/issues/$($epic.number)" --jq .node_id +$childIssues = @($t1, $t2, $t3, $t4) + +foreach ($child in $childIssues) { + $childNodeId = gh api "repos/$Repo/issues/$($child.number)" --jq .node_id + $mutation = "mutation { addSubIssue(input: {issueId: `"$epicNodeId`", subIssueId: `"$childNodeId`"}) { issue { id } } }" + $null = gh api graphql -f query=$mutation + if ($LASTEXITCODE -ne 0) { + Write-Warning "Could not link #$($child.number) as sub-issue of #$($epic.number) (sub-issues API may not be available)" + } else { + Write-Host " Linked #$($child.number) -> parent #$($epic.number)" + } +} + +# ── Summary ────────────────────────────────────────────────────────────── + +Write-Host "" +Write-Host "Done. Created issues in $Repo :" +Write-Host "" +Write-Host " Epic: #$($epic.number) $($epic.url)" +Write-Host " Task 1: #$($t1.number) $($t1.url)" +Write-Host " Task 2: #$($t2.number) $($t2.url)" +Write-Host " Task 3: #$($t3.number) $($t3.url)" +Write-Host " Task 4: #$($t4.number) $($t4.url)" +Write-Host "" +Write-Host "To run shepherd-task on these issues:" +Write-Host "" +Write-Host " shepherd-task-given-list.ps1 `"$($t1.number),$($t2.number),$($t3.number),$($t4.number)`" $BaseBranch $Repo" diff --git a/skills/shepherd-task-approve-workflows-and-wait-for-completion/SKILL.md b/skills/shepherd-task-approve-workflows-and-wait-for-completion/SKILL.md new file mode 100644 index 0000000000..6f309d789f --- /dev/null +++ b/skills/shepherd-task-approve-workflows-and-wait-for-completion/SKILL.md @@ -0,0 +1,81 @@ +--- +name: shepherd-task-approve-workflows-and-wait-for-completion +description: "Use this skill to approve pending workflow runs on a PR branch and wait for them to complete." +--- + +# Skill: Approve Workflows and Wait for Completion + +## Purpose + +Approve all pending workflow runs (`action_required` status) on a PR's topic branch and wait for them to complete. This is a reusable sub-skill invoked by other shepherd skills whenever workflow approval is needed. + +## Inputs + +- `REPO`: Repository in `OWNER/REPO` format (e.g., `github/copilot-sdk`). +- `JTBDTASK_BRANCH`: The topic branch name associated with the PR (used to find workflow runs). +- `PR_NUMBER`: The PR number (used for `gh pr checks --watch`). + +## Prerequisites + +- `gh` CLI authenticated with sufficient permissions (actions, PRs). +- The PR exists and has workflow runs triggered on `JTBDTASK_BRANCH`. + +--- + +## ⚠️ CRITICAL: Never go idle while waiting + +The `copilot --yolo` runtime **terminates the session shortly after the agent goes idle** (i.e., when there are no pending tool calls). If you launch a long-running polling command with a short `initial_wait` and then say "I'll check back when it completes," the runtime will kill the session before the command finishes. + +**Rules for all polling and waiting steps:** + +1. ✅✅✅ **ALWAYS use `initial_wait` ≥ 600 seconds** (10 minutes) on any polling/waiting command. This keeps the agent blocked on the tool call rather than going idle. ✅✅✅ +2. ❌❌❌ **NEVER background a polling command and then end your turn with no tool calls.** If a command exceeds `initial_wait`, immediately issue another tool call (e.g., `read_powershell`) to stay active. ❌❌❌ +3. ❌❌❌ **NEVER say "I'll check back when it completes" or "Waiting for notification."** These phrases mean you are going idle, which KILLS THE SESSION. ❌❌❌ +4. ✅✅✅ **ALWAYS prefer a single blocking poll** over launching a background command and waiting for a notification. ✅✅✅ + +--- + +## Procedure + +### Step 1: Approve pending workflow runs + +For each run in `action_required` status on the PR's branch, re-run it. The correct mechanism is `gh run rerun` (the `POST .../actions/runs/{id}/approve` endpoint is for fork PRs only and will return HTTP 403 here). + +```bash +# Get all action_required runs for the PR branch +PENDING_RUNS=$(gh run list -R $REPO --branch "$JTBDTASK_BRANCH" \ + --json databaseId,conclusion --jq '.[] | select(.conclusion == "action_required") | .databaseId') + +for RUN_ID in $PENDING_RUNS; do + gh run rerun $RUN_ID -R $REPO +done +``` + +### Step 2: Wait for workflow runs to complete + +```bash +# Watch all runs on the branch until they complete +# Use gh pr checks with --watch for convenience +gh pr checks $PR_NUMBER -R $REPO --watch --fail-fast +``` + +Alternatively, poll with: + +```bash +gh run list -R $REPO --branch "$JTBDTASK_BRANCH" \ + --json databaseId,status,conclusion,name \ + --jq '.[] | select(.status != "completed")' +``` + +--- + +## Error handling + +- **No pending runs found**: This is not an error — it means runs were already approved (possibly manually). Proceed directly to waiting for completion. +- **`gh run rerun` fails**: Retry up to 3 times with 10-second backoff, then report and stop. +- **Runs do not complete within a reasonable time**: The `--watch` flag on `gh pr checks` will block until completion or failure. If it times out, report and stop. + +## Notes + +- This skill is extracted from Steps 4 and 5 of `shepherd-task-from-assignment-to-ready` for reuse across multiple shepherd skills. +- The `gh api .../actions/runs/{id}/approve` endpoint does NOT work for same-repo PRs (returns HTTP 403 "This run is not from a fork pull request"). Always use `gh run rerun` instead. diff --git a/skills/shepherd-task-create-ignorance-reduction-plan/SKILL.md b/skills/shepherd-task-create-ignorance-reduction-plan/SKILL.md new file mode 100644 index 0000000000..615787fa12 --- /dev/null +++ b/skills/shepherd-task-create-ignorance-reduction-plan/SKILL.md @@ -0,0 +1,566 @@ +--- +name: shepherd-task-create-ignorance-reduction-plan +description: "Use this skill when creating a new ignorance reduction plan — a structured document that maps unknowns, spikes, and phased implementation steps for a multi-day engineering campaign." +--- + +# Skill: Create Ignorance Reduction Plan + +## Purpose + +Generate a structured **ignorance reduction plan** — a markdown document that identifies unknowns, formulates precise questions, and lays out phased implementation steps for a multi-day engineering campaign. The plan is created with **empty Resolution sections**; a human expert later fills in each resolution, sometimes producing engineering "spike" work that resides alongside the plan. + +## Inputs + +- `FILENAME`: The full relative path for the output markdown file (e.g., `my-project/1234-feature-remove-before-merge/1234-ignorance-reduction-plan.md`). +- `CONTEXT`: The user will have already loaded sufficient context (chat history, ADRs, issues, PRs, prior plans) into the conversation before invoking this skill. The skill uses that loaded context to formulate high-quality questions. + +## Prerequisites + +- The LLM must already have enough context loaded to reason deeply about what is being built. This skill does NOT gather context — it synthesizes what is already available into a structured plan. +- The user may specify which phases are already completed (mark them ✅). +- The user may specify constraints on which sections to produce (e.g., "only do phases before Phase 1" or "skip the Completed Phases section"). + +--- + +## Output Structure + +The generated plan MUST follow the structure below. Study the embedded examples carefully to understand the level of detail, the tone, and the formatting conventions. + +### Required Sections (in order) + +1. **Title line** — `# Implementation plan: ()` +2. **Metadata block** — Human DRI, relevant ADRs/references, issue links, related directories +3. **Goal section** (when the campaign is large enough to warrant one) — concise statement of what success looks like, optionally with a technology stack table and a list of SDK/framework features to exercise +4. **Completed phases** — phases already done, marked ✅, with brief summaries of what was accomplished and decided +5. **Ignorance reduction phase** — the core of the plan. A numbered list of questions/spikes, each with: + - `### N.M — Short title` + - `**Question:** ` + - Context paragraph explaining why this matters, options considered, trade-offs + - Optional table of options + - `**Spike needed:**` (when hands-on verification is required) + - `**Recommendation:**` (the plan author's suggested answer, when one exists) + - `**Resolution:**` (LEFT EMPTY — the human fills this in later) +6. **Implementation phase(s)** — ordered build steps, each with: + - What to build + - Files to create/modify + - Tests to write + - Gating criteria +7. **Optional trailing sections** — Documentation phase, reference material, cross-cutting concerns + +--- + +## Key Principles for Question Quality + +1. **Be specific** — "How should X work?" is too vague. "Should `@Foo` use `RUNTIME` or `SOURCE` retention, given that ADR-005 says 'compile-time preferred, runtime fallback'?" is good. +2. **Show your work** — Each question should include the options you've identified, the trade-offs, and a recommendation when possible. +3. **Include code sketches** — When the question is about API shape, include proposed code. +4. **Identify spikes** — When a question can only be answered by writing code, say "**Spike needed:**" and describe what to verify. +5. **Order questions by dependency** — Earlier questions inform later ones. +6. **Be exhaustive but not redundant** — Cover every genuine unknown; skip things that are already decided in ADRs or prior phases. + +--- + +## Embedded Examples + +The following are real, completed ignorance reduction plans. Study them to understand the expected depth, formatting, and tone. When you create a NEW plan, the **Resolution:** sections must be EMPTY (the human fills them in). + +### Example 1: Java `@CopilotTool` ergonomics (copilot-sdk, issue #1682) + +```markdown +# Implementation plan: `@CopilotTool` ergonomics (issue #1682) + +Human DRI: Ed Burns +ADR: `java/docs/adr/adr-005-tool-definition.md` +Issue: https://github.com/github/copilot-sdk/issues/1682 + +--- + +## Completed phases + +### Phase 1 ✅ — Define the problem and architectural decision + +- ADR-005 evaluates three options (status quo, record-as-schema, annotation-on-method). +- Decision: annotation-on-method with compile-time JSR 269 processor (langchain4j-style API, Micronaut-style implementation). + +### Phase 2 ✅ — Verify the existing low-level path works in Java + +- `test/snapshots/tools/low_level_tool_definition.yaml` created. +- `LowLevelToolDefinitionIT` passes with explicit `ToolDefinition.create()` / `createOverride()`. +- This proves the low-level API is correct and will serve as the foundation that the high-level API delegates to. + +--- + +## Phase 3 — Ignorance reduction: questions to answer before writing code + +This phase is about eliminating unknowns. Each item is a question or spike. Resolve these **before** writing production code. + +### 3.1 — Package placement + +**Question:** Where do `@CopilotTool` and `@Param` live? + +Current SDK structure is a single module (`copilot-sdk-java`). Two options: + +| Option | Location | Trade-off | +|--------|----------|-----------| +| A | `com.github.copilot.rpc` (alongside `ToolDefinition`) | Keeps everything together but the `rpc` package is already dense (40+ classes). | +| B | New package `com.github.copilot.tool` | Cleaner separation; the `tool` package holds annotations, processor, and `ToolDefinition.fromObject()`. But `ToolDefinition` itself stays in `rpc` (it's a JSON-RPC type). | + +**Recommendation:** Option B — new `com.github.copilot.tool` package for annotations + processor + schema generation. `ToolDefinition` stays in `rpc` and gets a new static method `fromObject(Object)` that delegates to `tool` package internals. + +**Action:** Decide; update `module-info.java` exports if new package is added. + +**Resolution:** Select Option B. + +### 3.2 — `@CopilotTool` annotation design + +**Question:** What attributes does `@CopilotTool` need? + +Based on ADR-005 and the C#/langchain4j comparisons: + +```java +@Documented +@Retention(RetentionPolicy.SOURCE) // only needed at compile time for processor +@Target(ElementType.METHOD) +@CopilotExperimental +public @interface CopilotTool { + /** Tool description (sent to the model). */ + String value(); + + /** Tool name. Defaults to method name converted to snake_case. */ + String name() default ""; + + /** Whether this tool overrides a built-in tool. */ + boolean overridesBuiltInTool() default false; + + /** Whether to skip permission checks. */ + boolean skipPermission() default false; +} +``` + +**Open questions:** + +1. Should `@CopilotTool` have `@Retention(SOURCE)` or `RUNTIME`? ADR-005 says "compile-time preferred, runtime fallback" — if we want a fallback path, we need `RUNTIME`. + +2. Is `ToolDefer` needed on the annotation, or is that too niche for v1? + +**Recommendation:** Start with `RUNTIME` retention so the reflection fallback works. Defer `ToolDefer` support to a follow-up. + +**Resolution:** Select `RUNTIME` and `ToolDefer` support. + +### 3.3 — Type-to-JSON-Schema mapping + +**Question:** What Java types do we need to map to JSON Schema, and how? + +Minimum viable set: + +| Java type | JSON Schema | +|-----------|-------------| +| `String` | `{"type": "string"}` | +| `int`, `Integer`, `long`, `Long` | `{"type": "integer"}` | +| `double`, `Double`, `float`, `Float` | `{"type": "number"}` | +| `boolean`, `Boolean` | `{"type": "boolean"}` | +| `enum` types | `{"type": "string", "enum": ["V1", "V2", ...]}` | +| `List`, `Collection` | `{"type": "array", "items": }` | + +**Key design decision:** The annotation processor must generate this schema at compile time from `javax.lang.model` types (`TypeMirror`, `DeclaredType`, etc.), NOT from `java.lang.reflect` types. + +**Spike needed:** Write a small proof-of-concept that maps `TypeMirror` → JSON Schema `Map` literal in generated source code. + +**Recommendation:** Start with flat types (primitives, String, enums) and `List`. Defer nested records to a follow-up. + +**Resolution:** + +### 3.4 — Generated code shape + +**Question:** What exactly does the processor generate? + +```java +// GENERATED — do not edit +final class MyTools$$CopilotToolMeta { + static List definitions(MyTools instance) { + return List.of( + new ToolDefinition("set_current_phase", "Sets the current phase", + Map.of("type", "object", + "properties", Map.of("phase", Map.of("type", "string", + "description", "The phase to transition to")), + "required", List.of("phase")), + invocation -> { + String phase = (String) invocation.getArguments().get("phase"); + return CompletableFuture.completedFuture( + instance.setCurrentPhase(phase)); + }, null, null, null) + ); + } +} +``` + +**Open questions:** + +1. Access levels — require public/package-private, or support private via MethodHandles? +2. Return type handling — String, void, CompletableFuture, other? +3. Argument deserialization — direct casts vs ObjectMapper.convertValue()? + +**Recommendation:** Require at least package-private. Support String, void, CompletableFuture. Use direct casts for simple types, ObjectMapper for complex. + +**Resolution:** + +--- + +## Phase 4 — Implementation (the build order) + +After Phase 3 questions are resolved, implement in this order. Each step should be a separately testable commit. + +### 4.1 — Annotations (`@CopilotTool`, `@Param`) + +**What:** Create the two annotation classes. + +**Files to create:** +- `java/src/main/java/com/github/copilot/tool/CopilotTool.java` +- `java/src/main/java/com/github/copilot/tool/Param.java` + +**Tests:** Compile-only: ensure they compile and can be applied. + +**Gating criteria:** `mvn clean compile` passes. + +### 4.2 — Schema generation utility + +**What:** Utility class that maps `javax.lang.model` types to JSON Schema `Map` literals. + +**Tests:** Unit tests exercising type-to-schema mapping. + +**Gating criteria:** Correct schema for String, int, boolean, enum, List. + +### 4.3 — Annotation processor + +**What:** JSR 269 processor that generates `$$CopilotToolMeta` classes. + +**Tests:** Compilation tests with programmatic JavaCompiler. + +**Gating criteria:** Generates correct metadata for 2-3 `@CopilotTool` methods. + +### 4.4 — `ToolDefinition.fromObject(Object)` + +**What:** Runtime bridge that loads generated metadata. + +**Tests:** End-to-end: annotated class → processor → fromObject() → working tools. + +**Gating criteria:** Returns working `List`. + +### 4.5 — E2E integration test + +**What:** Failsafe IT using ergonomic API against replay proxy. + +**Gating criteria:** Same assertions as low-level test pass. + +--- + +## Phase 5 — Documentation and examples + +- Update `java/README.md` with ergonomic tool definition example. +- Reference ADR-005 for design rationale. +``` + +### Example 2: Java Real Estate Demo (BRK-206, dd-3017826) + +```markdown +# Implementation plan: Java Real Estate Agent Orchestrator Demo (dd-3017826) + +Human DRI: Ed Burns +Reference C# demo: `BRK206-00/src/AgentOrchestrator/` +Copilot SDK for Java: `copilot-sdk/java/` (version 1.0.7-SNAPSHOT) +Related ADRs: `java/docs/adr/adr-005-tool-definition.md`, `java/docs/adr/adr-006-tool-definition-inline.md` + +--- + +## Goal + +Create a Java analog of the C# Blazor AgentOrchestrator demo that showcases the Copilot SDK for Java. The demo implements a real-estate lead-management pipeline powered by multiple concurrent `CopilotSession` agents with custom tools, system message customization, and real-time UI updates. + +### Technology stack + +| Concern | Technology | +|---------|-----------| +| Runtime | OpenLiberty 26.0.0.5 | +| Platform | Jakarta EE 11 (Faces 4.0, CDI 4.0, WebSocket 2.1) | +| UI framework | PrimeFaces 15.0.16 | +| AI orchestration | Copilot SDK for Java 1.0.7-SNAPSHOT | +| Database | H2 in-memory | +| Build | Maven, Liberty Maven Plugin | + +### SDK features to demonstrate + +1. Custom tool definition via `@CopilotTool` annotations +2. Agentic session loop (`session.sendAndWait(...)`) +3. System message customization +4. Real-time session events +5. Multiple concurrent sessions +6. Permission handling + +--- + +## Completed phases + +### Phase 1 ✅ — Define the architecture mapping (C# → Java) + +- Component mapping table (Program.cs → server.xml+CDI, AppState → @ApplicationScoped, etc.) +- Threading model (Task.Run → virtual threads, await → .join()) +- Project structure defined + +--- + +## Phase 2 — Ignorance reduction: questions to answer before writing code + +### 2.1 — CopilotClient lifecycle in Jakarta CDI + +**Question:** How should `CopilotClient` be created and managed in CDI? + +The C# demo creates `CopilotClient` as a singleton in `AppState`. In Java, `CopilotClient` implements `AutoCloseable`. We need: + +1. A `@Produces @ApplicationScoped` method that creates the client in `Empty` mode. +2. A `@Disposes` method that calls `close()` on shutdown. + +**Spike needed:** Verify `CopilotClient` constructor options in 1.0.7-SNAPSHOT support `Empty` mode and `baseDirectory`. + +**Resolution:** + +### 2.2 — `sendAndWait` blocking semantics on virtual threads + +**Question:** Does the Java SDK's `sendAndWait(...)` block the calling thread, or does it return a `CompletableFuture`? + +If it returns `CompletableFuture`, we call `.join()` on a virtual thread. If synchronous, call directly. + +**Resolution:** + +### 2.3 — Session event subscription in Java SDK + +**Question:** What is the Java equivalent of `Session.On(handler)`? + +Need to confirm: +- The method name and signature +- Whether it accepts a `Class` event type filter +- Whether the handler receives events on the calling thread or a callback thread + +**Resolution:** + +### 2.4 — WebSocket push from CDI to JSF + +**Question:** How do we push UI updates from a background virtual thread to the browser? + +Options: +| Option | Mechanism | +|--------|-----------| +| A | `f:websocket` (Jakarta Faces 4.0 built-in) | +| B | PrimeFaces `p:socket` | +| C | Raw Jakarta WebSocket endpoint | + +**Recommendation:** Option A — standard, no PrimeFaces-specific coupling. + +**Resolution:** + +--- + +## Phase 3 — Implementation + +### 3.1 — Project skeleton + +**What:** Maven project with Liberty plugin, server.xml, minimal Faces page. + +**Gating criteria:** `mvn liberty:run` starts; index.xhtml renders. + +### 3.2 — Property database + JPA seeding + +**What:** H2 in-memory DB, JPA entities, seed 100 properties from JSON. + +**Gating criteria:** Properties queryable from a test CDI bean. + +### 3.3 — CopilotClient + session lifecycle + +**What:** CDI producer, agent class, single-agent run loop. + +**Gating criteria:** One agent completes a session with a custom tool. + +### 3.4 — Full pipeline + UI + +**What:** Multi-agent orchestration, WebSocket push, PrimeFaces UI. + +**Gating criteria:** Full pipeline demo works end-to-end. +``` + +### Example 3: Python Agent Demo (BRK-206, issue #28) + +```markdown +# Implementation plan: Python Real Estate Agent Orchestrator Demo (28-python-agent-demo) + +Human DRI: Ed Burns +Reference C# demo: `src/AgentOrchestrator/` +Reference Java demo: `src/java-agent-orchestrator/` +Python project root: `src/python_agent_orchestrator/` + +--- + +## Goal + +Create a Python analog of the C# Blazor AgentOrchestrator demo that showcases the GitHub Copilot SDK for Python. Same pipeline (Queued → Validating → Searching → Writing Report → Done/Rejected) with multiple concurrent agent sessions, custom tools, and real-time UI updates. + +### Technology stack + +| Concern | Technology | +|---------|-----------| +| Runtime | CPython 3.14 | +| Backend | FastAPI (ASGI, asyncio-native) | +| UI | Jinja2 + HTMX + Alpine.js + Tailwind CSS | +| Real-time | FastAPI WebSocket | +| AI orchestration | Copilot SDK for Python | +| Database | SQLite in-memory via SQLModel | + +### SDK features to demonstrate + +1. Custom tool definition (`@define_tool` decorator) +2. Agentic session loop +3. System message customization +4. Real-time session events +5. Multiple concurrent sessions + +--- + +## Phase 1 — Define the architecture mapping (C# → Python) + +### 1.1 — Component mapping + +| C# Component | Python Equivalent | Notes | +|---|---|---| +| `Program.cs` | `main.py` (FastAPI app factory) | ASGI with lifespan | +| `AppState.cs` | `app_state.py` singleton | Holds client + agents dict | +| `Agent.cs` | `agent.py` async class | Each agent = asyncio.Task | +| `PropertyDatabase.cs` | `property_database.py` (SQLModel) | SQLite in-memory | +| Blazor interactive | HTMX + Alpine.js | Server-rendered + progressive | +| `CopilotTool.DefineTool(method)` | `@define_tool` decorator | SDK native | + +### 1.2 — Concurrency model + +| C# | Python | +|---|---| +| `Task.Run(...)` | `asyncio.create_task(...)` | +| `await SendAndWaitAsync(...)` | `await session.send()` + `SessionIdleData` | +| `Task.Delay(...)` | `await asyncio.sleep(...)` | + +--- + +## Phase 2 — Ignorance reduction: questions to answer before writing code + +### 2.1 — CopilotClient lifecycle in FastAPI + +**Question:** How should `CopilotClient` be created and managed in a FastAPI app? + +Need: +1. Lifespan handler that creates client on startup, calls `stop()` on shutdown. +2. Decision on `mode="empty"` vs default mode. + +**Spike needed:** Confirm `CopilotClient(mode="empty")` works for headless server-side orchestration. + +**Resolution:** + +### 2.2 — Session send-and-wait pattern + +**Question:** The Python SDK's `session.send(msg)` just sends — completion is signaled by `SessionIdleData`. How do we replicate the agentic loop? + +**Spike needed:** Confirm multi-turn tool calls work with the wait pattern. + +**Resolution:** + +### 2.3 — WebSocket push from asyncio to browser + +**Question:** How do we push UI updates from a `session.on()` callback to the browser? + +**Spike needed:** Verify broadcasting from inside a sync callback works (may need `asyncio.run_coroutine_threadsafe()`). + +**Resolution:** + +### 2.4 — Property database: SQLModel + SQLite + +**Question:** Can synchronous SQLModel handle the load without blocking the event loop? + +**Spike needed:** Measure query latency for in-memory reads. + +**Resolution:** + +--- + +## Phase 3 — Implementation + +### 3.1 — Project skeleton + +**What:** FastAPI app, pyproject.toml, basic route. + +**Gating criteria:** `uvicorn main:app` starts; health endpoint responds. + +### 3.2 — Property database + seeding + +**What:** SQLModel models, seed from JSON files. + +**Gating criteria:** `pytest test_property_database.py` passes. + +### 3.3 — CopilotClient + agent lifecycle + +**What:** Lifespan handler, agent class, single-agent run. + +**Gating criteria:** One agent completes with a custom tool. + +### 3.4 — Full pipeline + WebSocket UI + +**What:** Multi-agent orchestration, HTMX/Alpine UI, WebSocket push. + +**Gating criteria:** Full pipeline demo works end-to-end. +``` + +--- + +## Procedure + +When this skill is invoked: + +### Step 1: Verify context is sufficient + +Before generating the plan, confirm that you have enough loaded context to formulate **specific, actionable questions**. The questions must be grounded in real code, real APIs, real trade-offs — not generic placeholders. + +If context is insufficient, tell the user what additional context you need before proceeding. + +### Step 2: Identify the plan structure + +Based on the user's instructions, determine: +- What phases are already completed (mark ✅) +- What the ignorance reduction questions should cover +- What implementation phases follow +- Any constraints the user specified (e.g., "only phases before Phase 1") + +### Step 3: Generate the plan + +Write the ignorance reduction plan to the specified `FILENAME`. Follow these rules: + +1. **All `Resolution:` sections must be empty** (no text after the label). +2. **Questions must be specific and grounded** in the loaded context — reference actual class names, method signatures, ADRs, configuration options, etc. +3. **Include code sketches** where relevant (proposed API shapes, configuration examples). +4. **Include tables** for comparing options. +5. **Mark spikes** when a question requires hands-on verification. +6. **Include recommendations** — your best assessment of the right answer, clearly labeled as a recommendation (the human may disagree). +7. **Order questions by dependency** — answers to earlier questions inform later ones. +8. **Implementation phases** should have concrete file paths, gating criteria, and test descriptions. +9. **Use the exact formatting conventions** shown in the examples (heading levels, bold labels, horizontal rules, code fences). + +### Step 4: Confirm completion + +After writing the file, report: +- The filename written +- The number of ignorance reduction questions generated +- A one-line summary of the plan's scope + +--- + +## Anti-patterns to avoid + +- ❌ Generic questions like "What framework should we use?" (this should already be decided) +- ❌ Questions that are already answered in loaded ADRs or prior plans +- ❌ Filling in Resolution sections (they must be empty) +- ❌ Skipping code sketches for API design questions +- ❌ Vague implementation phases without file paths or gating criteria +- ❌ Producing a plan without sufficient context (ask for more context instead) diff --git a/skills/shepherd-task-create-issues-from-plan/SKILL.md b/skills/shepherd-task-create-issues-from-plan/SKILL.md new file mode 100644 index 0000000000..3e099d545b --- /dev/null +++ b/skills/shepherd-task-create-issues-from-plan/SKILL.md @@ -0,0 +1,172 @@ +--- +name: shepherd-task-create-issues-from-plan +description: 'Use this skill to turn the ordered implementation section of an ignorance reduction plan into detailed, serial child Task issues under an existing GitHub parent issue, incorporating resolved research, spike artifacts, concrete example-issue style, branch instructions, gating tests, persistent run artifacts, and verified sub-issue ordering. All 12 inputs are required.' +--- + +# Skill: Create Shepherd Task Issues from a Plan + +## Purpose + +Satisfy the `shepherd-task` precondition that a job specification is encoded as an ordered set of GitHub issues. Create one coding-agent-ready child issue for each direct task subsection in an ignorance reduction plan's implementation section, preserving build order and carrying the relevant resolved research into each issue. + +The created issues are specifications, not summaries. A coding agent must be able to complete each issue without guessing about scope, prior decisions, files, tests, or completion criteria. + +## Inputs (all required) + +1. **`REPO`** — GitHub repository in `OWNER/REPO` format. +2. **`BASE_BRANCH`** — Non-`main` topic branch all task PRs target. +3. **`PARENT_ISSUE`** — Positive integer issue number of the existing parent issue that children are linked to. URLs are not accepted. +4. **`PLAN_DIRECTORY`** — Repo-relative path to the directory on `BASE_BRANCH` that contains the plan, spikes, and all supporting resources. +5. **`PLAN_FILE_NAME`** — Name of the ignorance reduction plan file within `PLAN_DIRECTORY`. +6. **`QUESTIONS_SECTION`** — Exact heading of the resolved "questions to answer before writing code" section in the plan. +7. **`IMPLEMENTATION_SECTION`** — Exact heading of the implementation/build-order section whose direct task subsections become child issues. +8. **`EXAMPLE_ISSUES`** — One or more comma-separated full GitHub issue URLs whose title/body style, specificity, and formatting establish the expected standard. Bare issue numbers are not accepted. +9. **`BASE_REMOTE`** — Remote name agents should use (e.g. `upstream` or `origin`). +10. **`ISSUE_TYPE`** — GitHub issue type for children (e.g. `Task`). +11. **`SUPPORTING_ARTIFACTS`** — Repo-relative paths or path constraints for spike reports, prototypes, screenshots, etc. that task issues must cite. +12. **`LOG_DIRECTORY`** — Absolute path to the existing run log directory. The launcher supplies this input; store all drafted issue bodies and the creation ledger here. + +## Fixed behaviors + +- Use the signed-in `gh` CLI for all interactions with GitHub, especially creating the issues and setting them as child issues. +- Create one child per direct task subsection, in plan order. +- Leave every child unassigned. Work starts only when the user assigns each issue in turn. +- State that tasks are assigned, completed, and merged serially in the listed order. +- Require the agent to read the entire plan and explicitly list the exact sections to re-read. +- Include relevant `Resolution:` values and explicit directions for accessing relevant spike research or other artifacts. +- Add discriminating gating tests where they reduce downstream rework. +- Each issue must prominently include text stating that on the base branch, the `PLAN_DIRECTORY` contains the `PLAN_FILE_NAME` and supporting resources. +- Never write vague references such as "read the relevant sections"; enumerate exact headings. +- Never cite a resolution without its concrete value or operational consequence. + +## Bundled examples + +The following files in this skill directory are real prompts from prior campaigns where issues were created manually. Study them to understand the expected level of specificity, the instructions given to the coding agent, and the campaign-specific adaptations made in each case: + +- [examples/01-1682-java-tool-ergonomics.md](./examples/01-1682-java-tool-ergonomics.md) +- [examples/02-1810-java-tool-as-lambda.md](./examples/02-1810-java-tool-as-lambda.md) +- [examples/03-dd-3017826-java-real-estate-demo.md](./examples/03-dd-3017826-java-real-estate-demo.md) +- [examples/04-28-python-agent-demo.md](./examples/04-28-python-agent-demo.md) + +When creating issues, produce issue bodies at least as specific and structured as those examples demand. + +## Procedure + +### Step 1: Validate the invocation + +1. Verify `PARENT_ISSUE` matches `^[1-9][0-9]*$`. Reject URLs and other non-numeric values. +2. Split `EXAMPLE_ISSUES` on commas and trim surrounding whitespace from each item. Verify the list is non-empty and every item is a full GitHub issue URL matching `https://github.com/OWNER/REPO/issues/NUMBER`. Reject bare issue numbers, pull request URLs, and other URL forms. +3. Verify `LOG_DIRECTORY` is an absolute path to an existing writable directory. Create `LOG_DIRECTORY/issue-bodies`; fail before creating any GitHub issues if this cannot be done. +4. Verify `BASE_BRANCH` is not `main` or the repository's default branch. +5. Verify `BASE_BRANCH` exists. +6. Verify `PARENT_ISSUE` exists, is open, and belongs to `REPO`. +7. Discover the repository owner's issue types. Verify `ISSUE_TYPE` exists and is enabled. +8. Read `PLAN_DIRECTORY/PLAN_FILE_NAME` from `BASE_BRANCH`. Prefer `git show "$BASE_BRANCH:$PLAN_DIRECTORY/$PLAN_FILE_NAME"`; fall back to `gh api`. +9. Verify both `QUESTIONS_SECTION` and `IMPLEMENTATION_SECTION` headings occur exactly once. +10. Verify every question that gates implementation has a non-empty resolution block: + - Treat `Resolution:` as a marker, not as a single-line value. Its block includes content on the marker line and all following paragraphs, lists, tables, code blocks, and other Markdown until the next peer question/subsection heading or the end of `QUESTIONS_SECTION`. + - A standalone `**Resolution:**` line followed by substantive block content is resolved. Never classify it as empty merely because no value appears on the marker line, and never use a same-line-only regular expression as the resolution check. + - After ignoring blank lines and Markdown formatting delimiters, classify a resolution as unresolved only when its entire block has no substantive content or explicitly states that the gating decision remains unresolved. + - Before stopping, list each blocking question and quote its complete parsed resolution block, or explicitly state that no resolution block exists. If the block contains a concrete decision, answer, or operational consequence, do not report that question as unresolved. + +### Step 2: Study examples and existing children + +1. Parse each URL in `EXAMPLE_ISSUES` into its owner, repository, and issue number. Fetch every issue body and extract conventions for structure, specificity, and formatting. +2. List current children of `PARENT_ISSUE` via `gh api "repos/$REPO/issues/$PARENT_ISSUE/sub_issues"` and retain their issue IDs and numbers as the pre-creation baseline. +3. Treat issue creation as a one-shot operation, not an idempotent or resumable operation. Do not infer matches between existing children and implementation subsections. + +### Step 3: Build a traceability map + +For each direct child heading beneath `IMPLEMENTATION_SECTION`: + +1. Exact subsection number and title. +2. Files, APIs, behavior to build or change. +3. Tests and gating criteria from the plan. +4. Prerequisite task (if not the first). +5. Every question/resolution from `QUESTIONS_SECTION` that constrains this task — with concrete resolution values. +6. Relevant spike reports, prototypes, screenshots within `PLAN_DIRECTORY` and `SUPPORTING_ARTIFACTS`. +7. Additional gating tests that catch contract, integration, or regression failures before the next serial task starts. + +### Step 4: Draft all issues before creating any + +Each issue body must include: + +- A prominent statement: "On the `BASE_BRANCH` branch, the directory `PLAN_DIRECTORY` contains the plan (`PLAN_FILE_NAME`) and supporting resources (spikes, prototypes, diagrams)." +- Instruction to read the entire plan before working. +- Exact section headings to re-read, with relevant resolved decisions spelled out. +- Explicit paths to spike research and artifacts within `PLAN_DIRECTORY`. +- Branch and execution order instructions. +- Concrete specification of what to build. +- Tests and gating criteria. +- Out-of-scope boundaries. + +Title each issue with its implementation subsection identity and an actionable outcome. + +### Step 5: Create and link issues in order + +Before creating the first issue, write every drafted body to `LOG_DIRECTORY/issue-bodies/NN-SUBSECTION-body.md`, where `NN` is its zero-padded creation order and `SUBSECTION` is a filesystem-safe form of the implementation subsection identity. Verify that the expected number of non-empty body files exists. Never write issue bodies to a temporary directory and never delete these files. + +Create issues with the REST API so the issue type is set at creation. Use `-F/--field`, not `-f/--raw-field`, for `body=@...`; only `-F` reads the body from the referenced file: + +```bash +gh api "repos/$REPO/issues" \ + -X POST \ + -f title="$TITLE" \ + -F "body=@$BODY_FILE" \ + -f type="$ISSUE_TYPE" \ + --jq '{id,number,node_id,html_url,title}' +``` + +Link using integer `sub_issue_id`: + +```bash +printf '{"sub_issue_id": %s}' "$CHILD_ID" | \ + gh api "repos/$REPO/issues/$PARENT_ISSUE/sub_issues" -X POST --input - +``` + +Before creating the first issue, initialize `LOG_DIRECTORY/creation-ledger.json` as a JSON array. Immediately after each successful create call, append the implementation subsection, body-file path, returned issue ID, number, title, and URL with `body_verified=false` and `linked=false`, then persist the ledger. Never keep the ledger only in memory. + +Before linking the new issue, fetch its body from GitHub and verify it exactly equals the complete contents of `BODY_FILE` (allowing only a single trailing newline difference). If it differs, record the observed body in `LOG_DIRECTORY/issue-bodies/NN-SUBSECTION-observed-body.md` and enter the failure flow without creating another issue. After a match, set `body_verified=true` and persist the ledger. Immediately after successfully linking it, set `linked=true` and persist the ledger. + +Create and link one at a time in plan order. On linking failure, retry up to 3 times. If any create, link, or postcondition-verification step fails: + +1. Stop immediately. Do not create, link, edit, or delete anything else. +2. Use read-only GitHub queries to reconcile every ledger entry against current repository and parent-child state. Update each `linked` value from observed server state. +3. Report the failed operation and its error. +4. Print the complete reconciled creation ledger in creation order, including issue number, title, URL, body-file path, `body_verified`, and whether it was linked to `PARENT_ISSUE`. +5. Print one cleanup command per created issue: + + ```bash + gh issue delete ISSUE_NUMBER --repo "$REPO" --yes + ``` + +6. Tell the invoking user that the operation did not complete, that the skill performed no automatic rollback, and that they must delete every issue in the ledger before invoking the skill again. + +If the ledger is empty, explicitly report that no issues were created and no cleanup is required. + +### Step 6: Verify postconditions + +- Relative to the pre-creation baseline, the child count increased by exactly the number of ledger entries. +- Every ledger entry is linked exactly once and corresponds, in creation order, to one implementation subsection. +- The newly linked child order matches plan order. +- Every issue in the ledger has a body exactly matching its persisted body file, has `ISSUE_TYPE`, is open, and has no assignees. + +### Step 7: Report the ordered handoff + +Return: +1. Ordered table of implementation subsection, issue number, title, URL. +2. Comma-separated child issue numbers for `shepherd-task-given-list`. +3. Suggested invocation using `BASE_BRANCH` and `REPO`. + +## Guardrails + +- Never create issues from unresolved implementation decisions. +- Never target `main` or the default branch. +- Never assign the created issues. +- Never collapse multiple subsections into one issue or split one subsection without explicit user approval. +- Never write vague references — enumerate exact headings. +- Never cite a resolution without its concrete value. +- Never invent spike findings — cite available evidence or identify missing evidence as a blocker. +- Never attempt automatic rollback or resume a partial run. Report the creation ledger and cleanup commands, then stop. +- Never rerun after a partial failure until the invoking user confirms that every issue in the creation ledger has been deleted. +- Do not run `shepherd-task` or assign Copilot. This skill only creates and verifies the ordered issue backlog. diff --git a/skills/shepherd-task-create-issues-from-plan/examples/01-1682-java-tool-ergonomics.md b/skills/shepherd-task-create-issues-from-plan/examples/01-1682-java-tool-ergonomics.md new file mode 100644 index 0000000000..4a880ee1b0 --- /dev/null +++ b/skills/shepherd-task-create-issues-from-plan/examples/01-1682-java-tool-ergonomics.md @@ -0,0 +1,27 @@ +# Example 01: 1682-java-tool-ergonomics-prompts-remove-before-merge + +Read `copilot-sdk/1682-java-tool-ergonomics-prompts-remove-before-merge/dd-3018003-ignorance-reduction-for-implementation-plan.md` (the plan) to get the context. + +✅✅Pay very close attention to each **Resolution:** to the questions in `Phase 3 — Ignorance reduction: questions to answer before writing code`. + +In this session, we are creating issues of type **Task** as children of Epic https://github.com/github/copilot-sdk/issues/1682 . These issues correspond to the sub-sections in `## Phase 4 — Implementation (the build order)`. + +I want you to use `gh` to create issues as sub-issues of 1682. + +- I will dispatch these issues by assigning them to an agent. Leave them unassigned for now. + +- The description you put in the issues you create is the specification for the work the agent assigned to the issue must do. Therefore, you need to be very specific and targeted when writing the description. + +- Set the type to **Task**. + +- Each issue will be worked on and resolved in serial. You can assume: + - All issues will operate in this topic branch of `upstream`: `edburns/1682-java-tool-ergonomics`. + + - The assignee agent has access to the topic branch and, most importantly, to the prompts in the directory `1682-java-tool-ergonomics-prompts-remove-before-merge`, which includes the plan. Therefore, you can instruct the agent to access the plan by reference, you don't need to reference it by value directly in the issue. + +- The sub-sections of `## Phase 4 — Implementation (the build order)` have essential details for each corresponding **Task**. + +- Instruct the agent to read the entire plan before working. + +- Instruct the agent to carefully re-read the relevant sections of the of the plan. ❌❌You can't just say, "read the relevant sections of the plan". ✅✅In the description of the issue, list exactly what sections of the plan apply to this particular issue. +- Though the sub-sections of `## Phase 4 — Implementation (the build order)` have essential details for each corresponding **Task**, I want you to carefully consider if there additional gating tests you can add so that as the agent proceeds from one task to the next, the likelihood of rework is reduced. diff --git a/skills/shepherd-task-create-issues-from-plan/examples/02-1810-java-tool-as-lambda.md b/skills/shepherd-task-create-issues-from-plan/examples/02-1810-java-tool-as-lambda.md new file mode 100644 index 0000000000..eced657994 --- /dev/null +++ b/skills/shepherd-task-create-issues-from-plan/examples/02-1810-java-tool-as-lambda.md @@ -0,0 +1,23 @@ +# Example 02: 1810-java-tool-ergonomics-tool-as-lambda-remove-before-merge + +Carefully read the plan `copilot-sdk/1810-java-tool-ergonomics-tool-as-lambda-remove-before-merge/1810-ignorance-reduction-for-implementation-plan.md`. + +We have completed Phases 1 - 3. We are now on Phase 4. + +For each sub item in Phase 4, file a child issue of Feature #1810. + +Use `gh` to look at Epic #1682 and its child issues for the standard of verbosity and concreteness for issue descriptions. + +The issue should be assignable to a coding agent and there must be sufficient detail in the description for the agent to proceed to completion. + +Pull out the specific relevant content from Phase 3 as it applies to this current issue in Phase 4. + +Tell the agent to take `edburns/1810-java-tool-ergonomics-tool-as-lambda` as the base branch. When I assign the issue to the coding agent, I will set this as the base branch. + +Tell the agent to read `1810-java-tool-ergonomics-tool-as-lambda-remove-before-merge/1810-ignorance-reduction-for-implementation-plan.md`, which is present on the branch, for context. + +Tell the agent which sub-item within Phase 4 they are solving. + +Tell the agent that each issue will be assigned, resolved and merged to the base branch in the order in which they appear in the plan. + +No work is to start until I assign each issue to the coding agent, in turn. diff --git a/skills/shepherd-task-create-issues-from-plan/examples/03-dd-3017826-java-real-estate-demo.md b/skills/shepherd-task-create-issues-from-plan/examples/03-dd-3017826-java-real-estate-demo.md new file mode 100644 index 0000000000..0c94589dc5 --- /dev/null +++ b/skills/shepherd-task-create-issues-from-plan/examples/03-dd-3017826-java-real-estate-demo.md @@ -0,0 +1,25 @@ +# Example 03: dd-3017826-java-real-estate-demo-remove-before-merge + +Read `dd-3017826-java-real-estate-demo-remove-before-merge\dd-3017826-java-real-estate-demo-ignorance-reduction-plan.md` (the plan) to get the context. + +✅✅Pay very close attention to each **Resolution:** to the questions in `Phase 2 — Ignorance reduction: questions to answer before writing code`. + +In this session, we are creating sub-issues as children of https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/2 . These issues correspond to the sub-sections in `## Phase 3 — Implementation (build order)`. + +I want you to use `gh` to create issues as sub-issues of #2. + +- I will dispatch these issues by assigning them to an agent. Leave them unassigned for now. + +- The description you put in the issues you create is the specification for the work the agent assigned to the issue must do. Therefore, you need to be very specific and targeted when writing the description. + +- Each issue will be worked on and resolved in serial. You can assume: + - All issues will operate in this topic branch of `origin`: `edburns/2-build-out-demo`. + + - The assignee agent has access to the topic branch and, most importantly, to the prompts in the directory `dd-3017826-java-real-estate-demo-remove-before-merge`, which includes the plan. Therefore, you can instruct the agent to access the plan by reference, you don't need to reference it by value directly in the issue. + +- The sub-sections of `## Phase 3 — Implementation (build order)` have essential details for each corresponding sub-issue. + +- Instruct the agent to read the entire plan before working. + +- Instruct the agent to carefully re-read the relevant sections of the of the plan. ❌❌You can't just say, "read the relevant sections of the plan". ✅✅In the description of the issue, list exactly what sections of the plan apply to this particular issue. +- Though the sub-sections of `## Phase 3 — Implementation (build order)` have essential details for each corresponding sub-issue, I want you to carefully consider if there additional gating tests you can add so that as the agent proceeds from one task to the next, the likelihood of rework is reduced. diff --git a/skills/shepherd-task-create-issues-from-plan/examples/04-28-python-agent-demo.md b/skills/shepherd-task-create-issues-from-plan/examples/04-28-python-agent-demo.md new file mode 100644 index 0000000000..9d9ac66416 --- /dev/null +++ b/skills/shepherd-task-create-issues-from-plan/examples/04-28-python-agent-demo.md @@ -0,0 +1,29 @@ +# Example 04: 28-python-agent-demo-remove-before-merge + +Read `28-python-agent-demo-remove-before-merge/28-python-agent-demo-ignorance-reduction-plan.md` (the plan) to get the context. + +✅✅Pay very close attention to each **Resolution:** to the questions in `Phase 2 — Ignorance reduction: questions to answer before writing code`. These pertain to the implementation tasks in Phase 3. ✅✅✅ Make sure to put into the description explicit instructions on how to access the relevant spike research. + +In this session, we are creating sub-issues as children of https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/28 . These issues correspond to the sub-sections in `## Phase 3 — Implementation (build order)`. + +I want you to use `gh` to create issues as sub-issues of #28 + +- I will dispatch these issues by assigning them to an agent. Leave them unassigned for now. + +- The description you put in the issues you create is the specification for the work the agent assigned to the issue must do. Therefore, you need to be very specific and targeted when writing the description. + +- Again, ✅✅For each **Resolution:** to the questions in `Phase 2 — Ignorance reduction: questions to answer before writing code`, include the relevant Resolution values in the issue description. These pertain to the implementation tasks in Phase 3. ✅✅✅ Make sure to put into the description explicit instructions on how to access the relevant spike research for that task. + +- For tasks dealing with the UI, make sure to refer to the relevant screengrabs, but only ones within `28-python-agent-demo-remove-before-merge/`. + +- Each issue will be worked on and resolved in serial. You can assume: + - All issues will operate in this topic branch of `origin`: `edburns/28-python-agent-demo`. + + - The assignee agent has access to the topic branch and, most importantly, to the prompts in the directory `28-python-agent-demo-remove-before-merge/`, which includes the plan. Therefore, you can instruct the agent to access the plan by reference, you don't need to reference it by value directly in the issue. + +- The sub-sections of `## Phase 3 — Implementation (build order)` have essential details for each corresponding sub-issue. + +- Instruct the agent to read the entire plan before working. + +- Instruct the agent to carefully re-read the relevant sections of the of the plan. ❌❌You can't just say, "read the relevant sections of the plan". ✅✅In the description of the issue, list exactly what sections of the plan apply to this particular issue. +- Though the sub-sections of `## Phase 3 — Implementation (build order)` have essential details for each corresponding sub-issue, I want you to carefully consider if there additional gating tests you can add so that as the agent proceeds from one task to the next, the likelihood of rework is reduced. diff --git a/skills/shepherd-task-create-post-mortem/SKILL.md b/skills/shepherd-task-create-post-mortem/SKILL.md new file mode 100644 index 0000000000..d19d3314ea --- /dev/null +++ b/skills/shepherd-task-create-post-mortem/SKILL.md @@ -0,0 +1,134 @@ +--- +name: shepherd-task-create-post-mortem +description: 'Create a structured post-mortem report from shepherd-task run artifacts, including metrics, timeline, failures, and actionable recommendations.' +--- + +# Skill: Create Shepherd Task Post-Mortem + +## Purpose + +Create a comprehensive post-mortem report for a completed (or failed) `shepherd-task` run. + +This skill is designed to be invoked from `shepherd-task-given-list.ps1` / `shepherd-task-given-list.sh` in a `finally` / `trap EXIT` path so it runs for **all outcomes**. + +--- + +## Inputs + +- `SHEPHERD_LOG_DIR` (**required**) + Absolute path to the shepherd run log directory (for example: `C:\Users\edburns\workareas\BRK206-02\28-python-agent-demo-remove-before-merge\shepherd-tasks-20260718-1827`). +- `SCRIPT_EXIT_CODE` (optional but recommended) + Exit code from the caller script, to classify success vs failure. +- `TASK_ISSUES` (optional) +- `BASE_BRANCH` (optional) +- `REPO` (optional) + +--- + +## Required Output + +Write the report to: + +`\YYYYMMDD-HHMM-post-mortem.md` + +Use local time for `YYYYMMDD-HHMM`. + +--- + +## Bundled Examples (style + structure references) + +Use both of these as concrete examples: + +1. `examples/dd-3029269-post-mortem-report.md` + (prior complete Java post-mortem; canonical sectioning and depth) +2. `examples/28-python-agent-demo-post-mortem.md` + (Python shepherd-task run post-mortem) + +Match their structure and tone: concise executive summary, clear sectioning, metrics tables, explicit timeline, and action-oriented recommendations. + +--- + +## Data Collection Procedure + +Given `SHEPHERD_LOG_DIR`: + +1. Validate the directory exists; fail clearly if it does not. +2. Collect all run artifacts from that directory: + - `phase1-task-*.json`, `phase2-task-*.json` + - `phase1-task-*.md`, `phase2-task-*.md` + - any supporting markdown notes +3. Determine the parent campaign directory (`PARENT_DIR = dirname(SHEPHERD_LOG_DIR)`), then collect context files there: + - `*memory*.md` (if present) + - `*prompts.md` (if present) + - `*job-logs.txt` (if present) +4. Extract quantitative metrics from JSON/MD artifacts: + - issues/PRs touched + - per-task phase durations + - review rounds (`Comments generated`) + - success/failure and failure signatures + - idle/timeout markers + - token usage where available (`assistant.message.outputTokens` / `inputTokens`) +5. Build the report so it is useful for both: + - **successful runs** (throughput/convergence/quality) + - **failed runs** (root cause and corrective actions) + +--- + +## Report Structure (required) + +1. **Section 1: Executive Summary** + High-level outcome, completion rate, elapsed time, key totals. +2. **Section 2: System Architecture** + CCA, CCRA, Local Copilot CLI responsibilities. +3. **Section 3: Per-Task Metrics** + Table with issue, PR, phase timings, rounds, comments, result. +4. **Section 4: Aggregate Statistics** + Totals/averages and convergence signals. +5. **Section 5: AI Credits and Token Usage** + Include measured values; state clearly when data is unavailable. +6. **Section 6: Wall-Clock Timeline** + Batch windows and notable events. +7. **Section 7: Failure Analysis (if any)** + Root cause(s), evidence, and fixes. +8. **Section 8: Observations and Recommendations** + What worked, what failed, and specific script/skill improvements. + +Use Markdown tables for metrics and keep assertions tied to observable logs. + +### Link Formatting (required) + +Whenever an issue or PR is referenced in the report body, render it as a Markdown hyperlink using `REPO`: + +- Issue format: `[#123](https://github.com//issues/123)` +- PR format: `[#456](https://github.com//pull/456)` + +Apply this consistently in narrative text, metric tables, legends, and comparisons. +Do not leave plain-text references like `#123` or `PR #456` when `REPO` is known. +Do not nest Markdown links inside another Markdown link. + +For Table of Contents entries, use plain text labels for issue/PR identifiers (for example, `3.1 — Issue #34 / PR #44`) and link only to the local anchor target. + +--- + +## Invocation Pattern + +The caller should invoke this skill with a prompt like: + +```text +Invoke skill `shepherd-task-create-post-mortem` with these inputs: + +- SHEPHERD_LOG_DIR: +- SCRIPT_EXIT_CODE: +- TASK_ISSUES: +- BASE_BRANCH: +- REPO: +``` + +--- + +## Guardrails + +- Always produce a post-mortem file, even when the run fails. +- Do not require GitHub API/network calls unless local artifacts are insufficient. +- Prefer evidence from local run logs over assumptions. +- Keep report content factual and reproducible from the captured artifacts. diff --git a/skills/shepherd-task-create-post-mortem/examples/28-python-agent-demo-post-mortem.md b/skills/shepherd-task-create-post-mortem/examples/28-python-agent-demo-post-mortem.md new file mode 100644 index 0000000000..642f50f632 --- /dev/null +++ b/skills/shepherd-task-create-post-mortem/examples/28-python-agent-demo-post-mortem.md @@ -0,0 +1,286 @@ +# Post-Mortem Report: Shepherd-Task Runs for Python Demo Tasks [#34-#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues) + +**Campaign:** Python agent demo shepherding in `28-python-agent-demo-remove-before-merge` +**Report generated:** 2026-07-20 +**Period covered:** 2026-07-17 19:34 ET -> 2026-07-18 22:34 ET +**Primary successful batch:** `shepherd-tasks-20260718-1827` + +## Table of Contents + +- [Section 1: Executive Summary](#section-1-executive-summary) +- [Section 2: System Architecture](#section-2-system-architecture) + - [2.1 Copilot Coding Agent (CCA)](#21-copilot-coding-agent-cca) + - [2.2 Copilot Code Review Agent (CCRA)](#22-copilot-code-review-agent-ccra) + - [2.3 Local Copilot CLI (Shepherd)](#23-local-copilot-cli-shepherd) +- [Section 3: Per-Task Metrics](#section-3-per-task-metrics) + - [Issue Legend](#issue-legend) + - [3.1 — Issue #34 / PR #44](#31--issue-34--pr-44) + - [3.2 — Issue #35 / PR #45](#32--issue-35--pr-45) + - [3.3 — Issue #36 / PR #46](#33--issue-36--pr-46) + - [3.4 — Issue #37 / PR #47](#34--issue-37--pr-47) + - [3.5 — Issue #38 / PR #48](#35--issue-38--pr-48) + - [3.6 — Issue #39 / PR #49](#36--issue-39--pr-49) +- [Section 4: Aggregate Statistics](#section-4-aggregate-statistics) + - [4.1 Final Batch Summary](#41-final-batch-summary) + - [4.2 Cross-Batch Outcomes](#42-cross-batch-outcomes) + - [4.3 Convergence Snapshot](#43-convergence-snapshot) +- [Section 5: AI Credits and Token Usage](#section-5-ai-credits-and-token-usage) + - [5.1 Local Copilot CLI Tokens](#51-local-copilot-cli-tokens) + - [5.2 Credit Visibility Limits](#52-credit-visibility-limits) +- [Section 6: Wall-Clock Timeline](#section-6-wall-clock-timeline) + - [6.1 Batch Timeline](#61-batch-timeline) + - [6.2 Final Batch Timeline](#62-final-batch-timeline) +- [Section 7: Failure Analysis Before Final Success](#section-7-failure-analysis-before-final-success) + - [7.1 Idle-Kill Timeout Pattern](#71-idle-kill-timeout-pattern) + - [7.2 Missing Initial Copilot Review Request](#72-missing-initial-copilot-review-request) + - [7.3 Intermediate Stabilization Run](#73-intermediate-stabilization-run) +- [Section 8: Observations and Recommendations](#section-8-observations-and-recommendations) + - [8.1 What Worked Well](#81-what-worked-well) + - [8.2 What Didn’t Work Well](#82-what-didnt-work-well) + - [8.3 Recommendations](#83-recommendations) + - [8.4 Comparison to Prior Java Run](#84-comparison-to-prior-java-run) + +--- + +## Section 1: Executive Summary + +The shepherding campaign converged to full success after three failed/partial iterations. The final run (`shepherd-tasks-20260718-1827`) merged all target Python tasks ([#34](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/34), [#35](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/35), [#36](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/36), [#37](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/37), [#38](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/38), [#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39)), with terminal output `=== All tasks shepherded successfully ===` in `20260718-1826-job-logs.txt`. + +| Metric | Value | +|--------|-------| +| Target tasks in final run | 6 ([#34](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/34)-[#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39)) | +| Completed and merged | 6/6 (100%) | +| Final run elapsed | ~4h 07m (18:27 -> 22:34 ET) | +| Total CCRA rounds (final run) | 20 | +| Total CCRA comments (final run) | 30 | +| Average task duration (final run) | ~40m 57s | +| Idle-kill failures (final run) | 0 | +| Local CLI output tokens (final run JSON logs) | 136,022 | + +Earlier runs (`20260717-1936`, `20260717-2022`, `20260718-1648`) provided failure evidence and fixes that enabled final success. + +--- + +## Section 2: System Architecture + +### 2.1 Copilot Coding Agent (CCA) + +CCA created/updated task PRs and performed initial implementation on GitHub infrastructure. In these runs, relevant PRs were [#42](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/42)-[#49](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/49). + +### 2.2 Copilot Code Review Agent (CCRA) + +CCRA (`copilot-pull-request-reviewer[bot]`) produced iterative review rounds with `Comments generated` summaries. It was the primary convergence signal for phase 2. + +### 2.3 Local Copilot CLI (Shepherd) + +`copilot --yolo` executed two shepherd skills, orchestrated local fixes, re-requested reviews, and merged PRs to `edburns/28-python-agent-demo` after clean review state. + +--- + +## Section 3: Per-Task Metrics + +### Issue Legend + +| Issue | PR | Notes | +|------:|---:|-------| +| [#34](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/34) | [#44](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/44) | Phase 1 skipped; PR pre-existed from earlier run | +| [#35](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/35) | [#45](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/45) | Transient local path lookup errors recovered | +| [#36](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/36) | [#46](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/46) | Longest phase 1 in final run before [#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) | +| [#37](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/37) | [#47](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/47) | Fastest end-to-end completion | +| [#38](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/38) | [#48](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/48) | Long phase 2 despite low comment count | +| [#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) | [#49](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/49) | Deepest review loop in final run | + +### 3.1 — Issue [#34](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/34) / PR [#44](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/44) + +| Metric | Value | +|--------|-------| +| Phase 1 duration | skipped (PR already existed) | +| Phase 2 duration | 24m 17s | +| Total duration | 24m 17s | +| CCRA rounds | 4 | +| CCRA comments | 8 | +| Outcome | merged | + +### 3.2 — Issue [#35](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/35) / PR [#45](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/45) + +| Metric | Value | +|--------|-------| +| Phase 1 duration | 14m 41s | +| Phase 2 duration | 14m 23s | +| Total duration | 29m 04s | +| CCRA rounds | 5 | +| CCRA comments | 5 | +| Outcome | merged | + +Phase 2 logs include four transient `Path does not exist` tool failures during local reads; run still converged and merged. + +### 3.3 — Issue [#36](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/36) / PR [#46](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/46) + +| Metric | Value | +|--------|-------| +| Phase 1 duration | 39m 44s | +| Phase 2 duration | 17m 47s | +| Total duration | 57m 31s | +| CCRA rounds | 3 | +| CCRA comments | 5 | +| Outcome | merged | + +### 3.4 — Issue [#37](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/37) / PR [#47](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/47) + +| Metric | Value | +|--------|-------| +| Phase 1 duration | 14m 23s | +| Phase 2 duration | 1m 26s | +| Total duration | 15m 49s | +| CCRA rounds | 0 | +| CCRA comments | 0 | +| Outcome | merged | + +### 3.5 — Issue [#38](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/38) / PR [#48](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/48) + +| Metric | Value | +|--------|-------| +| Phase 1 duration | 10m 35s | +| Phase 2 duration | 41m 11s | +| Total duration | 51m 46s | +| CCRA rounds | 1 | +| CCRA comments | 2 | +| Outcome | merged | + +### 3.6 — Issue [#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) / PR [#49](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/49) + +| Metric | Value | +|--------|-------| +| Phase 1 duration | 27m 53s | +| Phase 2 duration | 39m 20s | +| Total duration | 1h 07m 13s | +| CCRA rounds | 7 | +| CCRA comments | 10 | +| Outcome | merged | + +--- + +## Section 4: Aggregate Statistics + +### 4.1 Final Batch Summary + +| Metric | Value | +|--------|-------| +| Tasks | 6 | +| Merged PRs | 6 | +| CCRA rounds | 20 | +| CCRA comments | 30 | +| Avg rounds/task | 3.33 | +| Avg comments/task | 5.00 | +| Avg comments/round | 1.50 | +| Tasks with zero comments | 1 ([#37](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/37)) | +| Longest task | [#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) (1h 07m 13s) | +| Shortest task | [#37](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/37) (15m 49s) | + +### 4.2 Cross-Batch Outcomes + +| Directory | JSON sessions | Outcome | +|-----------|---------------|---------| +| `shepherd-tasks-20260717-1936` | 2 | failed (PR [#42](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/42) left OPEN) | +| `shepherd-tasks-20260717-2022` | 1 | failed (idle-kill while waiting for review) | +| `shepherd-tasks-20260718-1648` | 5 (+ one empty phase2 JSON) | partial success ([#41](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/41) and [#33](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/33) merged) | +| `shepherd-tasks-20260718-1827` | 11 | full success ([#34](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/34)-[#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) merged) | + +### 4.3 Convergence Snapshot + +- **Strong convergence:** [#37](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/37) (0 comments), [#36](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/36) (3 rounds, 5 comments). +- **Moderate convergence:** [#34](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/34) and [#35](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/35). +- **Long convergence tail:** [#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) (7 rounds). +- **Throughput bottleneck:** strictly serialized issue processing; wall clock scales with per-issue sum. + +--- + +## Section 5: AI Credits and Token Usage + +### 5.1 Local Copilot CLI Tokens + +| Scope | Output tokens | +|-------|---------------| +| Final successful batch (`20260718-1827`) | 136,022 | +| All four referenced run directories | 186,132 | + +### 5.2 Credit Visibility Limits + +CCA/CCRA billing-credit totals were not present in local artifacts. This report uses rounds/comments and local token usage as measurable proxies. + +Additional observability limitation: `20260718-1855-copilot-cli-otel-not-working.md` documents OTEL file export not flushing in piped-stdin mode ([copilot-agent-runtime#13047](https://github.com/github/copilot-agent-runtime/issues/13047)). + +--- + +## Section 6: Wall-Clock Timeline + +### 6.1 Batch Timeline + +| Batch | Window (ET) | Summary | +|------|--------------|---------| +| `20260717-1936` | ~19:36-19:59 | First phase 2 failure on [#41](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/41) | +| `20260717-2022` | ~20:23-20:26 | Retry failed despite review arrival | +| `20260718-1648` | ~16:49-18:09 | Stabilization run; [#41](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/41) and [#33](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/33) merged | +| `20260718-1827` | ~18:27-22:34 | Final full success for [#34](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/34)-[#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) | + +### 6.2 Final Batch Timeline + +- Start marker: `phase2-task-20260718-1827-34.*` (~18:27 ET). +- End marker: `phase2-task-20260718-2155-39.*` duration 39m 20s (ending ~22:34 ET). +- Total elapsed: ~4h 07m. + +--- + +## Section 7: Failure Analysis Before Final Success + +### 7.1 Idle-Kill Timeout Pattern + +Failed phase 2 runs show this pattern: + +1. Poll command exceeds `initial_wait` and backgrounds. +2. Assistant reports "I'll check back..." then goes idle. +3. `assistant.idle` appears in JSON logs. +4. Session terminates before workflow can continue, leaving PR open. + +### 7.2 Missing Initial Copilot Review Request + +Earlier phase-2 flow assumed `gh pr ready` auto-requested Copilot review. It does not. This created wait loops for reviews that were never requested. + +Applied fix: explicit `gh pr edit $PR_NUMBER --add-reviewer "copilot-pull-request-reviewer"` in phase-2 step 1. + +### 7.3 Intermediate Stabilization Run + +`shepherd-tasks-20260718-1648` validated key fixes: + +- [#41](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/41) merged with `Comments generated: 0`. +- [#33](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/33) converged through 8 review rounds then merged. + +That run reduced unknowns and set up the final successful batch. + +--- + +## Section 8: Observations and Recommendations + +### 8.1 What Worked Well + +- Final run achieved full completion (6/6 merged). +- Idempotent skip behavior (existing PRs) supported restartability. +- Review/fix loop handled both low- and high-iteration tasks. + +### 8.2 What Didn’t Work Well + +- Early runs were vulnerable to idle termination behavior. +- Review polling logic was verbose and produced long wait overhead. +- OTEL capture gap reduced telemetry quality. + +### 8.3 Recommendations + +1. Keep explicit initial reviewer request in phase 2 as mandatory. +2. Keep anti-idle directives in all shepherd skills. +3. Add explicit exit-code verification for `gh pr edit --base` in `shepherd-task.ps1`. +4. Keep writing top-level `*job-logs.txt` for every batch (success and failure). +5. Continue reporting rounds/comments/tokens as core performance indicators. + +### 8.4 Comparison to Prior Java Run + +The Java post-mortem (`dd-3029269-post-mortem-report.md`) showed long-lived sessions and deep review cycles (for example, 7 rounds on PR [#43](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/pull/43)). After stabilization, the Python campaign reached comparable convergence behavior (notably [#39](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/39) with 7 rounds) and delivered full merge completion for its target set. diff --git a/skills/shepherd-task-create-post-mortem/examples/dd-3029269-post-mortem-report.md b/skills/shepherd-task-create-post-mortem/examples/dd-3029269-post-mortem-report.md new file mode 100644 index 0000000000..69d4dde650 --- /dev/null +++ b/skills/shepherd-task-create-post-mortem/examples/dd-3029269-post-mortem-report.md @@ -0,0 +1,707 @@ +# Post-Mortem Report: Agentic Development of Epic [#2](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/2) + +**Epic:** [Java demo implementation](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/2) +**Report generated:** 2026-07-09 +**Period covered:** 2026-07-08 16:03 UTC → 2026-07-09 13:02 UTC + +## Table of Contents + +- [Section 1: Executive Summary](#section-1-executive-summary) +- [Section 2: System Architecture](#section-2-system-architecture) + - [2.1 Copilot Coding Agent (CCA)](#21-copilot-coding-agent-cca) + - [2.2 Copilot Code Review Agent (CCRA)](#22-copilot-code-review-agent-ccra) + - [2.3 Local Copilot CLI (Shepherd)](#23-local-copilot-cli-shepherd) +- [Section 3: Per-Task Metrics](#section-3-per-task-metrics) + - [Issue Legend](#issue-legend) + - [3.1 — Issue #13 / PR #14: Project Scaffolding](#31--issue-13--pr-14-project-scaffolding) + - [3.2 — Issue #4 / PR #15: Domain Model & Database Seeding](#32--issue-4--pr-15-domain-model--database-seeding) + - [3.3 — Issue #5 / PR #16: Core Agent Infrastructure](#33--issue-5--pr-16-core-agent-infrastructure) + - [3.4 — Issue #6 / PR #17: WebSocket Push Infrastructure](#34--issue-6--pr-17-websocket-push-infrastructure) + - [3.5 — Issue #7 / PR #18: JSF Pipeline View](#35--issue-7--pr-18-jsf-pipeline-view) + - [3.6 — Issue #20 / PR #21: Dynamic UI Updates](#36--issue-20--pr-21-dynamic-ui-updates) + - [3.7 — Issue #9 / PR #22: Agent Detail View](#37--issue-9--pr-22-agent-detail-view) + - [3.8 — Issue #10 / PR #23: End-to-End Integration Testing](#38--issue-10--pr-23-end-to-end-integration-testing) + - [3.9 — Issue #11 / PR #24: Demo Polish and README](#39--issue-11--pr-24-demo-polish-and-readme) +- [Section 4: Aggregate Statistics](#section-4-aggregate-statistics) + - [4.1 Summary Table](#41-summary-table) + - [4.2 Aggregate Metrics](#42-aggregate-metrics) + - [4.3 Convergence Analysis](#43-convergence-analysis) +- [Section 5: AI Credits](#section-5-ai-credits) + - [5.1 Local Copilot CLI Token Usage](#51-local-copilot-cli-token-usage) + - [5.2 CCA and CCRA Credits](#52-cca-and-ccra-credits) +- [Section 6: Wall-Clock Timeline](#section-6-wall-clock-timeline) + - [6.1 Overall](#61-overall) + - [6.2 Batch Timeline](#62-batch-timeline) + - [6.3 Per-Issue Timeline](#63-per-issue-timeline) + - [6.4 Notable Events](#64-notable-events) +- [Section 7: Human-Directed Changes After the Agentic Work Completed](#section-7-human-directed-changes-after-the-agentic-work-completed) + - [7.1 Pipeline Layout Restructure (commit `f6d9ddb`)](#71-pipeline-layout-restructure-commit-f6d9ddb) + - [7.2 Canned Query "+" Button (commit `d7e2b56`)](#72-canned-query--button-commit-d7e2b56) + - [7.3 Dashboard Sidebar (commit `c6168d0`)](#73-dashboard-sidebar-commit-c6168d0) + - [7.4 How to Improve the Issues So That the Human-Directed Changes Would Be Less](#74-how-to-improve-the-issues-so-that-the-human-directed-changes-would-be-less) +- [Section 8: Observations and Recommendations](#section-8-observations-and-recommendations) + - [8.1 What Worked Well](#81-what-worked-well) + - [8.2 What Didn't Work Well](#82-what-didnt-work-well) + - [8.3 Recommendations](#83-recommendations) + - [For the CCA (Copilot Coding Agent)](#for-the-cca-copilot-coding-agent) + - [For the CCRA (Copilot Code Review Agent)](#for-the-ccra-copilot-code-review-agent) + - [For the Local Copilot CLI Shepherd](#for-the-local-copilot-cli-shepherd) + - [For the Shepherd Orchestration Script](#for-the-shepherd-orchestration-script) + - [8.4 Patterns Observed](#84-patterns-observed) + +--- + +## Section 1: Executive Summary + +Epic [#2](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/2) tasked a three-agent pipeline with implementing a complete Java EE 11 + OpenLiberty port of the BRK206 real-estate demo across 9 discrete sub-issues (sections 3.1–3.9 of the implementation plan). Two additional sub-issues were aborted before completion and excluded from this analysis. + +| Metric | Value | +|--------|-------| +| Sub-issues attempted | 11 | +| Sub-issues completed (merged) | 9 | +| Sub-issues aborted | 2 ([#3](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/3), [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8)) | +| Total PRs merged | 9 (PR [#14](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/14)–18, [#21](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/21)–24) | +| Total wall-clock time | ~21 hours (2026-07-08 16:03 – 2026-07-09 13:02 UTC) | +| Total lines added by CCA (across all PRs) | 7,453 | +| Total lines deleted | 124 | +| Total CCRA review rounds | 47 | +| Total inline review comments | 287 | +| Local CLI output tokens | 467,288 | +| Tasks hitting 8-round CCRA cap | 2 (issues [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5), [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6)) | +| Manual interventions | 1 (abort of issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) / PR [#19](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/19)) | + +All 9 non-aborted tasks resulted in merged PRs. No task required manual code fixes by the human developer. + +--- + +## Section 2: System Architecture + +The pipeline consisted of three collaborating agents: + +### 2.1 Copilot Coding Agent (CCA) + +The CCA performed the initial implementation of each issue. It ran on GitHub's infrastructure, triggered by assigning the issue to Copilot. For 8 of 9 tasks, the `shepherd-task-to-ready` skill (phase 1) monitored the CCA run, polled for PR creation and CI completion, and approved any pending workflow runs. Issue [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13)'s CCA had already completed before the first shepherd batch started. + +The CCA produced draft PRs targeting the `edburns/2-build-out-demo` base branch. Initial implementations ranged from 1 commit (issue [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11)) to 7 commits (issue [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20)) before any CCRA involvement. + +### 2.2 Copilot Code Review Agent (CCRA) + +The CCRA (`copilot-pull-request-reviewer[bot]`) reviewed each PR once it was marked "Ready for Review." It posted inline comments identifying bugs, missing requirements, style violations, and constraint violations. The CCRA ran on GitHub's infrastructure asynchronously, typically completing a review within 5–15 minutes of being requested. + +### 2.3 Local Copilot CLI (Shepherd) + +The local CLI (`copilot --yolo`) ran the `shepherd-task-from-ready-to-merged-to-base` skill (phase 2). For each CCRA review batch, it: + +1. Fetched and read all open review comments +2. Applied each fix locally (via `edit`, `create`, or `powershell` tool calls in a worktree) +3. Made a single commit per batch and pushed to the head branch +4. Re-requested a CCRA review +5. Repeated until no comments remained or 8 rounds were reached +6. Merged the PR via `gh pr merge` + +The local CLI ran in `--yolo` mode, autonomously approving all tool permission requests. Each phase-2 session was a single long-lived `copilot` process that polled GitHub for CCRA completion between rounds. + +--- + +## Section 3: Per-Task Metrics + +### Issue Legend + +| Issue | Section | Title | PR | +|-------|---------|-------|----| +| [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) | 3.1 | Project scaffolding: Maven, server.xml, empty source dirs | [#14](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/14) | +| [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) | 3.2 | Domain model & database seeding: JPA entities, Jakarta Data, JSON loader | [#15](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/15) | +| [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) | 3.3 | Core agent infrastructure: Phase enum, Agent, AppState, CopilotClientProducer, tools | [#16](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/16) | +| [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) | 3.4 | WebSocket push infrastructure: `f:websocket` for real-time UI | [#17](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/17) | +| [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) | 3.5 | JSF pipeline view: static layout with PrimeFaces | [#18](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/18) | +| [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) | 3.6 | Dynamic UI updates: WebSocket-driven re-render with CSS transitions | [#21](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/21) | +| [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) | 3.7 | Agent detail view: side panel with session events, tool calls, report | [#22](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/22) | +| [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10) | 3.8 | End-to-end integration testing: full pipeline validation | [#23](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/23) | +| [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) | 3.9 | Demo polish and README: error handling, auto-removal, docs | [#24](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/24) | + +--- + +### 3.1 — Issue [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) / PR [#14](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/14): Project Scaffolding + +**Phase 1 (CCA):** PR created at 2026-07-08 00:25 UTC — before the first shepherd batch. CCA created the Maven + OpenLiberty skeleton independently. + +**Phase 2 (CCRA + Local CLI):** Shepherd batch `shepherd-tasks-20260708-1203`, session 22m 32s. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 2 | +| CCRA rounds | 1 | +| Local CLI fix commits | 1 | +| Total PR commits | 3 | +| 8-round cap hit? | No | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 143 | +| Deletions | 0 | +| Changed files | 7 | +| Inline CCRA comments | 2 | +| Merge time | 2026-07-08 16:25 UTC | +| Wall-clock (phase 2 only) | 22 min | + +#### Assessment + +The scaffolding task was the simplest of all sub-issues — a Maven POM, `server.xml`, and empty source directories. The CCA produced correct structure on the first try. The single CCRA round caught 2 minor issues (likely naming or packaging), resolved in 1 commit. The low comment count (2) and single review round indicate strong CCA accuracy for this well-bounded task. No constraint violations observed; the output correctly targeted EE 11 and OpenLiberty. + +--- + +### 3.2 — Issue [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) / PR [#15](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/15): Domain Model & Database Seeding + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1233` / `shepherd-tasks-20260708-1244`. A quick 13-second phase-1 run (20260708-1234) was aborted and restarted at 16:44 (20260708-1244), running 47 min. CCA produced PR [#15](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/15) at 16:45 UTC. + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1340`, session 57m 46s. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 2 | +| CCRA rounds | 7 | +| Local CLI fix commits | 7 | +| Total PR commits | 9 | +| 8-round cap hit? | No (converged at round 7) | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 3,485 | +| Deletions | 1 | +| Changed files | 107 | +| Inline CCRA comments | 24 | +| Merge time | 2026-07-08 18:37 UTC | +| Wall-clock (phase 1 + 2) | ~2h 3min | + +#### Assessment + +This was the most code-intensive task (107 files, 3,485 additions) — the CCA seeded a full H2 database with JPA entities, a Jakarta Data repository, and a JSON loader. The 7 CCRA rounds reflect genuine complexity: the CCRA caught issues across multiple rounds without clear convergence until round 7, suggesting the initial implementation had several layered defects. The large file count (107 files — many likely generated JSON seed data) may have overwhelmed the CCRA's attention, contributing to sustained comment volume. The CCA correctly used Jakarta Data `@Repository` as required by constraints, with CCRA flagging correctness issues in the JPA mappings. + +The aborted phase-1 attempt (13-second session, 94 tokens) was a script restart with no code impact. + +--- + +### 3.3 — Issue [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) / PR [#16](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/16): Core Agent Infrastructure + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1244`, session 19 min. CCA produced PR [#16](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/16) at 18:38 UTC. + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1340`, session 71m 15s. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 2 | +| CCRA rounds | **8 (cap reached)** | +| Local CLI fix commits | 8 | +| Total PR commits | 10 | +| 8-round cap hit? | **Yes** | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 399 | +| Deletions | 0 | +| Changed files | 6 | +| Inline CCRA comments | 46 | +| Merge time | 2026-07-08 20:08 UTC | +| Wall-clock (phase 1 + 2) | ~1h 30min | + +#### Assessment + +The 8-round cap indicates the CCRA and local CLI did not reach a stable state within the allowed iterations. With 46 inline comments across 8 rounds, the average was ~5.75 comments per round — no meaningful convergence trend. This is the second-highest comment density per round after issues [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) and [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20). + +The core agent infrastructure task required implementing the `@CopilotTool` annotation API (a headline SDK feature) alongside CDI producers and state management. The complexity of interleaving Jakarta EE CDI lifecycle with Copilot SDK session management likely generated recurring CCRA concerns across rounds. Possible oscillation: CCRA may have introduced new comments on code touched in earlier rounds (a common sign of the CCRA re-evaluating context). + +The task did merge at round 8, meaning some CCRA comments were likely unaddressed at merge time. + +--- + +### 3.4 — Issue [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) / PR [#17](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/17): WebSocket Push Infrastructure + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1244`, session 18 min. CCA produced PR [#17](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/17) at 20:09 UTC. + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1340`, session 77m 42s. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 2 | +| CCRA rounds | **8 (cap reached)** | +| Local CLI fix commits | 8 | +| Total PR commits | 10 | +| 8-round cap hit? | **Yes** | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 145 | +| Deletions | 37 | +| Changed files | 4 | +| Inline CCRA comments | 32 | +| Merge time | 2026-07-08 21:45 UTC | +| Wall-clock (phase 1 + 2) | ~1h 35min | + +#### Assessment + +Notably, 37 deletions suggest the CCRA directed the local CLI to remove code (more than any other small-file task). Despite only 4 changed files, the CCRA generated 32 comments over 8 rounds — the highest comments-per-file ratio (8.0) of all tasks. WebSocket integration with JSF's `f:websocket` channel involves tight coupling between server-push semantics and CDI scopes, a notoriously finicky area in Jakarta EE 11. The CCRA likely kept catching scope and lifecycle violations that the local CLI fixed incompletely. Cap hit at 8 rounds; some comments likely unresolved at merge. + +--- + +### 3.5 — Issue [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) / PR [#18](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/18): JSF Pipeline View + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1244`, session 12 min. CCA produced PR [#18](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/18) at 21:46 UTC. + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1340`, session 66m 42s. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 4 | +| CCRA rounds | 6 | +| Local CLI fix commits | 6 | +| Total PR commits | 10 | +| 8-round cap hit? | No (converged at round 6) | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 634 | +| Deletions | 29 | +| Changed files | 6 | +| Inline CCRA comments | 54 | +| Merge time | 2026-07-08 23:04 UTC | +| Wall-clock (phase 1 + 2) | ~1h 18min | + +#### Assessment + +The highest absolute comment count (54) of all tasks, yet this task converged without hitting the cap. The 4 initial CCA commits (vs. the typical 2) suggest the CCA iterated internally before marking the PR ready. PrimeFaces 15.0 JSF layout involves considerable boilerplate (XHTML, bean bindings, CSS), giving the CCRA many opportunities to comment. The convergence at round 6 (despite 54 comments) suggests the CCRA's concerns were genuinely resolvable — each round produced meaningful reduction, unlike the cap-hit tasks. + +--- + +### 3.6 — Issue [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) / PR [#21](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/21): Dynamic UI Updates + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1918`, session 31 min. CCA produced PR [#21](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/21) at 23:19 UTC. + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1918`, session 93m 17s. + +> **Note:** This issue replaced the aborted issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8). The original CCA run for issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) (PR [#19](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/19), 7,546 additions across 130 files) was manually aborted after 5 minutes because the scope far exceeded the task specification. Issue [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) was created as a replacement with a tighter prompt. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 7 | +| CCRA rounds | 5 | +| Local CLI fix commits | 5 | +| Total PR commits | 12 | +| 8-round cap hit? | No (converged at round 5) | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 1,149 | +| Deletions | 0 | +| Changed files | 11 | +| Inline CCRA comments | 72 | +| Merge time | 2026-07-09 01:23 UTC | +| Wall-clock (phase 1 + 2) | ~2h 4min | + +#### Assessment + +The highest total inline comment count (72) across all tasks, yet the fewest CCRA rounds of the UI tasks (5). This implies each CCRA round generated many comments but the local CLI addressed them effectively in bulk. The 7 initial CCA commits reflect the complexity of coordinating CSS transitions, WebSocket push events, and PrimeFaces re-render directives. No lines deleted suggests the CCA added net-new code only. Convergence in 5 rounds shows this replacement issue (with a tighter scope than [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8)) was better suited to agentic implementation. + +The manual abort of issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) / PR [#19](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/19) represents the single manual intervention in the entire epic, triggered by CCA scope creep (130 files vs. the expected ~10). + +--- + +### 3.7 — Issue [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) / PR [#22](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/22): Agent Detail View + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1918`, session 10 min. CCA produced PR [#22](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/22) at 01:24 UTC (July 9). + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1918`, session **602m 53s** (10 hours 2 minutes). + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 3 | +| CCRA rounds | 7 | +| Local CLI fix commits | 7 | +| Total PR commits | 10 | +| 8-round cap hit? | No (converged at round 7) | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 500 | +| Deletions | 37 | +| Changed files | 8 | +| Inline CCRA comments | 41 | +| Merge time | 2026-07-09 11:35 UTC | +| Wall-clock (phase 1 + 2) | ~10h 12min | + +#### Assessment + +The 602-minute phase-2 session is the single most extreme outlier in the dataset. The code work itself was modest (500 additions, 8 files, 7 rounds), but the session ran from 01:33 to 11:36 UTC — overnight. The elapsed time reflects **wall-clock wait time for asynchronous CCRA reviews**, not active processing. Each CCRA round took 20–60 minutes on GitHub's side, and with 7 rounds the session naturally spanned the night. Active Local CLI processing time was far shorter. + +The session log shows the CLI correctly handling thread resolution, polling loops, and multiple re-review requests without human intervention. The task eventually converged and merged cleanly. This illustrates a systemic issue with the shepherd script: long-running overnight sessions consume a copilot CLI process for extended periods without benefit. + +--- + +### 3.8 — Issue [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10) / PR [#23](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/23): End-to-End Integration Testing + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1918`, session 17 min. CCA produced PR [#23](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/23) at 11:36 UTC. + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1918`, session 35m 1s. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 2 | +| CCRA rounds | 4 | +| Local CLI fix commits | 4 | +| Total PR commits | 6 | +| 8-round cap hit? | No (converged at round 4) | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 691 | +| Deletions | 14 | +| Changed files | 7 | +| Inline CCRA comments | 12 | +| Merge time | 2026-07-09 12:28 UTC | +| Wall-clock (phase 1 + 2) | ~52min | + +#### Assessment + +The lowest comment count (12) among the complex tasks, second only to the scaffolding task ([#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13)). The CCA's test suite for pipeline validation was well-scoped and the CCRA converged quickly. The 4-round convergence with only 3 comments per round on average suggests the CCRA found genuinely discrete, resolvable issues without oscillation. The `FEATURE-VERIFICATION.md` artifact in the PR title suggests the CCA created documentation alongside unit tests. + +--- + +### 3.9 — Issue [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) / PR [#24](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/24): Demo Polish and README + +**Phase 1:** Shepherd batch `shepherd-tasks-20260708-1918`, session 15 min. CCA produced PR [#24](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/24) at 12:29 UTC. + +**Phase 2:** Shepherd batch `shepherd-tasks-20260708-1918`, session 19m 5s. + +#### Throughput & Convergence + +| Metric | Value | +|--------|-------| +| CCA initial commits | 1 | +| CCRA rounds | 1 | +| Local CLI fix commits | 1 | +| Total PR commits | 2 | +| 8-round cap hit? | No | + +#### PR Stats + +| Metric | Value | +|--------|-------| +| Additions | 307 | +| Deletions | 6 | +| Changed files | 7 | +| Inline CCRA comments | 4 | +| Merge time | 2026-07-09 13:02 UTC | +| Wall-clock (phase 1 + 2) | ~34min | + +#### Assessment + +The fastest-completing complex task (34 minutes total). The polish/README task is inherently documentation-heavy, with the CCRA generating only 4 comments in a single round. This reflects the CCRA's strength on documentation: it caught real issues (likely missing sections or broken links) without over-commenting. The 1-round convergence and minimal deletions indicate the CCA produced near-final quality output for documentation tasks. + +--- + +## Section 4: Aggregate Statistics + +### 4.1 Summary Table + +| Issue | PR | CCA commits | CCRA rounds | CLI commits | Comments | +Lines | −Lines | Cap? | Merged | +|-------|-----|-------------|-------------|-------------|----------|--------|--------|------|--------| +| [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) | [#14](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/14) | 2 | 1 | 1 | 2 | 143 | 0 | — | 2026-07-08 16:25 | +| [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) | [#15](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/15) | 2 | 7 | 7 | 24 | 3,485 | 1 | — | 2026-07-08 18:37 | +| [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) | [#16](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/16) | 2 | **8** | 8 | 46 | 399 | 0 | ✓ | 2026-07-08 20:08 | +| [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) | [#17](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/17) | 2 | **8** | 8 | 32 | 145 | 37 | ✓ | 2026-07-08 21:45 | +| [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) | [#18](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/18) | 4 | 6 | 6 | 54 | 634 | 29 | — | 2026-07-08 23:04 | +| [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) | [#21](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/21) | 7 | 5 | 5 | 72 | 1,149 | 0 | — | 2026-07-09 01:23 | +| [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) | [#22](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/22) | 3 | 7 | 7 | 41 | 500 | 37 | — | 2026-07-09 11:35 | +| [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10) | [#23](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/23) | 2 | 4 | 4 | 12 | 691 | 14 | — | 2026-07-09 12:28 | +| [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) | [#24](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/24) | 1 | 1 | 1 | 4 | 307 | 6 | — | 2026-07-09 13:02 | +| **Total** | | **25** | **47** | **47** | **287** | **7,453** | **124** | **2/9** | | + +### 4.2 Aggregate Metrics + +| Metric | Value | +|--------|-------| +| Total commits by CCA (initial) | 25 | +| Total fix commits by Local CLI | 47 | +| Total merged commits | 72 | +| Total CCRA review rounds | 47 | +| Average CCRA rounds per task | 5.2 | +| Median CCRA rounds per task | 6 | +| Tasks hitting 8-round cap | 2 (issues [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5), [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6)) | +| Total CCRA comments generated | 287 | +| Average comments per task | 31.9 | +| Average comments per CCRA round | 6.1 | +| Tasks requiring manual intervention | 1 (issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) abort) | +| Total lines added | 7,453 | +| Total lines deleted | 124 | +| Net lines added | 7,329 | + +### 4.3 Convergence Analysis + +Convergence (successive CCRA rounds producing fewer comments) was observed in most non-cap tasks. Tasks hitting the cap ([#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5), [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6)) showed no clear downward trend — the comment count remained roughly flat across rounds, suggesting oscillation (CCRA re-flagging code touched in earlier rounds). + +| Convergence pattern | Tasks | +|--------------------|-------| +| Fast convergence (1–2 rounds) | [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13), [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) | +| Gradual convergence (3–5 rounds) | [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10), [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) | +| Slow convergence (6–7 rounds) | [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4), [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7), [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) | +| No convergence (cap hit at 8) | [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5), [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) | + +--- + +## Section 5: AI Credits + +### 5.1 Local Copilot CLI Token Usage + +Token counts were parsed from `data.outputTokens` fields in all JSONL event logs. Input tokens were not recorded in the event stream (field absent = 0 in all files). + +| File | Phase | Issue | Output Tokens | +|------|-------|-------|---------------| +| phase1-task-20260708-1234-4.json | 1 | [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) (aborted restart) | 94 | +| phase1-task-20260708-1244-4.json | 1 | [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) | 19,019 | +| phase1-task-20260708-1438-5.json | 1 | [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) | 10,886 | +| phase1-task-20260708-1609-6.json | 1 | [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) | 10,280 | +| phase1-task-20260708-1745-7.json | 1 | [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) | 5,099 | +| phase1-task-20260708-1904-8.json | 1 | [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) (aborted) | 3,119 | +| phase1-task-20260708-1918-20.json | 1 | [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) | 11,213 | +| phase1-task-20260708-2123-9.json | 1 | [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) | 5,777 | +| phase1-task-20260709-0736-10.json | 1 | [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10) | 22,667 | +| phase1-task-20260709-0828-11.json | 1 | [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) | 9,561 | +| **Phase 1 subtotal** | | | **97,715** | +| phase2-task-20260708-1203-13.json | 2 | [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) | 4,644 | +| phase2-task-20260708-1332-4.json | 2 | [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) (aborted) | 0 | +| phase2-task-20260708-1340-4.json | 2 | [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) | 42,390 | +| phase2-task-20260708-1457-5.json | 2 | [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) | 54,463 | +| phase2-task-20260708-1628-6.json | 2 | [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) | 49,706 | +| phase2-task-20260708-1757-7.json | 2 | [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) | 56,398 | +| phase2-task-20260708-1950-20.json | 2 | [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) | 65,900 | +| phase2-task-20260708-2133-9.json | 2 | [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) | 69,517 | +| phase2-task-20260709-0753-10.json | 2 | [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10) | 18,384 | +| phase2-task-20260709-0843-11.json | 2 | [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) | 8,171 | +| **Phase 2 subtotal** | | | **369,573** | +| **Grand total** | | | **467,288** | + +> **Note:** Input tokens were not present in the event log (all values were 0). The `outputTokens` field on `assistant.message` events reflects only the local CLI's generation; CCA and CCRA tokens are tracked separately by GitHub billing. + +### 5.2 CCA and CCRA Credits + +CCA and CCRA credits are billed by GitHub based on coding agent runs and review rounds respectively. These cannot be directly queried from the repository API. Estimates: + +- **CCA credits:** 9 implementations × ~1 coding agent run each (some tasks like [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) may have required multiple CCA runs) = **~9–11 coding agent runs** +- **CCRA credits:** 47 review rounds = **47 CCRA invocations** +- **Local CLI (copilot --yolo):** 20 shepherd sessions (10 phase-1 + 10 phase-2) consuming 467,288 output tokens + +--- + +## Section 6: Wall-Clock Timeline + +### 6.1 Overall + +- **Start:** 2026-07-08 16:03:31 UTC (first event in `shepherd-tasks-20260708-1203`) +- **End:** 2026-07-09 13:02:40 UTC (last event in `shepherd-tasks-20260708-1918`) +- **Total elapsed:** 20 hours 59 minutes + +### 6.2 Batch Timeline + +| Batch directory | Start (UTC) | End (UTC) | Issues processed | Notes | +|----------------|-------------|-----------|-----------------|-------| +| `shepherd-tasks-20260708-1203` | 16:03 | 16:26 | [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) (phase 2) | First batch; issue [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) already ready | +| `shepherd-tasks-20260708-1233` | — | — | (empty) | Script directory created but no tasks run | +| `shepherd-tasks-20260708-1244` | 16:34 | 23:09 | [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) (ph1), [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) (ph1), [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) (ph1), [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) (ph1), [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) (ph1, aborted) | Aborted issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) ended this batch | +| `shepherd-tasks-20260708-1340` | 17:40 | 23:04 | [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) (ph2), [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) (ph2), [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) (ph2), [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) (ph2) | Longest single batch; 4 sequential phase-2 runs | +| `shepherd-tasks-20260708-1918` | 23:04 | 13:02+1d | [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) (ph1+2), [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) (ph1+2), [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10) (ph1+2), [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) (ph1+2) | Ran overnight; issue [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) phase-2 ran 10+ hours | + +### 6.3 Per-Issue Timeline + +``` +2026-07-08 UTC +00:00 12:00 2026-07-09 UTC +| | | +·-- [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) CCA (overnight) --------merge(16:25) + ·-- [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) ph1 ·-- [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) ph2 --------merge(18:37) + ·-- [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) ph1 ·-- [#5](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/5) ph2 ----merge(20:08) + ·-- [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) ph1 ·-- [#6](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/6) ph2 --merge(21:45) + ·-- [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) ph1 ·-- [#7](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/7) ph2 --merge(23:04) + ·-- [ABORT [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8)] + ·-- [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) ph1+ph2 --------merge(01:23+1d) + ·-- [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) ph1+ph2 (overnight!) ------merge(11:35+1d) + ·-- [#10](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/10) ph1+ph2 --merge(12:28+1d) + ·-- [#11](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/11) ph1+ph2 --merge(13:02+1d) +``` + +### 6.4 Notable Events + +| Time (UTC) | Event | +|-----------|-------| +| 2026-07-08 00:25 | PR [#14](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/14) created by CCA (pre-batch) | +| 2026-07-08 16:03 | First shepherd batch starts (issue [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) phase 2) | +| 2026-07-08 16:25 | PR [#14](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/14) merged — issue [#13](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/13) complete | +| 2026-07-08 16:34 | Phase-1 for issue [#4](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/4) aborts after 13 seconds (script restart) | +| 2026-07-08 23:04 | Phase-1 for issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) manually aborted at 5 min; PR [#19](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/19) closed | +| 2026-07-08 23:05 | PR [#19](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/19) (7,546 additions, 130 files) created then immediately closed | +| 2026-07-08 23:09 | Issue [#8](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/8) aborted; issue [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) created as replacement | +| 2026-07-08 23:19 | New CCA started on replacement issue [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) | +| 2026-07-09 01:23 | PR [#21](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/21) merged — issue [#20](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/20) complete | +| 2026-07-09 01:33 | Phase-2 for issue [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) starts (10-hour overnight session begins) | +| 2026-07-09 11:35 | PR [#22](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/22) merged — issue [#9](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/9) finally complete after 10h of CCRA wait | +| 2026-07-09 13:02 | PR [#24](https://github.com/edburns/Build26-BRK206-your-agent-anywhere-multiclient-multidevice-with-github-copilot-sdk/issues/24) merged — epic complete | + +--- + +## Section 7: Human-Directed Changes After the Agentic Work Completed + +This section details the human-directed changes to the app to make it acceptable to the customer after the agentic work completed. + +After the nine-issue agentic pipeline completed and all PRs were merged, the human developer tested the app end-to-end. Three categories of UI deficiencies were identified that required human-directed changes (commits `f6d9ddb`–`c6168d0`, tags `20260709-1644-02-column-layout-success` through `20260709-1800-04-dashboard-success`). These changes spanned 5 files, adding 500 lines and removing 96 lines. + +### 7.1 Pipeline Layout Restructure (commit `f6d9ddb`) + +**Problem:** The agent built the pipeline grid as a horizontal auto-fill CSS grid (`repeat(auto-fill, minmax(180px, 1fr))`), rendering all 7 phases as equal-width columns in a single row. The Blazor reference app uses a vertical two-column layout: lifecycle states (Queued → Validating → Searching → Writing Report) in the left column with downward arrows, and end states (Rejected, No Matches, Done) in the right column with rightward arrows connecting them to the corresponding lifecycle phase. + +**Changes made:** +- **index.xhtml:** Replaced the single `ui:repeat` over all phases with an explicit two-column structure — 4 `pipeline-row` divs, each with a `lifecycle-col`, `arrow-col`, and `endstate-col`. Added column headers ("Lifecycle state" / "End state"). Created a reusable `agent-card.xhtml` include fragment to avoid repeating card markup across 7 phase slots. +- **pipeline.css:** Replaced `.pipeline-grid` with `.pipeline-layout` using CSS grid rows (`1fr 60px 1fr`). Added vertical arrow styling (`.arrow-line`) and horizontal arrow styling (`.arrow-right`) with triangle pseudo-elements. +- **PipelineView.java:** Added `String` overloads for `getAgentsAtPhase()` and `getPhaseHeaderClass()` since the new XHTML passes phase names as string literals rather than enum references. + +### 7.2 Canned Query "+" Button (commit `d7e2b56`) + +**Problem:** The agent built a free-text input field with a "Submit" button for entering enquiries. The Blazor reference app uses a "+" button in the top-right corner that toggles a popup list of 10 pre-defined sample queries. The free-text input was not part of the Blazor design and would require the presenter to type queries during the demo. + +**Changes made:** +- **PipelineView.java:** Added a `SAMPLE_ENQUIRIES` array (10 canned queries matching the Blazor demo) and a `submitSampleEnquiry(String)` action method. +- **index.xhtml:** Removed the `h:inputText` and `p:commandButton`. Added a circular "+" button with an absolutely-positioned popup that uses `c:forEach` over the sample enquiries list. (Note: `ui:repeat` was tried first but produced no output for the `List`; `c:forEach` was required.) Added inline `