diff --git a/docs/design/development-loop.md b/docs/design/development-loop.md index 38315b2..3133858 100644 --- a/docs/design/development-loop.md +++ b/docs/design/development-loop.md @@ -5,7 +5,7 @@ status: pre-release audience: [human, ai] tags: [caliber-webhooks, github, ci, automation, governance, autonomy, process] related: [./roadmap.md, ./decisions.md] -updated: 2026-06-23 +updated: 2026-06-24 --- # The Self-Building Development Loop @@ -84,6 +84,8 @@ The in-CI agent (`claude-code-action`, headless on Actions runners) is **not** t 4. **Rule system + support automation** — branch protection/rulesets, CODEOWNERS, path-classifier, native auto-merge, Dependabot (first dogfood), CodeQL, dependency-review. ✅ *(all live — files in `ff51da1`; the four public-gated pieces activated on the flip below)* 5. **Agent-in-the-loop** — a trusted human mentions `@claude` on an issue; the agent implements on a `claude/issue--*` branch, a post-step opens the PR, and the rest of the loop gates it unchanged. ✅ *(done — [`claude.yml`](../../.github/workflows/claude.yml): trusted-actor guard, `risk:*` → model routing, provision-either auth, deterministic auto-PR step. Setup: install the [Claude GitHub App](https://github.com/apps/claude); add an auth secret — `CLAUDE_CODE_OAUTH_TOKEN` (subscription) or `ANTHROPIC_API_KEY` (metered); and add **`AGENT_PAT`** (a fine-grained PAT, this repo, `contents:RW` + `pull-requests:RW`) so the agent's PR is opened by a non-`GITHUB_TOKEN` identity — see the note below.)* +> **Tooling enrichment (dotnet-claude-kit / Roslyn MCP) — verified follow-up, issue #29.** The in-CI agent does **not** inherit the repo's `.claude/settings.json` plugin config: `claude-code-action` installs plugins only from its `plugins` / `plugin_marketplaces` **inputs**, and writes home settings from its `settings` input — it never reads the committed repo file (confirmed by code inspection *and* by observing a live `@claude` run whose `~/.claude/settings.json` held only the action's no-input default). To give the CI agent the kit's Roslyn MCP tools + agents/skills, `claude.yml` needs those inputs set plus a `setup-dotnet` step (the Roslyn server is a .NET process). The Roslyn server indexes the whole solution on first use, a **per-run cold-start cost**, so the recommendation is to **gate the enrichment behind a label** (only `risk:core`/`critical` or an explicit opt-in) rather than pay it on every trivial run — consistent with *cost control is gate control* above. Full finding + proposed diff: [the adoption doc](./dotnet-claude-kit-adoption.md#roslyn-mcp-and-the-whole-kit-in-ci--verified-finding). This is a `risk:core` workflow edit the CI App cannot self-apply (no `workflows` permission); a maintainer applies it. + > **Why a PAT is needed for hands-off PRs.** In tag mode `claude-code-action` pushes a branch but does not open the PR (it leaves a "Create PR" link). We open it in a post-step — but a PR opened with the workflow's default `GITHUB_TOKEN` triggers **no** workflows (GitHub suppresses `GITHUB_TOKEN`-initiated runs to prevent recursion), so neither `ci.yml` nor `classify.yml` would run and the PR could never gate or auto-merge. Opening it with a **PAT** (`AGENT_PAT`) gives it a normal user identity, so CI + classify fire and the agent's PR flows through the identical `classify → ruleset/CODEOWNERS → auto-merge` path as a human's. Without the PAT the post-step no-ops and a human clicks the agent's "Create PR" link (the loop still works, just one click less hands-off). ## The public flip — done diff --git a/docs/design/dotnet-claude-kit-adoption.md b/docs/design/dotnet-claude-kit-adoption.md index ccbf067..c189f2b 100644 --- a/docs/design/dotnet-claude-kit-adoption.md +++ b/docs/design/dotnet-claude-kit-adoption.md @@ -18,7 +18,7 @@ updated: 2026-06-24 1. **Travels with the repo** — any clone or contributor gets the kit, not just the maintainer's machine. 2. **Templates the portfolio** — the same `.claude/settings.json` is the copy-paste starting point for the other Caliber projects. -3. **Reaches the CI agent** — the `@claude` agent-in-the-loop ([`claude.yml`](../../.github/workflows/claude.yml)) runs in the checked-out repo, so it inherits the kit's agents, commands, and skills for issue-driven work. *(The Roslyn MCP server additionally needs a .NET runtime in that job to expose its 15 tools — a deliberate follow-up, see [Open items](#open-items).)* +3. **Does *not* reach the CI agent on its own.** `claude-code-action` does **not** read this file's plugin config, so the `@claude` agent-in-the-loop ([`claude.yml`](../../.github/workflows/claude.yml)) sees none of the kit unless it is wired in **explicitly** via the action's inputs (plus a .NET runtime for the Roslyn MCP server). This was assumed to be free; it is not. See [Roslyn MCP in CI — verified finding](#roslyn-mcp-and-the-whole-kit-in-ci--verified-finding). There is nothing to "turn on" per session: the kit is also enabled in the maintainer's user settings, so its hooks and tools are already live locally. This doc exists so the capability is **used deliberately** and its opinions are **reconciled**, not absorbed by accident. @@ -62,9 +62,84 @@ The kit is opinionated. "Adopt as much as fits" means adopting what aligns and * We also keep our own **memory / OKF docs** as the system of record rather than the kit's `instinct-system` / `convention-learner` / `split-memory` skills — those would duplicate (and could drift from) the handoff docs and agent memory. -## Open items +## Roslyn MCP (and the whole kit) in CI — verified finding -- **Roslyn MCP in CI.** Skills/agents/commands reach the `@claude` agent for free via repo settings; the `cwm-roslyn-navigator` MCP server needs a `dotnet` setup step in `claude.yml` to expose its 15 tools there. Wire it once confirmed the action picks up the repo's plugin settings on a live `@claude` run. +> **Investigated 2026-06-24 (issue #29), from inside a live `@claude` CI run.** The earlier assumption that "skills/agents/commands reach the `@claude` agent for free via repo settings" is **wrong**. They do not. Here is what actually happens, and the wiring that fixes it. + +### AC1 — does the in-CI agent inherit the repo-scoped `.claude/settings.json` plugins? **No.** + +Two independent lines of evidence, both conclusive: + +1. **Code inspection of `anthropics/claude-code-action@v1`.** The action never reads the repo's `.claude/settings.json` plugin config: + - `base-action/src/setup-claude-code-settings.ts` writes **`~/.claude/settings.json`** (home scope) by merging only the action's `settings` **input** (`INPUT_SETTINGS`) over what's already there, then force-sets `enableAllProjectMcpServers: true`. Our `claude.yml` passes **no** `settings` input, so nothing from the repo's `extraKnownMarketplaces` / `enabledPlugins` is ever loaded here. + - `base-action/src/install-plugins.ts` is the **only** thing that runs `claude plugin marketplace add` / `claude plugin install`, and it is driven **exclusively** by the action's `plugins` + `plugin_marketplaces` **inputs** (`INPUT_PLUGINS` / `INPUT_PLUGIN_MARKETPLACES`). With both unset, it logs "No marketplaces specified" / "No plugins specified" and installs nothing. +2. **Empirical, from this very `@claude` run.** The job's `~/.claude/settings.json` contained exactly `{"enableAllProjectMcpServers": true}` — the no-input default the action writes — with **none** of the repo's marketplace/plugin keys merged in. Correspondingly, **no** kit capability was live in the session: none of the kit's agents (`security-auditor`, `ef-core-specialist`, …), none of its 47 skills (`idesign`, `ef-core`, …), and **none** of the 15 `cwm-roslyn-navigator` MCP tools were present. + +**Why.** The repo's `.claude/settings.json` is still on disk in the checkout, and `enabledPlugins` only *enables* a plugin from an **already-installed** marketplace — it does not itself clone/fetch one. In a fresh CI checkout the marketplace was never added, so there is nothing to enable. `enableAllProjectMcpServers` only auto-approves project `.mcp.json` servers, **not** plugin-provided MCP servers. **Conclusion: the kit must be wired explicitly via the action's inputs.** (It would help every contributor too, but in CI it is mandatory.) + +### Recommended wiring (proposed `claude.yml` change) + +This is `.github/workflows/**` → **`risk:core`**, CODEOWNERS-reviewed, never auto-merges. The CI agent (GitHub App token) **cannot** push workflow edits — the `workflows` permission is withheld — so this diff is recorded here for a maintainer to apply by hand. + +1. **A .NET setup step**, pinned to `global.json` (the Roslyn MCP server is a .NET process and needs a runtime), added after `Checkout` and before `Run Claude`: + + ```yaml + # The Roslyn MCP server (cwm-roslyn-navigator, from dotnet-claude-kit) is a + # .NET process; it needs a runtime to start. Pin to global.json so CI uses the + # same SDK the library builds against. + - name: Setup .NET (for the Roslyn MCP server) + uses: actions/setup-dotnet@v5 + with: + global-json-file: global.json + ``` + +2. **Explicit plugin install** on the `Run Claude` step (the fix for AC1) — the action installs only what these inputs name: + + ```yaml + - name: Run Claude + uses: anthropics/claude-code-action@v1 + with: + claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} + anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} + # The in-CI agent does NOT inherit the repo's .claude/settings.json plugin + # config — the action installs ONLY plugins named here. Wire the kit + # explicitly so its Roslyn MCP tools + agents/skills/commands are live in CI. + plugin_marketplaces: | + https://github.com/codewithmukesh/dotnet-claude-kit.git + plugins: | + dotnet-claude-kit@dotnet-claude-kit + claude_args: | + --model ${{ steps.route.outputs.model }} + --max-turns 30 + ``` + + The trusted-actor guard, `risk:* → model` routing, `--max-turns 30` cap, and the `AGENT_PAT` auto-PR step are **unchanged** — only the `Setup .NET` step and the two plugin inputs are added. + +3. **Provisioning `cwm-roslyn-navigator` itself — confirm before applying.** The plugin declares its MCP server as `cwm-roslyn-navigator --solution ${workspaceFolder}`, i.e. it expects that command on `PATH`. Whether the plugin **bundles** the binary or expects a host-installed **.NET global tool** could not be confirmed from inside the CI sandbox (outbound network to the plugin repo is blocked there). Before this lands, a maintainer should check the plugin's `.mcp.json` / install docs and, if it is a global tool, add a restore step, e.g.: + + ```yaml + - name: Install the Roslyn navigator tool # only if the plugin does NOT bundle it + run: dotnet tool install --global cwm-roslyn-navigator # confirm the exact package id + ``` + + Acceptance ("confirm `get_project_graph` is callable in a CI run") can only be closed by a maintainer-applied workflow change plus one observed run, since the agent cannot self-modify `claude.yml`. + +### Cold-start cost — and the recommended gate + +Turning this on adds, to **every** `@claude` run, the sum of: the plugin marketplace clone + plugin install; `setup-dotnet` (cached, ~5–15 s warm); any `dotnet tool` restore; and — the real cost — the Roslyn MCP **indexing the whole solution** (a full MSBuild/Roslyn workspace load) on first tool call. That last item grows with solution size and is paid *per run*, even for a one-line docs PR that never needs navigation. This cuts against the loop's **"cost control is gate control"** principle. + +**Recommendation: gate the kit/Roslyn wiring behind a label** (e.g. only enrich when the work is `risk:core` / `risk:critical`, or behind an explicit `agent:deep` label), rather than paying the index cost on trivial/low work. Mechanically, the existing `Route risk band → model` step already inspects the labels — have it also emit an `enable_roslyn` output and pass the plugin inputs conditionally: + +```yaml + plugin_marketplaces: ${{ steps.route.outputs.enable_roslyn == 'true' && 'https://github.com/codewithmukesh/dotnet-claude-kit.git' || '' }} + plugins: ${{ steps.route.outputs.enable_roslyn == 'true' && 'dotnet-claude-kit@dotnet-claude-kit' || '' }} +``` + +(and guard the `Setup .NET` step with the same `if:`). Today the repo is an M0 shell, so the index cost is small — but the gate keeps it from scaling into every trivial run as the solution grows. + +### Status / deferral + +AC1 is **answered** (not inherited; wire explicitly). The remaining acceptance criteria — the `setup-dotnet` step, the plugin inputs, and the live `get_project_graph` confirmation — require a **maintainer-applied** `claude.yml` change (the App cannot push to `.github/workflows/**`). A **label-gated** rollout is the recommended shape so the per-run Roslyn cold-start is only paid where navigation earns it; an always-on rollout is also viable while the solution is small. Either is a deliberate maintainer call at review time. ## See also - [The self-building development loop](./development-loop.md) — how a change merges.