Skip to content

update-project-guidance cannot refresh an existing AGENTS.md without redrafting it #1280

Description

@williamthorsen

Problem

update-project-guidance generates AGENTS.md from a skeleton. Phase 1 notes what is already there; nothing afterward preserves it, verifies it, or checks the result. Run against a mature guidance file, it drafts a replacement rather than an edit, and the densest section is usually the one carrying the most value.

Nothing in the skill catches a claim that has drifted from the codebase either, because it explores to draft rather than to check what the file already asserts. Claim drift is the dominant failure mode of a mature ambient file, and the skill is blind to it.

Context

  • Surfaced while refreshing the repo-root AGENTS.md in Improve the quality of the repo-root AGENTS.md eslint-config#159, where the file's value was concentrated in a Gotchas section the skill would have summarized away. The pass was done by hand, using the skill's own Phase 2 classification table and quality checklist as the rubric. That rubric is the part that transfers; the generative process around it is what does not.
  • Every defect that pass found was a drifted claim: a .config/ list gone stale, a fixture count off by two, a tool named one hop away from what actually runs. All three are checkable tokens rather than narrative prose.
  • The guidance checklist codeassembly publishes measures presence, wiring, size, and commit-count freshness. None of those detects a false claim: a drifted file passes every check it can answer, and its only signal is a freshness count that any edit resets.
  • Two constraints follow from the root AGENTS.md being a harness-neutral destination, and appear nowhere in the skill: it can carry no harness-scoped path, since one body of text serves every harness; and sync strips a <!-- rulebook:… --> region it finds there, so a region introduced by hand disappears on the next run with no warning to the author. The sweep matches complete marker pairs only, so a lone unpaired opener survives indefinitely.
  • Two constraints on any solution, established during design. Decomposing a mature file into atomic assertions does not scale: this repo's own 35-line AGENTS.md yields 70-90 of them, and a file at the 200-line budget yields several hundred. And a window anchored on the last commit touching AGENTS.md collapses after any drive-by edit; in this repo it currently stands at two commits, behind a refactor: PR that appended a single clause to a bullet.

Proposed solution

Give the skill a refresh path distinct from its authoring path, selected on whether substantive guidance content already exists, at the repo root or at a legacy path awaiting migration. The refresh path reconciles against that content: it emits edits rather than rendering a replacement, and it audits the content's claims against the codebase before proposing any. The two paths diverge only in what they produce, sharing discovery and converging on the existing write, include-repair, legacy, and general-guidance steps. Both paths gain the harness-neutrality constraints, and the published guidance checklist gains checks for them so they hold beyond the skill's own run.

Acceptance criteria

Must have

  • The skill branches on whether substantive guidance content already exists, at the repo root or at a legacy path, and follows a different path for each case.
  • Both paths share discovery and converge on the existing file-writing, include-repair, legacy-file, and general-guidance steps, so neither closes by failing a check it has removed its own ability to fix.
  • The refresh path's only outputs against the existing content are repairs, removals, relocations, and additions; a repair quotes the text it replaces verbatim, and the content is rendered in no other form.
  • The refresh path reports a per-claim verdict with the evidence behind it, cannot report a claim as holding without evidence, and states what it left unaudited.
  • The refresh path's search for content the file is missing is bounded to a window that no single drive-by edit to AGENTS.md can collapse.
  • Both paths forbid a harness-scoped path and any <!-- rulebook:… --> marker in the repo-root AGENTS.md, and the refresh path repairs an existing violation rather than carrying it through.
  • The published guidance checklist detects a harness-scoped path and a <!-- rulebook: marker in the repo-root AGENTS.md, including an unpaired marker.
  • The skill closes by running the guidance checklist, skips that step when the runner is unavailable, and states that a freshness failure on an uncommitted refresh is expected rather than actionable.
  • The content test suite passes, and new behavior in this change is covered by tests.

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions