Skip to content

Make lede-voice audit produce a visible classification table #574

Description

@williamthorsen

Problem

The lede-voice audit step in _partials/voice-checklist.md is self-attested: it instructs the agent to "classify each sentence" and "verify each identifier" without requiring any visible artifact. Agents — including the one that authored the doctrine — routinely declare "audited" without producing evidence of the classification, and ship content that fails the rules.

Concrete evidence from a single session producing #555 / PR #573:

  • Initial PR body Para 1 enumerated three internal authoring surfaces and named "via a shared partial" — the exact pattern banned by the format/glyph adoption Bad/Good example added in the same change.
  • Initial PR body Para 2 enumerated five internal source-tree paths under a self-claimed "migration exception" that does not actually apply to internal source paths.
  • Initial commit message included a "Note" describing back-and-forth review process — the exact pattern banned by the title-content rule against ephemeral references.
  • Initial review-response disposition R1 accepted a recommendation that contradicted the destination doctrine (imperative voice into a deliberately-declarative SSOT) on the reviewer's "plausibility" argument alone.

In every case the agent had completed its self-attested audit before the user surfaced the violation. Self-attestation does not work when the discipline test is non-trivial.

Context

The current audit step (in _partials/voice-checklist.md):

Audit before save:

  1. For each sentence, classify it: Outcome / Invariant / Migration. Rewrite anything that fits none.
  2. For each identifier, confirm it's in the allowed list. Drop or rephrase anything that isn't.

Nothing in this requires the classification to be visible. The agent can declare the audit complete by referencing the rules without applying them per-sentence.

Proposed solution

Replace the audit step with a forcing-output version: require the agent to produce a visible classification table as part of the same response that contains the drafted content. The table must appear before the content is saved or committed.

Format:

**Audit table** (produce this before saving):

| #   | Sentence                       | Rule 1 category | Rule 2 verdict                      | Action  |
| --- | ------------------------------ | --------------- | ----------------------------------- | ------- |
| 1   | "First sentence text..."       | Outcome         | All identifiers allowed             | OK      |
| 2   | "Second sentence text..."      | Mechanism       | n/a                                 | REWRITE |
| 3   | "Rewritten second sentence..." | Outcome         | All identifiers allowed             | OK      |
| 4   | "Third sentence text..."       | Migration       | Banned: `internal-foo.md` (line N)  | REWRITE |
| 5   | "Rewritten third sentence..."  | Migration       | All identifiers allowed             | OK      |
  • One row per sentence in the drafted passage.
  • Rule 1 column: one of Outcome, Invariant, Migration, or Mechanism. Mechanism is the failure label; the other three are the doctrine's three permissible categories.
  • Rule 2 column: either "All identifiers allowed" or a comma-separated list of banned identifiers found.
  • Action column: OK, REWRITE (Rule 1 violation), or DROP (sentence has no salvageable content).
  • Any row with Action ≠ OK requires rewriting and adding a new row for the rewritten version. The audit completes when every row has Action = OK.

Considerations

  • The table itself becomes a review artifact. A human reviewer or downstream subagent can check "did the agent classify correctly" much faster than re-reading the body and re-applying the rules. Invisible failure becomes visible failure.
  • An agent can still write a sloppy table (e.g., mis-classify Mechanism as Outcome). The forcing function is "the table exists in the response", not "the table is correct". This is a meaningful improvement, not a complete fix — programmatic enforcement (the deferred linter from Restructure voice and format rules to enforce inline at point-of-use #555) is the durable answer.
  • Cost: the audit table adds tokens to every voice-authoring response. Acceptable given the alternative — mis-shipped release notes that need manual rewrite later — costs more in aggregate.
  • The table fits naturally into a subagent dispatch context (a deferred discussion): a subagent's only output for voice authoring would be the audit table plus the drafted passage, so producing it is the natural shape of the work.
  • Out of scope: changing the rules themselves (Rule 1, Rule 2, or the migration exception). This change targets only the audit-step enforcement mechanism.

Acceptance criteria

  • _partials/voice-checklist.md audit step is replaced with the forcing-output version requiring a visible classification table.
  • The format of the table is specified in the partial: columns (sentence, Rule 1 category, Rule 2 verdict, action), allowed values per column, and the rule that any Action ≠ OK row requires a rewrite plus re-audit.
  • The partial includes a worked example showing a sentence-by-sentence audit table for a small drafted passage with at least one Rule 1 violation and at least one Rule 2 violation, plus the rewritten version's audit rows.
  • No other authoring guidance in lede-voice.md or the consuming skills is reworded to depend on the new format — the change is localized to the audit-step partial.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureAdded or improved external functionalityscope:agents

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions