Skip to content

fix(cli): render StoryboardStepResult.hints[] in all reporting surfaces#894

Closed
bokelley wants to merge 3 commits into
mainfrom
claude/issue-879-runner-hints-cli-rendering
Closed

fix(cli): render StoryboardStepResult.hints[] in all reporting surfaces#894
bokelley wants to merge 3 commits into
mainfrom
claude/issue-879-runner-hints-cli-rendering

Conversation

@bokelley

@bokelley bokelley commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Wires StoryboardStepResult.hints[] into every human-readable and CI reporting surface that was silently dropping it. The runner already populates hints with context_value_rejected diagnostics (landed in #875/#870); this PR makes them visible.

Also extracts formatStoryboardResultsAsJUnit from bin/adcp.js into src/lib/testing/storyboard/junit.ts (marked @internal) so the formatter can be unit-tested without bootstrapping the full CLI dependency tree.

What changed

bin/adcp.js

  • adcp storyboard run (single + multi-instance): prints Hint: <message> below the Error: line for each failing step with hints
  • adcp storyboard step: same, matching that command's no-indent error style
  • Adds comment at each hint block documenting the runner invariant (hints only emitted on failed steps)
  • Removes the inline formatStoryboardResultsAsJUnit function (now imported from dist/lib)

src/lib/testing/storyboard/junit.ts (new, @internal)

  • Hints included in <failure> body between step.error and failing validations
  • message= attribute falls back to step.hints?.[0]?.message when step.error is absent — so CI systems that only read the attribute still surface the diagnostic
  • Named export makes it reachable from tests; @internal JSDoc and absence from top-level barrel signal it is not part of the public @adcp/client API

Test plan

  • node --test test/lib/storyboard-junit-hints.test.js — 5/5 pass (new snapshot tests)
  • node --test test/lib/storyboard-rejection-hints.test.js — 21/21 pass (existing)
  • node --test test/lib/storyboard-runner-contract.test.js — 29/30 pass (1 pre-existing skip unchanged)
  • npm run format:check — clean
  • CI green

Note for reviewer: comply path

Planning experts flagged whether to also thread hints through ComplianceFailure so they appear in adcp comply human-readable output. The dx-expert called this a follow-up concern; the protocol-expert called it out-of-scope (the comply JSON path already serializes hints verbatim). This PR takes the narrower scope per the issue's stated acceptance criteria.

Nits noted (not fixed)

  • No test for XML-special characters in hint messages (the xmlEscape call is correct; a test would pin it)
  • message= attribute omits the Hint: prefix intentionally for brevity (pinned by test on line 108 of storyboard-junit-hints.test.js)

Closes #879

Pre-PR review:

  • code-reviewer: approved — no blockers; documented runner invariant in CLI hint blocks per review
  • dx-expert: approved after fix — resolved one blocker (@internal added to exported symbol to clarify public/internal boundary); 2 nits noted above

Session: https://claude.ai/code/session_01CokHw4pHKBK2v1aYVE9LfL

The runner emits context_value_rejected hints when a seller's error
traces back to a $context.* substitution, but all reporting surfaces
(CLI step printers, JUnit XML, storyboard step command) silently dropped
them. This patch wires up the three human-readable printers and the JUnit
formatter so the diagnostic is visible wherever a developer looks.

The JUnit formatter is extracted from bin/adcp.js into
src/lib/testing/storyboard/junit.ts so it can be unit-tested without
bootstrapping the full CLI dependency tree.

Closes #879

https://claude.ai/code/session_01CokHw4pHKBK2v1aYVE9LfL
@github-actions

Copy link
Copy Markdown
Contributor

IPR Policy Agreement Required

Thank you for your contribution! Before we can accept your pull request, you must agree to our Intellectual Property Rights Policy.

By making a Contribution, you agree that:

  • You grant the Foundation a perpetual, irrevocable, worldwide, non-exclusive, royalty-free copyright license to your Contribution
  • You grant a patent license under any Necessary Claims
  • You represent that you own or have sufficient rights to grant these licenses

To agree, please comment below with the exact phrase:

I have read the IPR Policy

You can read the full IPR Policy here.


I have read the IPR Policy


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

- Add @internal to formatStoryboardResultsAsJUnit JSDoc: resolves the
  public/internal signal mismatch flagged by dx-expert (the function is
  CLI tooling, not a public @adcp/client API)
- Add comment to CLI hint blocks noting the runner only emits hints on
  failed steps, documenting the invariant raised by code-reviewer
- Update changeset to mention the formatter extraction and new tests

https://claude.ai/code/session_01CokHw4pHKBK2v1aYVE9LfL
Auto-generated by sync-version script during build; version.ts was
behind package.json on this branch.

https://claude.ai/code/session_01CokHw4pHKBK2v1aYVE9LfL
@bokelley

Copy link
Copy Markdown
Contributor Author

Closing as obsolete — the core hint-rendering work in this PR landed via #891 before this branch was rebased. The three remaining deltas (extract formatStoryboardResultsAsJUnit into a library module, wire hints into adcp storyboard step, and message= attribute fallback to hints?.[0]?.message) are being cherry-picked into a fresh, smaller PR branched off current main to avoid a messy conflict resolution.

@bokelley bokelley closed this Apr 24, 2026
bokelley added a commit that referenced this pull request Apr 24, 2026
…ck (#900)

* feat(cli): extract JUnit formatter + step-cmd hints + message= fallback

Cherry-picks the three salvageable deltas from closed PR #894 (which
overlapped heavily with merged #891 after a long rebase drift):

1. Extracts `formatStoryboardResultsAsJUnit` from `bin/adcp.js` into
   `src/lib/testing/storyboard/junit.ts` (@internal). Now unit-testable
   as a pure function — 10 tests pinning XML structure, hint integration,
   and escaping added in `test/lib/storyboard-junit-formatter.test.js`.

2. Wires `printStepHints` into `adcp storyboard step` — a printer site
   that #891 missed. Parameterizes the indent so both the phase-nested
   (`   `) and column-zero step printers share the helper.

3. Adds `<failure message=…>` attribute fallback to `hints?.[0]?.message`
   when `step.error` is absent. Closes an edge opened by the #883 gate
   widening (validation-only failures with no task-level error): CI
   dashboards that only read the attribute now surface the hint.

Drops the now-unused `formatHintsForFailureBody` export from
`adcp-step-hints.js` and its tests — the JUnit module owns its own hint
rendering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: move @internal onto the function + drop dead re-export (reviewer)

Code-reviewer flagged: the file-level `@internal` JSDoc was binding to
the next declaration (the import of `StoryboardResult`), so
stripInternal removed the import but left the function signature
referencing an undeclared name. Emitted `.d.ts` was:

    export declare function formatStoryboardResultsAsJUnit(
      results: StoryboardResult[]
    ): string;

— any downstream TS consumer hit TS2304. Fixed by moving the @internal
annotation onto the function itself; d.ts now correctly emits
`export {};`.

Also drops the re-export from `storyboard/index.ts` — stripInternal
already strips it from the public type surface and the CLI reaches into
`dist/lib/testing/storyboard/junit.js` directly, so the re-export was
dead for every consumer.

Testing-reviewer nits addressed:
- New test: empty results produces valid empty testsuites element
- New test: multi-storyboard + multi-phase aggregation (catches off-by-
  ones in totalTests / suiteTests reducers)
- New test: `step.error === ''` (empty string, not undefined) → first
  hint message fallback kicks in
- Replaced brittle escape-pipeline-reconstruction regex with explicit
  entity assertions; scoped `&apos;` check to the `message=` attribute
  where it's actually load-bearing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot mentioned this pull request Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runner hints: CLI and reporter don't render StoryboardStepResult.hints[]

2 participants