feat(conformance): add FormatMismatchHint to StoryboardStepHint taxonomy#956
Closed
bokelley wants to merge 2 commits into
Closed
feat(conformance): add FormatMismatchHint to StoryboardStepHint taxonomy#956bokelley wants to merge 2 commits into
bokelley wants to merge 2 commits into
Conversation
- Fix misleading expected_format fallback: 'format' → '(unknown)' when AJV message template doesn't match (avoids "must match format 'format'") - Add comment explaining why no stepFailed guard is needed for format hints - Add RFC 6901 §3 decode-order comment in pointerToDotPath - Add unit tests for detectFormatMismatchHints (13 cases: gate, extraction, pointer decoding, truncation, fallback, edge cases) https://claude.ai/code/session_01DuuYnsXwSs3znAi9b3bbHH
Contributor
Author
|
Superseded by #959 — landed there via strict-AJV partitioning in . |
Contributor
Author
|
Superseded by #959 — |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #947
Summary
StoryboardStepHintfromContextValueRejectedHinttoContextValueRejectedHint | FormatMismatchHintFormatMismatchHintinterface with:kind,message,tool,instance_path(RFC 6901 pointer — renamed from issue'sfield_pathto matchSchemaValidationError.instance_pathconvention),expected_format, and optionalobserved_valueformat-mismatch-hints.ts(parallel torejection-hints.ts): fires only onstrict_only_failurevalidations — lenient Zod passed, strict AJV rejected onkeyword === 'format'detectFormatMismatchHintsas a separate parallel pass outside thestepFailedguard, since format hints are most valuable when the step is green (silent strictness gap)FormatMismatchHintanddetectFormatMismatchHintsfrom the public storyboard and testing indexes~0/~1escapes, array indices),observed_valueextraction and truncation,expected_formatregex fallback, edge casesWhat's not changing
hints[]stays non-fatalValidationResult.warningstring still fires (unchanged) — the hint is additivedetectContextRejectionHintsgate (stepFailed) is unchangedImplementation notes
instance_path: renamed fromfield_path(issue spec) toinstance_pathto align withSchemaValidationError.instance_path— this type is a filtered projection of that typeexpected_format: parsed from AJV's stable message templatemust match format "<name>"via regex; falls back to'(unknown)'on mismatch (avoids confusingmust match format "format"when using the keyword as fallback)observed_value: resolved by converting RFC 6901 pointer → dot-bracket notation, thenresolvePath(). Strings only, truncated at 200 chars via code-point slicing. Non-string values (objects, arrays, numbers) omittedstepFailedguard needed: skipped/probe/webhook-assertion steps have emptyvalidations[]so no hints emitNits noted (not fixed)
pointerToDotPathinformat-mismatch-hints.tsduplicatesnormalizeFieldPathinrejection-hints.ts. Both could be extracted topath.ts. Deferring to keep this PR scoped.What was tested
@types/node+moduleResolutionenv errors only — same as PR feat(conformance): add ShapeDriftHint to StoryboardStepHint taxonomy #937)prettier --checkpassesnode --test test/lib/storyboard-format-mismatch-hints.test.js)Pre-PR review
field_path→instance_path); nit onpointerToDotPathduplication noted above; no protocol change confirmed; minor changeset classification correct(unknown)fallback and added guard comment; added 13 unit tests covering all non-trivial cases flagged in reviewSession: https://claude.ai/code/session_01DuuYnsXwSs3znAi9b3bbHH