Description
The is_valid_json_escape() function in the JSON repair state machine does NOT include 'u' in its valid escape characters. When encountering valid \uXXXX unicode escapes, the state machine falls into the invalid escape branch and doubles the backslash to \\uXXXX, corrupting valid JSON.
The existing code in the valid-escape branch already handles consuming the 4 hex digits for \u — but 'u' is missing from the is_valid_json_escape match.
Impact
- Any LLM response containing unicode escapes (e.g.,
\u0041) will be corrupted by the repair function
- The repair function introduced in v0.1.4 can introduce bugs that weren't there before
Suggested Fix
Add 'u' to matches in is_valid_json_escape:
matches!(c, '"' | '\\' | '/' | 'b' | 'f' | 'n' | 'r' | 't' | 'u')
Discovered by
Cora self-review on v0.1.3..v0.1.4
Description
The
is_valid_json_escape()function in the JSON repair state machine does NOT include'u'in its valid escape characters. When encountering valid\uXXXXunicode escapes, the state machine falls into the invalid escape branch and doubles the backslash to\\uXXXX, corrupting valid JSON.The existing code in the valid-escape branch already handles consuming the 4 hex digits for
\u— but'u'is missing from theis_valid_json_escapematch.Impact
\u0041) will be corrupted by the repair functionSuggested Fix
Add
'u'to matches inis_valid_json_escape:Discovered by
Cora self-review on v0.1.3..v0.1.4