Skip to content

fix(core): infer whole numbers outside the target's integer range as double#2938

Merged
schani merged 1 commit into
masterfrom
fix/2931-int64-range-overflow
Jul 14, 2026
Merged

fix(core): infer whole numbers outside the target's integer range as double#2938
schani merged 1 commit into
masterfrom
fix/2931-int64-range-overflow

Conversation

@schani

@schani schani commented Jul 13, 2026

Copy link
Copy Markdown
Member

Fixes #2931.

A whole number in input JSON that lies outside the target language's integer range was always inferred as integer, so e.g. Go got int64 for 123456789012345678901234567890 and the generated code failed to unmarshal the very sample it was generated from:

json: cannot unmarshal number 123456789012345678901234567890 into Go struct field Edge.bigger of type int64

What this does

Each TargetLanguage now declares its supported integer range — the inclusive range of whole numbers its integer type can round-trip, given as decimal strings because the boundaries (e.g. INT64_MAX) are not exactly representable as doubles. Whole numbers outside the range are inferred as double; null means arbitrary-precision.

Languages Range
default (Go, C#, Java, Swift, Rust, …) int64
JavaScript, TypeScript, Flow, PropTypes, Zod, Effect Schema, Elm ±(2^53 − 1)
Crystal int32 (its renderer emits Int32)
cJSON follows the integer-size renderer option (int64_t by default)
Python, Ruby, Elixir, Pike, JSON Schema unlimited

Because cJSON's width depends on a renderer option, the range is exposed as getSupportedIntegerRange(rendererOptions); the CLI and jsonInputForTargetLanguage (which gained an optional rendererOptions parameter) pass the renderer options through when constructing JSON input.

Both JSON parsers consult the range:

  • the CLI's streaming parser accumulates the literal's digit string and checks it exactly;
  • the JSON.parse-based core parser (which previously hardcoded the JS safe-integer cutoff for every language) checks the double's exact decimal value via toFixed(0), erring toward double at range boundaries the double can't represent. Side benefit: Go via the core API now keeps int64 for whole numbers between 2^53 and INT64_MAX instead of widening them to double.

Testing

  • Unit tests: comparator boundaries (INT64_MAX/MIN ± 1, 2^53 ± 1, leading zeros, -0), Go, Python, and Crystal end-to-end through both parsers, cJSON under integer-size=int16_t, per-language range declarations, and a test showing the same literal classifying differently under different ranges.
  • Verified with the built CLI: the issue's repro now generates float64 and the generated Go unmarshals its own sample; python keeps int; -l schema keeps "type": "integer"; --lang crystal turns 2147483648 into Float64; --lang cjson --integer-size int8_t turns 128 into double.

🤖 Generated with Claude Code

@schani

schani commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

But this needs to be language-specific, no? JS different limits than Go for example!

@schani

schani commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Good point — reworked in 93b65a7. TargetLanguage now declares its own range:

  • supportedIntegerRange on TargetLanguage returns the inclusive range of whole numbers the language's integer type can round-trip (as decimal strings, since the int64 boundaries aren't exact doubles), or null for arbitrary-precision. The base default is int64.
  • JS family (JavaScript, TypeScript, Flow, PropTypes, Zod, Effect Schema) → ±(2^53 − 1).
  • Python, Ruby, Elixir, Pikenull (bignum), likewise JSON Schema, whose integer is unbounded — so 123456789012345678901234567890 stays int in Python and "type": "integer" in schema output, while Go gets float64.
  • Both parsers consult the range: the CLI's streaming parser checks the exact digit string; the JSON.parse-based core parser (which previously hardcoded the 2^53 cutoff for every language) checks the double's exact decimal value via toFixed(0), erring toward double at boundaries the double can't represent. A side benefit: Go via the core API now keeps int64 for whole numbers between 2^53 and INT64_MAX instead of widening them to double.

Tests cover the comparator boundaries, Go/Python end-to-end through both parsers, and a case showing the same literal classifying differently under different ranges.

🤖 Generated with Claude Code

@schani schani changed the title fix(core): classify whole numbers outside int64 range as double fix(core): infer whole numbers outside the target's integer range as double Jul 13, 2026

schani commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Review result: one correctness issue remains before merge.

The new TargetLanguage.supportedIntegerRange default assumes every non-overridden target uses signed 64-bit integers. That is not true for all renderers. Crystal renders inferred integers as Int32; I reproduced { "n": 2147483648 } generating property n : Int32, so the generated code still cannot consume the sample. CJSON is also renderer-option-dependent: { "n": 128 } with integer-size=int8_t still generates int8_t n.

The effective range therefore needs to cover fixed non-int64 targets such as Crystal and renderer-dependent widths such as CJSON's integer-size, rather than relying on an int64 base default alone.

Validation otherwise passed: the PR builds cleanly, all 82 unit tests pass, and the GitHub Test workflow is successful.

…double

Instead of a fixed int64 check, each TargetLanguage now declares its
supported integer range — the inclusive range of whole numbers its
integer type can round-trip, as decimal strings since the boundaries
exceed exact doubles.  The default is int64; JavaScript-family languages
(including Elm, whose Int is a JS number at runtime) are exact only
within ±(2^53 - 1); Crystal renders integers as Int32; Python, Ruby,
Elixir, and Pike have arbitrary-precision integers, and JSON Schema's
`integer` is unbounded, so they have no limit.

Some languages' integer width depends on a renderer option — cJSON's
integer-size can narrow it down to int8_t — so the range is exposed as
getSupportedIntegerRange(rendererOptions), and both the CLI and
jsonInputForTargetLanguage pass the renderer options through when
constructing JSON input.

Both JSON parsers consult the range: the CLI's streaming parser checks
the exact digit string, and the JSON.parse-based core parser checks the
double's exact decimal value via toFixed(0), which errs toward `double`
at range boundaries the double can't represent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@schani
schani force-pushed the fix/2931-int64-range-overflow branch from 93b65a7 to 751ac79 Compare July 14, 2026 13:05
@schani

schani commented Jul 14, 2026

Copy link
Copy Markdown
Member Author

Both fixed in 751ac79 (also rebased onto master):

  • Crystal now declares an int32 range, since its renderer emits Int32: your repro { "n": 2147483648 } now generates property n : Float64, and 2147483647 stays Int32.
  • cJSON's range now follows the integer-size renderer option. The range is exposed as getSupportedIntegerRange(rendererOptions) — same untyped option values makeRenderer gets — and both the CLI and jsonInputForTargetLanguage (new optional rendererOptions parameter) pass the options through when constructing JSON input. { "n": 128 } with --integer-size int8_t now generates double n.
  • While surveying every renderer's integer mapping for other offenders, I found one more: Elm compiles to JavaScript, so its Int is a double at runtime — it's now in the ±(2^53 − 1) family. Everything else genuinely uses 64-bit integers (Java/Kotlin/Scala Long, C# long, Rust i64, C++ int64_t, Swift/Dart/PHP native 64-bit Int, Objective-C NSInteger, Haskell 64-bit Int on GHC).

New tests cover the per-language range declarations, Crystal end-to-end through the streaming parser, and cJSON under integer-size=int16_t through the core parser.

🤖 Generated with Claude Code

@schani
schani merged commit f103f0a into master Jul 14, 2026
24 checks passed
@schani
schani deleted the fix/2931-int64-range-overflow branch July 14, 2026 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: Numbers beyond int64 range are typed as int64/long — generated Go can't unmarshal the sample it was generated from

1 participant