Fix crash on recursive allOf/anyOf schemas (OPDS 2.0, FHIR)

[asm0dey]

Description

The union-flattening fixpoint (Run.ts) never converged on JSON Schemas with cyclic composition (recursive allOf/anyOf), causing Error: Maximum call stack size exceeded.

FlattenUnions.replace built its result unions with a raw getUnionType callback that bypasses unifyTypes' in-progress memoization (registerUnion). As a result, a recursive reference back into the type currently being built was re-unified into a fresh copy on every pass instead of tying the knot. The recursion unrolled one level per pass, the type graph grew without bound, and the reconstitution eventually overflowed the stack.

Changes (quicktype-core):

• rewrites/FlattenUnions.ts — once the graph is intersection-free, unify recursively via unionBuilderForUnification (the same knot-tying path combineClasses already uses) instead of the raw callback. Gated on the absence of intersection types, whose members the unification accumulator can't process during the intersection/union resolution phase.
• UnifyClasses.ts (makeObject) — unifying a class with a map of any (a permissive additionalProperties: true) now drops the un-representable additional and produces the class, rather than asserting We have additional properties but want to make a class. A non-any additional would already have taken the map branch, so only permissive any additionals reach this point.
• Type/TypeGraphUtils.ts (combineIdenticalTypes) + Run.ts — after each flatten pass, merge types that are exactly bisimilar (structurallyCompatible, no number conflation) so recursive schemas don't leave behind piles of duplicate types. It only ever merges structurally identical types, so it never over-generalizes.

Related Issue

Fixes #2187 (OPDS 2.0 feed schema). Also fixes the same class of failure in #1376 (FHIR schema).

Motivation and Context

quicktype crashed outright — not just wrong output — on real-world, non-pathological schemas that use recursive composition. app.quicktype.io and the CLI both failed on the OPDS 2.0 feed schema for every target language, and the FHIR schema in #1376 (open since 2020) fails identically. Both share one root cause: the flatten fixpoint re-derives recursive merged types instead of tying the recursive knot.

Previous Behaviour / Output

$ quicktype -l ts -s schema --src feed.schema.json
Error: Maximum call stack size exceeded.

(Same for #1376's fhir.schema.json in any language.)

New Behaviour / Output

Both schemas now generate finite, sensible types in every target language:

• OPDS 2.0 feed: 47 Java classes / 25 TS interfaces (TS/C#/Python/Go/Rust also clean).
• FHIR (#1376, 3.4 MB): 377 Rust structs.

How Has This Been Tested?

• Reproduced both crashes on master (68ecb28…) and confirmed both now generate with this change.
• Regression: generated TypeScript for all 131 test/inputs/schema/*.json fixtures and 100 test/inputs/json inputs, before vs after — output is byte-for-byte identical. The only behavioural change is that previously-crashing recursive schemas now succeed.
• Existing recursive fixtures (mutually-recursive) unchanged.

I couldn't run the full test/test.ts compile-and-round-trip harness locally (needs every language toolchain). Happy to add a fixture (I have a 46 KB self-contained OPDS repro and the FHIR file) if a maintainer can validate the round-trip in CI.

This change was written with AI assistance (Claude). Human review before merge is expected.

[schani]

@asm0dey Thank you for your contribution!

It's failing CI.

Also I wonder (not having looked into this too much) whether this is more of a kludge than a proper fix. Can we fix the pass that screws up, instead of introducing a new pass that salvages the screw-up?

Also this should have a test that would previously fail.

[asm0dey]

This is indeed an interesting question. Before this, I tried to research more obvious paths — like "why can't we just cache url#json/path"? Turns out we already kinda do, and the issue is — we never actually reach the point of stabilisation

[asm0dey]

Aha, so now it merges too aggressively, I see…

[asm0dey]

Don't run it yet; I'm trying to experiment here

[asm0dey]

Pushed a follow-up commit fixing two issues that were failing CI, both in the new combineIdenticalTypes pass.

1. Provenance invariant failure (all schema/JSON jobs)

All schema/JSON fixtures run with --debug provenance. Merging a type orphans its (structurally identical but distinct) sub-types, so their attributes — including the provenance the graph tracks — are dropped, tripping Type attributes for N types were not carried over to the new graph. TypeGraph.remap now takes a lostTypeAttributes flag so combineIdenticalTypes can signal the intentional loss, the same way other knowingly-lossy rewrites call setLostTypeAttributes.

2. Attribute-blind over-merge (schema-roundtrip, every language)

Candidates were selected with structurallyCompatible, which compares kinds and structure but ignores type attributes. So a plain string and an enum-bearing string look "compatible", making a null | string union compatible with a null | <enum> union — merging the two collapsed a free-form field onto the enum. On us-senators.json, twitterid / youtubeid / leadership_title rendered as the Party enum when generated via JSON Schema, diverging from the direct-from-JSON output:

-    "twitterid": S.NullOr(S.String),
-    "youtubeid": S.NullOr(S.String),
+    "twitterid": S.NullOr(Party),
+    "youtubeid": S.NullOr(Party),

The identity map in TypeBuilder already deduplicates attribute-equal non-recursive duplicates at creation time; the only duplicates it misses — and the only ones this pass needs to remove — are recursive ones. Merging is now restricted to types that lie on a cycle (iterative Tarjan SCC pass), which still collapses the recursive duplicates the fixpoint leaves behind while leaving attribute-distinct non-recursive types alone.

Verification

Emulated the CI matrix locally on every toolchain available on my machine:

• ✅ typescript (+zod, +effect-schema), javascript, javascript-prop-types, comment-injection, golang, rust, kotlin (+jackson) — all pass
• ✅ All schema/JSON fixtures pass under --debug provenance, no stack overflows (recursive dedup on e.g. vega-lite still works)
• ⚠️ ruby / java / python fail only on local toolchain version drift (Ruby 3.4 dropped bigdecimal, JDK 21 rejects source 6, generated run.sh calls python3.7) — not code issues
• ⛔ csharp / flow / haskell / dart / php / scala3 / elixir / swift not installed locally; they run the same language-independent roundtrip, so CI is the confirmation there

🤖 Generated with Claude Code

[asm0dey]

Now it looks more correct and passes someci-like pipelines on my machine, but still can't remove replaceWithUnification — but claude had to add a proper cycle detection too to not overmerge

[schani]

@asm0dey Do you have a small repro for this? I'd like to try if I can maybe find a simpler fix.

[asm0dey]

Coming up with a small repro has proven to be extremely difficult - trying to minimize the example from #1376 rn

[asm0dey]

So, I've uploaded a slightly simplified version of the OPDS feed schema — without external dependencies. But I couldn't really minimize it, sadly :( It seems there should be more or less complex cross-dependencies.

[schani]

I was able to find a much simpler repro:

{
  "items": {
    "anyOf": [
      { "$ref": "#/definitions/a" },
      { "additionalProperties": { "$ref": "#" } }
    ]
  },
  "definitions": {
    "a": {
      "properties": {
        "x": { "items": { "$ref": "#/definitions/a" } }
      }
    }
  }
}

Not that it makes much sense, but it does crash quicktype. Will try to work with that.

[schani]

Explainer of what's happening:

https://hostr.flingit.run/s/schani/quicktype-unroll-explainer/

[asm0dey]

Is it fixed with this fix?

[schani]

This seems a better fix: #2891

Closing this in favor of that.

[asm0dey]

Your fix is so much more elegant!