[BUG](codegen) Unwrap RootModel fields to their bare root shape - #590
Merged
Conversation
A field typed as a RootModel subclass was extracted as a struct with a synthetic `root` member. RootModel is a BaseModel subclass, so it reached the model terminal and resolved to a ModelRef; the generated Spark schema declared `struct<root: ...>` while the Parquet data carries the bare root value. `analyze_type` now intercepts a RootModel before terminal classification, recursing into the root field's annotation and reattaching any root-level metadata onto the unwrapped layer. A `RootModel[dict[str, int]]` field becomes a bare `MapOf`, rendering as `MapType(...)` downstream. The reattach step reuses `attach_field_metadata`, extracted here into `type_analyzer` from `model_extraction` so field-level and root-level metadata share one implementation. Fixes #583. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
🗺️ Schema reference docs preview is live!
Note ♻️ This preview updates automatically with each push to this PR. |
Seth Fitzsimmons (sethfitz)
requested review from
Roel Bollens (RoelBollens-TomTom) and
Victor Schappert (vcschapp)
July 22, 2026 16:51
Collaborator
|
Looks good, just one remark, when you have a discovered model via entry point rather than used as a field it will still produces a RecordSpec with the root field. I'll leave it up to you if you want to harden against that already as RootModels as entry points could already cause additional issues upstream. |
Roel Bollens (RoelBollens-TomTom)
approved these changes
Jul 22, 2026
Victor Schappert (vcschapp)
approved these changes
Jul 22, 2026
Collaborator
Author
|
Good call on the entry point complication. I’ll have a follow-up PR up later this week. |
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #583. A field typed as a
pydantic.RootModelsubclass was extracted by the PySpark codegen as a struct with a syntheticrootmember, so the generated Spark schema declaredstruct<root: ...>while the Parquet data carries the bare root value. The mismatch surfaces only when validating real data.Root cause
RootModelis aBaseModelsubclass, so inextraction/type_analyzer.pya RootModel-typed field reached the model terminal and resolved to aModelRef— a struct whose one member isroot.Fix
analyze_type's_unwrapnow intercepts aRootModelbefore terminal classification: it recurses into therootfield's annotation and reattaches any root-level metadata via a sharedattach_field_metadatahelper, extracted here frommodel_extractionso field-level and root-level metadata share one implementation. ARootModel[dict[str, int]]field extracts to a bareMapOf, rendering asMapType(...)downstream — resolver or not.No current schema model uses
RootModel, so no generated output changes. This is a latent-bug fix that makes the codegen correct for RootModel-typed fields.