[BUG](codegen) Skip RootModel entry points from generation and document them as aliases - #594
[BUG](codegen) Skip RootModel entry points from generation and document them as aliases#594Seth Fitzsimmons (sethfitz) wants to merge 6 commits into
Conversation
GitHub Actions
Inactive
🗺️ Schema reference docs preview is live!
Note ♻️ This preview updates automatically with each push to this PR. |
Roel Bollens (@RoelBollens-TomTom) Victor Schappert (@vcschapp) this raises an interesting question (since I hadn't understood how |
A RootModel discovered as an entry point has no record structure of its own -- it serializes as its bare root value -- so extracting it as a top-level RecordSpec produced a spurious `root` column. Entry points are a contribution mechanism, not only a generation one: an extension may register a RootModel as a type used as a field elsewhere. So extract_model_spec now skips a RootModel entry rather than emitting a wrong schema, and the type still resolves to its bare shape wherever it is used as a field. Relates to #583. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
A RootModel entry point drops out of expression generation -- it has no record structure -- but not markdown: it is a named alias over its bare root value, like a NewType, and readers still need it documented. The split lives at the discovery bridge. extract_model_spec yields the feature/union specs both pipelines generate from (a RootModel returns None). extract_alias_spec turns a RootModel into a NewTypeSpec, reusing the NewType page since both are named aliases over an underlying shape. The markdown pipeline takes these aliases alongside the supplementary types it collects from field trees -- a RootModel is reachable no other way, since a RootModel field unwraps to its bare shape and names no type -- and places and cross-references them identically. Like every other non-feature type, a RootModel is simply absent from expression generation. Relates to #583. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
5d5b23f to
081b987
Compare
GitHub Actions
Inactive
081b987 to
9f4be78
Compare
GitHub Actions
Inactive
- spec_discovery: compute `partitions` after the RootModel early return, so the skipped path no longer builds a value it discards. - newtype_extraction: document why `extract_rootmodel_alias` needs no `is_custom_docstring` guard (a RootModel subclass without a docstring has `__doc__ = None` -- no inherited base docstring to filter). - type_analyzer: state the self-referential-RootModel precondition at the unwrap branch -- it bypasses the resolver's cycle detection, unguarded by the same convention `extract_model` states, since no schema defines a RootModel. - cli/pipeline: rename `alias_specs` -> `external_specs` at the markdown supplement-injection point; the parameter is typed `SupplementarySpec`, the open-ended door for externally-supplied supplements (extension contributions), not aliases specifically. Also drop its speculative default on `_generate_markdown` -- its one real caller always passes it. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
…mitation The note first appealed to a convention `extract_model` does not state, then to "no Overture schema defines one" -- but this codegen is built for use beyond Overture, where a downstream RootModel could be self-referential. State it as a known limitation instead: guarding would thread cycle state through the otherwise-stateless unwrap, deferred until a use case needs self-referential roots. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Unwrapping a RootModel erases its identity, so -- unlike a self-referential BaseModel, which the resolver terminates with a `starts_cycle` back-edge -- a self-referential root (`RootModel[list["Self"]]`) has no node to carry a back-edge and no finite bare-shape form. The unwrap recursed into it forever, surfacing as an opaque RecursionError. Thread a `seen_rootmodels` frozenset through `_unwrap` (defaulted on the `_recurse` closure, so ordinary descents carry it unchanged and only the RootModel branch augments it) and raise a clear TypeError on re-entry -- the analyzer's counterpart to the model layer's `ancestors`/`starts_cycle` detection, deterministic and catching the cause at the back-edge. The test rewires the self-reference at runtime: a statically recursive generic base crashes mypy 2.3.0 (an internal-error bug), so the base stays non-recursive and `model_fields["root"].annotation` is pointed back at the model after definition. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
`extract_rootmodel_alias` reads a RootModel's root description from `model_fields["root"].description`. Pydantic hoists that description from the root type's `Annotated` metadata onto the root `FieldInfo` only from 2.13.0 -- on 2.12.x it stays nested in `root.annotation` and `.description` is None. The `lowest-direct` CI job resolves the declared floor, so it installed 2.12.0 and failed `test_root_field_description_used_without_docstring`. Raise the floor to the first version that surfaces the description. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
9f4be78 to
22f605d
Compare
I'd say probably not, but still worth discussing. When I initially saw the NewTypes (and type alias for segment, and this is before the markdown and pyspark codegen) I did wonder why RootModel wasn't used, but when internalising this, I don't think you would actually have gained anything, and in fact would have added some clunkiness (you'd need to to through the |
|
I'll be working through it for a bit longer, but here's where I think I'm landing: migrate It seems like it makes the implementation of discovery and extension slightly messier, but it cleans up the interfaces a bit and allows things like |
2 participants
Summary
Fixes #593. Follow-up to #590, which unwrapped RootModel-typed fields to their bare root shape. This handles a RootModel registered as a top-level entry point, which was still extracted as a
RecordSpec-- producing a spuriousrootcolumn in the generated Spark schema -- and was omitted from the markdown reference.Changes
Expression generation skips a RootModel entry point silently (the same as would occur for
NewTypesor other entry point contributions) rather than emitting a wrong schema. An entry point is a contribution mechanism, not only a generation one -- the type still resolves to its bare shape wherever it is used as a field (the #590 path).Markdown generation documents a RootModel entry point as a type alias over its bare root value, reusing the NewType page since both are named aliases over an underlying shape. The split lives at the discovery bridge:
extract_model_specyields the feature/union specs both pipelines generate from (a RootModel returnsNone), andextract_alias_specturns a RootModel into aNewTypeSpec. The markdown pipeline takes these aliases alongside the supplementary types it collects from field trees -- a RootModel is reachable no other way, since a RootModel field unwraps to its bare shape and names no type.A self-referential RootModel (
RootModel[list["Self"]]) now raises a clearTypeErrorinstead of recursing forever into an opaqueRecursionError. Unwrapping a RootModel erases its identity, so -- unlike a self-referentialBaseModel, which the resolver terminates with astarts_cycleback-edge -- a self-referential root has no node to carry a back-edge and no finite bare-shape form. Aseen_rootmodelsfrozenset threaded through_unwrapdetects re-entry at the back-edge.Dependency: pydantic >= 2.13
Documenting the alias reads the RootModel's root description from
model_fields["root"].description. Pydantic surfaces it there only from 2.13.0: pydantic#13129 ("PreserveRootModelcore metadata") carries the root field'sFieldInfometadata -- description included -- through to the core schema. On 2.12.0 the description stays nested in the root type'sAnnotatedmetadata andmodel_fields["root"].descriptionisNone, so thelowest-directCI cell -- which resolves every direct dependency to its declared floor -- failedtest_root_field_description_used_without_docstring.This raises the pydantic floor from
>=2.12.0to>=2.13.0across the workspace packages (with a matchinguv.lockspecifier bump). Verified under the failing configuration: an isolatedUV_RESOLUTION=lowest-directinstall resolves pydantic to 2.13.0 and the root-description test passes.