[REFACTOR](build) Migrate workspace to uv_build and PEP 420 namespace packages - #623
Open
Seth Fitzsimmons (sethfitz) wants to merge 3 commits into
Open
[REFACTOR](build) Migrate workspace to uv_build and PEP 420 namespace packages#623Seth Fitzsimmons (sethfitz) wants to merge 3 commits into
Seth Fitzsimmons (sethfitz) wants to merge 3 commits into
Conversation
Switch every workspace package's build backend to uv_build. Because the packages share the overture.schema namespace, each declares an explicit [tool.uv.build-backend] module-name (e.g. overture.schema.common); name normalization would derive the wrong module. The overture-schema aggregator uses module-name = "overture.schema" with namespace = true and ships only overture/schema/py.typed -- uv_build requires a module root, so a deps-only package still owns the namespace marker. Convert the pkgutil namespaces to PEP 420: uv_build ships shared namespaces implicitly and never packages an overture/__init__.py, so the extend_path shims are removed. Wheels and editable installs both merge the namespace natively. Deleting those __init__.py files removes the signal ruff's isort uses to detect overture as first-party (detect-same-package walks __init__ chains), so add src = ["packages/*/src"]. With overture now first-party everywhere, imports across the test suite and a few pyspark modules recanonicalize into their own group; that reformat is folded in here because it cannot land as a separately-green commit. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Stop the pyspark codegen from emitting empty `__init__.py` files so the generated expression and test trees are PEP 420 namespace packages, matching the rest of the workspace after the hatchling->uv_build migration. The mirrored layout (generated/overture/schema/<theme>/) is kept -- only the init emission is dropped. Removing the inits broke three things the bead's plan assumed would keep working; each needed a real fix: - pytest collection: generated conformance tests reached `_support` via deep relative imports whose depth `_support_prefix` computed. Under PEP 420 the module resolves as `generated.*` (tests/ is on pythonpath), not `tests.generated.*`, so those relative imports overshoot the top level. Emit absolute `from _support.X import ...` instead and delete the `_support_prefix` machinery. To keep one import style across the test tree, the hand-written tests move to the same absolute form. Set `--import-mode=importlib` (via addopts; it has no ini form) and `consider_namespace_packages = true` so collection walks the namespace. - type checking: mypy resolves `_support` as top-level only when its parent is a base, so add the pyspark tests dir to `mypy_path`. The hand-written `tests/__init__.py` stays: dropping it would make pyspark's `test_cli` a top-level module colliding with codegen's under importlib. - runtime registry: `pkgutil.walk_packages` skips subdirectories without `__init__.py`, finding zero generated modules under the PEP 420 tree. Walk the namespace roots as files instead. test_registry.py covers this -- no existing test exercised the on-disk walk, so an empty registry would have passed the suite silently. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
check-namespace fails if overture/ or overture/schema/ gains an __init__.py, which would turn a namespace root into a regular package and shadow other workspace packages' contributions at import. Wired into the check gate. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
🗺️ Schema reference docs preview is live!
Note ♻️ This preview updates automatically with each push to this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three commits.
refactor(build): migrate all packages from hatchling to uv_buildSwitch every workspace package's build backend to uv_build. Because the packages share the
overture.schemanamespace, each declares an explicit[tool.uv.build-backend] module-name(e.g.overture.schema.common); name normalization would derive the wrong module. Theoverture-schemaaggregator usesmodule-name = "overture.schema"withnamespace = trueand ships onlyoverture/schema/py.typed— uv_build requires a module root, so a deps-only package still owns the namespace marker.Convert the pkgutil namespaces to PEP 420: uv_build ships shared namespaces implicitly and never packages an
overture/__init__.py, so theextend_pathshims are removed. Wheels and editable installs both merge the namespace natively.Deleting those
__init__.pyfiles removes the signal ruff's isort uses to detectovertureas first-party (detect-same-package walks__init__chains), so addsrc = ["packages/*/src"]. Withoverturenow first-party everywhere, imports across the test suite and a few pyspark modules recanonicalize into their own group; that reformat is folded in here because it cannot land as a separately-green commit.refactor(build): emit PEP 420 generated pyspark treeStop the pyspark codegen from emitting empty
__init__.pyfiles so the generated expression and test trees are PEP 420 namespace packages, matching the rest of the workspace. The mirrored layout (generated/overture/schema/<theme>/) is kept — only the init emission is dropped.Removing the inits broke three things, each given a real fix:
_supportvia deep relative imports. Under PEP 420 the module resolves asgenerated.*, so those imports overshoot the top level. Emit absolutefrom _support.X import ..., delete the_support_prefixmachinery, and move the hand-written tests to the same form. Set--import-mode=importlibandconsider_namespace_packages = true.mypy_path; keep the hand-writtentests/__init__.pyso pyspark'stest_clidoesn't collide with codegen's under importlib.pkgutil.walk_packagesskips directories without__init__.py, finding zero generated modules. Walk the namespace roots as files instead;test_registry.pycovers this — no existing test exercised the on-disk walk, so an empty registry would have passed silently.chore(build): guard PEP 420 namespace roots against __init__.pyAdd a
check-namespacegate that fails ifoverture/oroverture/schema/gains an__init__.py, which would turn a namespace root back into a regular package and shadow other workspace packages' contributions at import. Wired into the check gate so the PEP 420 invariant this stack establishes cannot silently regress.Stacked on #620. Closes #618