Skip to content

[REFACTOR](build) Migrate workspace to uv_build and PEP 420 namespace packages - #623

Open
Seth Fitzsimmons (sethfitz) wants to merge 3 commits into
extract-validation-packagefrom
pep-420
Open

[REFACTOR](build) Migrate workspace to uv_build and PEP 420 namespace packages#623
Seth Fitzsimmons (sethfitz) wants to merge 3 commits into
extract-validation-packagefrom
pep-420

Conversation

@sethfitz

Copy link
Copy Markdown
Collaborator

Three commits.

refactor(build): migrate all packages from hatchling to uv_build

Switch every workspace package's build backend to uv_build. Because the packages share the overture.schema namespace, each declares an explicit [tool.uv.build-backend] module-name (e.g. overture.schema.common); name normalization would derive the wrong module. The overture-schema aggregator uses module-name = "overture.schema" with namespace = true and ships only overture/schema/py.typed — uv_build requires a module root, so a deps-only package still owns the namespace marker.

Convert the pkgutil namespaces to PEP 420: uv_build ships shared namespaces implicitly and never packages an overture/__init__.py, so the extend_path shims are removed. Wheels and editable installs both merge the namespace natively.

Deleting those __init__.py files removes the signal ruff's isort uses to detect overture as first-party (detect-same-package walks __init__ chains), so add src = ["packages/*/src"]. With overture now first-party everywhere, imports across the test suite and a few pyspark modules recanonicalize into their own group; that reformat is folded in here because it cannot land as a separately-green commit.

refactor(build): emit PEP 420 generated pyspark tree

Stop the pyspark codegen from emitting empty __init__.py files so the generated expression and test trees are PEP 420 namespace packages, matching the rest of the workspace. The mirrored layout (generated/overture/schema/<theme>/) is kept — only the init emission is dropped.

Removing the inits broke three things, each given a real fix:

  • pytest collection: generated conformance tests reached _support via deep relative imports. Under PEP 420 the module resolves as generated.*, so those imports overshoot the top level. Emit absolute from _support.X import ..., delete the _support_prefix machinery, and move the hand-written tests to the same form. Set --import-mode=importlib and consider_namespace_packages = true.
  • type checking: add the pyspark tests dir to mypy_path; keep the hand-written tests/__init__.py so pyspark's test_cli doesn't collide with codegen's under importlib.
  • runtime registry: pkgutil.walk_packages skips directories without __init__.py, finding zero generated modules. Walk the namespace roots as files instead; test_registry.py covers this — no existing test exercised the on-disk walk, so an empty registry would have passed silently.

chore(build): guard PEP 420 namespace roots against __init__.py

Add a check-namespace gate that fails if overture/ or overture/schema/ gains an __init__.py, which would turn a namespace root back into a regular package and shadow other workspace packages' contributions at import. Wired into the check gate so the PEP 420 invariant this stack establishes cannot silently regress.

Stacked on #620. Closes #618

Switch every workspace package's build backend to uv_build. Because the
packages share the overture.schema namespace, each declares an explicit
[tool.uv.build-backend] module-name (e.g. overture.schema.common); name
normalization would derive the wrong module. The overture-schema
aggregator uses module-name = "overture.schema" with namespace = true
and ships only overture/schema/py.typed -- uv_build requires a module
root, so a deps-only package still owns the namespace marker.

Convert the pkgutil namespaces to PEP 420: uv_build ships shared
namespaces implicitly and never packages an overture/__init__.py, so the
extend_path shims are removed. Wheels and editable installs both merge
the namespace natively.

Deleting those __init__.py files removes the signal ruff's isort uses to
detect overture as first-party (detect-same-package walks __init__
chains), so add src = ["packages/*/src"]. With overture now first-party
everywhere, imports across the test suite and a few pyspark modules
recanonicalize into their own group; that reformat is folded in here
because it cannot land as a separately-green commit.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Stop the pyspark codegen from emitting empty `__init__.py` files so the
generated expression and test trees are PEP 420 namespace packages,
matching the rest of the workspace after the hatchling->uv_build
migration. The mirrored layout (generated/overture/schema/<theme>/) is
kept -- only the init emission is dropped.

Removing the inits broke three things the bead's plan assumed would keep
working; each needed a real fix:

- pytest collection: generated conformance tests reached `_support` via
  deep relative imports whose depth `_support_prefix` computed. Under
  PEP 420 the module resolves as `generated.*` (tests/ is on
  pythonpath), not `tests.generated.*`, so those relative imports
  overshoot the top level. Emit absolute `from _support.X import ...`
  instead and delete the `_support_prefix` machinery. To keep one import
  style across the test tree, the hand-written tests move to the same
  absolute form. Set `--import-mode=importlib` (via addopts; it has no
  ini form) and `consider_namespace_packages = true` so collection walks
  the namespace.

- type checking: mypy resolves `_support` as top-level only when its
  parent is a base, so add the pyspark tests dir to `mypy_path`. The
  hand-written `tests/__init__.py` stays: dropping it would make
  pyspark's `test_cli` a top-level module colliding with codegen's under
  importlib.

- runtime registry: `pkgutil.walk_packages` skips subdirectories without
  `__init__.py`, finding zero generated modules under the PEP 420 tree.
  Walk the namespace roots as files instead. test_registry.py covers
  this -- no existing test exercised the on-disk walk, so an empty
  registry would have passed the suite silently.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
check-namespace fails if overture/ or overture/schema/ gains an
__init__.py, which would turn a namespace root into a regular package
and shadow other workspace packages' contributions at import. Wired into
the check gate.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@github-actions

Copy link
Copy Markdown

🗺️ Schema reference docs preview is live!

🌍 Preview https://staging.overturemaps.org/schema/pr/623/schema/index.html
🕐 Updated Jul 31, 2026 18:41 UTC
📝 Commit 48c237f
🔧 env SCHEMA_PREVIEW true

Note

♻️ This preview updates automatically with each push to this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant