Skip to content

Infra: add bindings/python into root workspace#2506

Open
kevinjqliu wants to merge 15 commits into
apache:mainfrom
kevinjqliu:kevinjqliu/merge-python-bindings-to-workspace
Open

Infra: add bindings/python into root workspace#2506
kevinjqliu wants to merge 15 commits into
apache:mainfrom
kevinjqliu:kevinjqliu/merge-python-bindings-to-workspace

Conversation

@kevinjqliu
Copy link
Copy Markdown
Contributor

@kevinjqliu kevinjqliu commented May 25, 2026

Which issue does this PR close?

  • Closes #.

What changes are included in this PR?

Migrates bindings/python from a standalone, excluded crate into the root Cargo workspace.

This lets pyiceberg_core_rust inherit workspace versions, dependency declarations, package metadata, and the root lockfile instead of maintaining a parallel Rust dependency setup under bindings/python.

Changes

  • Add bindings/python as a workspace member.
  • Update bindings/python/Cargo.toml to inherit shared package metadata and dependencies with { workspace = true }.
  • Remove the standalone bindings/python/Cargo.lock; Python bindings now use the root Cargo.lock.
  • Remove pyo3/extension-module from workspace Cargo features and require maturin >= 1.9.4, so maturin sets extension-module build mode only for Python extension builds.
  • Add a dedicated py-release Cargo profile for Python wheel builds and configure maturin/release workflows to use it.
  • Remove the redundant Python check-rust CI job; Rust validation is covered by the main workspace CI.
  • Simplify release workflow version reads with cargo metadata.
  • Refactor overwrite-package-version to accept a PEP 440 version input and write it to pyproject.toml.
  • Fix nightly and release-candidate version overrides to update pyproject.toml instead of the workspace Cargo.toml, since .dev and -rc.N versions are not valid Cargo semver and can break workspace dependency resolution.
  • Update release documentation for the new Python binding workflow.

Notes

  • Plain Cargo workspace builds now include pyiceberg_core_rust, while actual Python wheels continue to be built through maturin. The Python-specific release optimization settings are isolated in the py-release profile so normal Rust release builds are not affected.

  • make build now builds bindings/python as part of the workspace because it runs cargo build --all-targets --all-features --workspace. This is intentional for now: after removing pyo3/extension-module, the Python Rust crate can be built by plain Cargo, so the main workspace build can validate it.

  • A possible follow-up improvement is to exclude bindings/python from the default root build/test path with workspace.default-members, while keeping full validation in cargo check --workspace, clippy, and the maturin-based Python CI/release workflows. That would avoid building the Python cdylib in normal Rust builds while still keeping workspace dependency inheritance and CI coverage.

Are these changes tested?

Yes

Tested affected workflows on my fork:

Copy link
Copy Markdown
Contributor Author

@kevinjqliu kevinjqliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i also made some refactor to .github/actions/overwrite-package-version/action.yml, .github/workflows/release_python.yml, and .github/workflows/release_python_nightly.yml to align Cargo workspace version with pyproject toml version

@kevinjqliu kevinjqliu marked this pull request as ready for review May 25, 2026 20:47
@kevinjqliu kevinjqliu requested a review from blackmwk May 25, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant