Build linux/aarch64 wheels by compiling VTK from source - #6
Open
jqmcginnis wants to merge 1 commit into
Open
Conversation
picsl-greedy publishes no linux aarch64 wheel. The blocker is not greedy
itself, which builds fine on arm64 as the macOS arm64 wheels show, but
Kitware's VTK wheel-SDK that prebuild.sh downloads: it exists for x86_64
only, with no aarch64 build at any published version (9.3.1 and 9.4.1
return 404 for the aarch64 tarball, and 9.5.0 and 9.6.0 publish no SDK).
Everything else prebuild.sh needs is already built from source and is
architecture-agnostic: Eigen, ITK and greedy's own libraries. Only VTK
has to change.
On aarch64, build VTK from source instead of downloading the SDK, with
only the ten modules greedy's CMakeLists requires and the
Rendering/Qt/Views/Web/Imaging/MPI groups refused. That is what keeps it
cheap: the whole prebuild takes about 10 minutes in manylinux2014_aarch64.
Two details worth noting:
* The ubuntu branch previously matched on OS alone, so enabling an arm
runner would have downloaded the x86_64 SDK rather than failing. It
now tests uname -m.
* VTK's exported targets resolve library paths relative to the config
file, walking three levels up from vtk-9.3.1.data/headers/cmake. VTK
is therefore installed into install/vtk so that arithmetic lands
correctly, and the config is symlinked to the path the wheel-SDK
would have provided, so CIBW_ENVIRONMENT's VTK_DIR needs no
per-architecture variant and every other platform is untouched.
build_wheels.yml enables the ubuntu-22.04-arm runner already present in
the commented-out matrix, and sets CIBW_ARCHS_LINUX to auto so each linux
runner builds its native architecture.
Verified on a DGX Spark (GB10, aarch64): the resulting
picsl_greedy-1.4.0-cp312-cp312-linux_aarch64 wheel imports and registers.
On five subjects of the Mendeley MS dataset, an affine registration
recovering a known 6 degree rotation plus 5 voxel translation raised
expert lesion-mask Dice from a range of 0.15 to 0.40 up to a mean of
0.990.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi, thank you very much for providing greedy. We are actively using it in LST-AI and really love the speed and performance.
Right now, we are actively trying to port our stack to non x86 architectures, most recently using the DGX Spark architecture (ARM-based CPU).
Why?
picsl-greedypublishes no linux aarch64 wheel. The blocker isn't greedy; it builds fine on arm64, as the macOS arm64 wheels demonstrate. It's Kitware's VTK wheel-SDK thatprebuild.shdownloads, which exists for x86_64 only:Everything else
prebuild.shneeds (Eigen, ITK, and greedy's own libraries) is already built from source and is architecture-agnostic. Only VTK has to change.What
On aarch64, build VTK from source instead of downloading the SDK, enabling only the ten modules greedy's
CMakeLists.txtrequires and refusing the Rendering/Qt/Views/Web/Imaging/MPI groups. That's what keeps it cheap, the whole prebuild takes about 10 minutes inmanylinux2014_aarch64.build_wheels.ymlenables theubuntu-22.04-armrunner already present in your commented-out matrix, and setsCIBW_ARCHS_LINUX: autoso each linux runner builds natively.Two things you may want to know regardless of this PR
The
ubuntu-*branch matches on OS alone, so enabling an arm runner without this change would have silently downloaded the x86_64 SDK rather than failing. It now testsuname -m.VTK's exported targets resolve library paths relative to the config file. CMake walks three levels up from
vtk-9.3.1.data/headers/cmake, so VTK is installed intoinstall/vtkfor that arithmetic to land correctly, and the config is symlinked to the SDK's path. This meansCIBW_ENVIRONMENT'sVTK_DIRneeds no per-arch variant and every other platform is byte-identical.Verification
Built on a DGX Spark (GB10, aarch64), producing
picsl_greedy-1.4.0-cp312-cp312-linux_aarch64.whl. The wheel imports and registers. On five subjects of the Mendeley MS dataset, an affine registration recovering a known 6 degree rotation plus 5 voxel translation raised expert lesion-mask Dice from a range of 0.15 to 0.40 (before) to a mean of 0.990 (after).Minor: PyPI metadata
picsl-greedy'sHomepageon PyPI points atpyushkevich/greedyrather than this repo, which is what initially sent me to open this against the wrong repository. Might be worth pointing it here.AI Disclosure
We used claude code in the development process.