Skip to content

Fix stardist worker crash: pin setuptools<82 for pkg_resources#144

Merged
arjunrajlab merged 4 commits into
masterfrom
fix/stardist-pkg-resources-setuptools
Jun 23, 2026
Merged

Fix stardist worker crash: pin setuptools<82 for pkg_resources#144
arjunrajlab merged 4 commits into
masterfrom
fix/stardist-pkg-resources-setuptools

Conversation

@arjunrajlab

@arjunrajlab arjunrajlab commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

Problem

The stardist worker crashed at runtime in the deployment build with:

File "/root/miniforge3/envs/worker/lib/python3.10/site-packages/stardist/bioimageio_utils.py", line 2, in <module>
    from pkg_resources import get_distribution
ModuleNotFoundError: No module named 'pkg_resources'

Root cause

stardist==0.9.1 imports from pkg_resources import get_distribution at import timestardist/__init__.py pulls in bioimageio_utils.py, which fails immediately. pkg_resources was bundled with setuptools, but setuptools 82.0.0 (released Feb 2026) removed it entirely.

workers/annotations/stardist/environment.yml did not pin setuptools, so fresh builds resolved setuptools ≥82 and shipped an image without pkg_resources. The image used to build/run fine only because earlier builds happened to get setuptools <82. This is unpinned-transitive-dependency drift.

Why the build didn't catch it

download_models.py runs at build time and does from stardist.models import StarDist2D, so the build touched the broken import — but it wrapped the import in except ImportError / except Exception that only printed the traceback and then exited 0. Docker saw a successful RUN, so the image built and pushed "successfully" (with the models never cached), and the failure only surfaced at runtime.

Fixes

  1. Pin setuptools<82 in the conda dependencies so pkg_resources stays available. This is the layer where setuptools is installed; none of the later pip steps require setuptools ≥82, so the pin holds through the build.
  2. Make download_models.py exit non-zero on import/download failure, so future dependency drift fails the Docker build loudly instead of shipping a broken image.

Tests

Added workers/annotations/stardist/tests/test_download_models.py (stdlib-only, runs under plain pytest). It runs the script with a fake stardist package shadowing any real install so the import fails deterministically, and asserts the script exits non-zero and reports the error. Written test-first (failed against the old swallow-and-exit-0 behavior, passes after the fix).

$ pytest workers/annotations/stardist/tests -q
2 passed

Verification

The setuptools<82 pin itself can't be validated by a local rebuild here (GPU/CUDA worker, not buildable on a Mac dev machine). After the next deployment build, confirm the worker imports cleanly:

docker run --rm --entrypoint conda <image> run -n worker \
  python -c "import pkg_resources, stardist; print('ok', stardist.__version__)"

Sources for the setuptools 82.0.0 pkg_resources removal:

🤖 Generated with Claude Code

arjunrajlab and others added 4 commits June 19, 2026 21:37
stardist==0.9.1 imports `from pkg_resources import get_distribution` at
import time (via stardist/__init__.py -> bioimageio_utils.py). setuptools
82.0.0 (released Feb 2026) removed the bundled `pkg_resources` module, so
fresh deployment builds resolved setuptools>=82 and the container crashed
on `import stardist` with:

    ModuleNotFoundError: No module named 'pkg_resources'

Pin setuptools<82 in the conda environment so pkg_resources stays
available. This is the layer where setuptools is installed; later pip
steps don't require >=82, so the pin holds through the build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Regenerated by the update-worker-docs Claude Code hook after PR creation.
download_models.py caught ImportError (and any Exception), printed a
traceback, and then exited 0. This swallowed dependency breakage at build
time: the setuptools/pkg_resources failure printed in the build logs but
the image built "successfully" and only crashed at runtime on
`import stardist`, with the models never even cached.

Add sys.exit(1) to both handlers so the Docker `RUN python /download_models.py`
step fails loudly when stardist can't be imported or the models can't be
downloaded.

Add tests (runnable with plain pytest, stdlib only) that run the script
with a fake `stardist` package shadowing any real install so the import
fails deterministically, and assert the script exits non-zero and reports
the error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Regenerated by the update-worker-docs Claude Code hook after PR creation.
@arjunrajlab arjunrajlab merged commit 8d1ddeb into master Jun 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant