ci: install test deps from uv.lock so upstream releases can't break every PR (BE-3292)

[mattmillerai]

ELI-5

Our CI didn't use the versions of test tools we'd locked in uv.lock. Every
time it ran, it fetched the newest versions of everything from the internet.
So when some unrelated package shipped a new release, CI could suddenly go red
on every open PR at once — for reasons that had nothing to do with the PR.
This makes CI install exactly the locked versions instead, so a surprise
upstream release can't ambush everyone.

What & why

.github/workflows/pytest.yml (and build-and-test.yml) installed test deps
unpinned:

pip install pytest pytest-cov jsonschema
pip install -e .

pip install -e . re-resolves transitive deps fresh on every run, ignoring the
versions pinned in uv.lock. This isn't hypothetical — on 2026-07-17,
tomlkit 0.15.1 shipped and 9 tests began failing
(tests/comfy_cli/registry/test_config_parser.py ×8 +
test_node_init.py::test_node_init_strips_credentials) with
ValueError: Comment cannot contain line breaks, red-ing PRs whose diffs
touched none of that. The tomlkit source fix is handled separately (#533 /
BE-3285); this PR closes the CI-reproducibility gap that let it happen.

Change

Both workflows now install from the lockfile via uv:

• uv sync --extra dev --frozen — install the exact pinned versions.
• uv run --frozen --extra dev pytest ... — run against them.
• --frozen makes CI fail loudly if uv.lock drifts from
  pyproject.toml, instead of silently resolving something new.
• --extra dev supplies pytest / pytest-cov / jsonschema (the repo's
  existing optional-dependencies.dev group), replacing the ad-hoc pip install.
• Python stays pinned to the version the jobs already used (--python 3.10 for
  pytest.yml's min-version job; --python ${{ matrix.python-version }} for
  build-and-test.yml's matrix), so uv provides the interpreter and
  actions/setup-python is no longer needed.

Upgrades now become deliberate: bump uv.lock in its own PR where a break is
attributable and reviewable, instead of ambushing whoever opens the next PR.

Verification (run locally on this branch)

• uv sync --extra dev --frozen --python 3.10 succeeds — lockfile is in sync
  with pyproject.toml; installs tomlkit==0.13.3.
• uv run --frozen --extra dev --python 3.10 pytest --cov=comfy_cli --cov-report=xml . → 2573 passed, 37 skipped (coverage.xml written, so
  pytest-cov/jsonschema are present under --extra dev).
• Lockfile-tracking proof: re-running the affected tests with the pin
  overridden — uv run ... --with tomlkit==0.15.1 pytest <the 9 tests> →
  9 failed (exactly the ones from the incident); the default lockfile run
  (0.13.3) passes them. Confirms CI now tracks uv.lock.
• Both YAML files parse; the build-and-test.yml CUDA step runs under uv run
  and skips gracefully with no GPU.

Judgment calls / scope

• Scope = the two workflows the ticket named (pytest.yml,
  build-and-test.yml), both of which run the pytest suites via the unpinned
  pattern.

• Deliberately left as follow-ups (noted, not changed here):

  • run-on-gpu.yml — same pattern, but runs on a self-hosted GPU runner
    I can't verify locally; converting it unverified would be risky.
  • test-windows.yml / test-mac.yml — these are CLI integration smoke
    tests (comfy install + comfy launch in an explicit venv), not
    pytest-suite runs, so the tomlkit-class break doesn't hit them and the
    conversion is structurally different. (test-windows is also a known,
    separate infra flake.)
  • ruff_check.yml already pins its tool (ruff==0.15.15); publish_package.yml
    is release-time, not per-PR.

  A separate PR could extend the lockfile approach to these if desired.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 20 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 82d7bdb7-a929-4450-b434-0031477bc500

📥 Commits

Reviewing files that changed from the base of the PR and between a732f5c and 3ef7546.

📒 Files selected for processing (2)

• .github/workflows/build-and-test.yml
• .github/workflows/pytest.yml

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch matt/be-3292-pin-ci-test-deps

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch matt/be-3292-pin-ci-test-deps

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 2 finding(s).

Severity	Count
🟡 Medium	2

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)