ci: install test dependencies from uv.lock instead of resolving fresh

[mattmillerai]

ELI-5

Our test CI used to download the newest version of every library each time it ran, ignoring the exact versions written down in uv.lock that we use on our own machines. So the day a library published a new release, CI broke on every open pull request at once — even PRs that had nothing to do with it. This makes CI read the list of exact versions instead, so it breaks only when we choose to upgrade.

What happened

On 2026-07-17, tomlkit 0.15.1 shipped and 9 tests started failing with ValueError: Comment cannot contain line breaks on PRs that touched none of the relevant code. uv.lock pins tomlkit==0.13.3, but pip install pytest ... + pip install -e . re-resolves transitive dependencies from scratch, so CI never saw that pin.

What changed

pytest.yml and build-and-test.yml now install with uv sync --extra dev --locked and run tests via uv run --locked --extra dev. Everything else — the tests run, coverage flags, Codecov upload, the 3-OS matrix, PYTHONPATH, env gates — is unchanged.

The repo already had the inputs: uv.lock and optional-dependencies.dev. ruff_check.yml already pins ruff==0.15.15, so this brings the test workflows in line with the instinct the lint workflow already had.

Verification

On real CI, this PR's build job is green — and the log shows it doing the thing the ticket asked for:

Install uv            python-version: 3.10
Install dependencies  Installed 69 packages
Install dependencies   + tomlkit==0.13.3        <-- the locked pin, not the 0.15.1 that broke CI
Run tests             2574 passed, 36 skipped in 171.42s

All checks green: build, ruff_check, codecov/project (so the coverage upload still works), CodeQL, Socket Security.

Locally, that CI now genuinely tracks the lockfile (the acceptance criterion) — against a uv sync --locked venv on Python 3.10.20:

• uv lock --upgrade-package tomlkit → 0.15.1 → exactly the 9 failures named in the report (8 in test_config_parser.py, plus test_node_init_strips_credentials).
• Reverting to the locked 0.13.3 → 82 passed. Full suite: 2573 passed, 37 skipped.

Judgment calls

1. --locked, not --frozen. The plan specified --frozen to "fail loudly if uv.lock drifts from pyproject.toml." It does not — --frozen uses the lockfile as-is but exits 0 on drift. Verified against the real drift below:

flag	drifted lock	in-sync lock
--frozen	exit 0 — drift ignored silently	installs pinned deps
--locked	exit 1 — "lockfile needs to be updated"	installs pinned deps

Both stop the fresh re-resolution that caused the outage, but only --locked delivers the second stated goal. Used --locked to match the plan's intent over its literal text.

2. uv.lock is relocked here (285 additive lines). --locked immediately failed on main: the bench extra (anthropic>=0.40) was added to pyproject.toml in #490 without a lock update. Without fixing that first, --locked would fail on day one. This is itself evidence for the ticket — the drift went unnoticed because nothing checked. The relock is purely additive: it adds anthropic + 6 transitive deps and changes no existing pinned version (verified by diffing every name/version pair; tomlkit stays 0.13.3).

3. setup-uv pinned by commit SHA. The plan said @v5; that resolves but is well behind (current is v8.3.2). @v8 does not resolve — setup-uv publishes floating major tags only through v7, which failed CI outright. Rather than fall back to a mutable tag, it's pinned to the v8.3.2 SHA with the version in a trailing comment — matching how this repo already pins its other third-party action (codecov-action), and fitting for a workflow whose whole purpose is reproducibility: a mutable tag is the same class of problem as an unpinned dependency.

4. --extra dev repeated on uv run. uv run re-syncs the environment, and today's uv leaves the dev extra in place (verified) — but that's a default, not a guarantee. Naming the extra on both commands states the requirement explicitly, so a future uv default can't silently strip pytest/pytest-cov/jsonschema out from under the test step.

5. actions/setup-python dropped in favor of setup-uv's python-version. Keeps one tool owning the interpreter while preserving the pinned 3.10 (and the matrix input in build-and-test.yml), along with the "min version in pyproject.toml" comment.

Known verification gap — worth a reviewer's eye

build-and-test.yml does not run on this PR. Its paths: filter matches only comfy_cli/** and tests/e2e/** and explicitly excludes .github/** — so a PR that only touches workflows and uv.lock can't exercise it. My changes to it are therefore verified locally but not on real CI, and it will first run for real on the next PR that touches comfy_cli/** (across Linux/Windows/macOS).

What I could check locally (macOS): uv sync --extra dev --locked succeeds; the comfy entrypoint resolves under uv run (the e2e tests shell out to it); the CUDA-detect step skips cleanly with no GPU (5 skipped); the e2e (19) and torch-backend (5) suites collect. Their full runs need CI's network/GPU. If you'd rather not take the first-run risk, the alternative is to drop the build-and-test.yml hunk and do it in a PR that also touches comfy_cli/** — happy to split it.

Scope / follow-ups (not done here)

• Three more workflows share the unpinned pattern and are unchanged: test-mac.yml, test-windows.yml, run-on-gpu.yml. run-on-gpu.yml runs on self-hosted GPU runners, where introducing uv deserves its own verified change rather than a blind sweep.
• uv.lock pins ruff==0.12.7 while ruff_check.yml pins ruff==0.15.15 — a real divergence (the older ruff flags 14 UP038 errors the pinned one doesn't). Neither workflow here runs ruff, so this PR leaves it alone rather than bundling a lint-behavior change.

Trade-off worth knowing

--locked means a pyproject.toml change without a matching uv lock now fails CI. That's the intended trade: the failure is loud, attributable to the PR that caused it, and fixed by running uv lock. The alternative — a stale lock silently missing a newly added dependency — surfaces as a confusing ImportError instead.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c40d99f9-4afd-4bb6-b96d-27874b641083

📥 Commits

Reviewing files that changed from the base of the PR and between a732f5c and ab6c412.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (2)

• .github/workflows/build-and-test.yml
• .github/workflows/pytest.yml

---

📝 Walkthrough

Walkthrough

Both GitHub Actions test workflows now use uv to install locked development dependencies and execute existing pytest commands, while preserving environment variables, test targets, and coverage reporting.

Changes

CI uv migration

Layer / File(s)	Summary
Locked uv environment and test execution .github/workflows/build-and-test.yml, .github/workflows/pytest.yml	CI installs uv, runs uv sync --locked --extra dev, and invokes the existing CUDA, end-to-end, torch backend, and coverage tests through uv run.

Suggested reviewers: robinjhuang

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch matt/be-3292-pin-test-deps

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch matt/be-3292-pin-test-deps

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 2 finding(s).

Severity	Count
🟡 Medium	2

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)

[github-actions]

🟡 Medium — astral-sh/setup-uv@​v8 is pinned to a mutable major-version tag rather than a full commit SHA. This job exposes secrets.CODECOV_TOKEN, so a retargeted or compromised upstream tag could run attacker-controlled code and exfiltrate that token; note codecov/codecov-action is already SHA-pinned in this same file, making this an inconsistent supply-chain gap. Pin the action to a full commit SHA. Raised by 2 of 8 reviewers (gpt-5.3-codex-xhigh adversarial, claude-opus-4-8-thinking-xhigh adversarial).

[github-actions]

🟡 Medium — astral-sh/setup-uv@​v8 is referenced by a mutable tag instead of a pinned commit SHA, so a retagged or compromised upstream release would execute arbitrary code in CI with the job token and repository access. Pin the action to a full commit SHA (as codecov/codecov-action already is in pytest.yml). Raised by 2 of 8 reviewers (gpt-5.3-codex-xhigh adversarial, claude-opus-4-8-thinking-xhigh adversarial).