feat: pip-compile -> uv migration - #38915
Open
irfanuddinahmad wants to merge 58 commits into
Open
Conversation
openedx-webhooks
added
open-source-contribution
|
Thanks for the pull request, @irfanuddinahmad! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
irfanuddinahmad
changed the title
This was referenced Jul 22, 2026
salman2013
reviewed
Jul 24, 2026
Contributor
Author
|
Re: the Good catch, and worth noting: the org already ran a dedicated SHA-pinning sweep for this exact reason (openedx/.github#165, prompted by the tj-actions/changed-files supply-chain incident), but openedx-platform wasn't part of that ~121-repo effort. That said, I don't think pinning just |
…ration 1/5) Populates [project.dependencies] (from kernel.in + bundled.in), adds [project.optional-dependencies] for the legacy openstack storage backend, adds PEP 735 [dependency-groups] (coverage/testing/doc/assets/development/ semgrep/ci/dev, mirroring the current .in file composition), and [tool.edx_lint].uv_constraints + generated [tool.uv].constraint-dependencies for the ~20 repo-specific version pins, with a committed uv.lock. This is purely additive: the Makefile, tox.ini, and CI workflows are untouched and continue to use pip-compile/requirements/*.txt as the source of truth. Part of the pip-compile -> uv migration tracked in openedx/public-engineering#543 (1 of 5 PRs). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ions Found via real CI runs: a fresh uv resolution picked social-auth-core 5.0.2 (previously locked at 4.9.1 via pip-compile), which changes the OAuth pipeline's post-login redirect behavior and breaks common/djangoapps/third_party_auth's integration test suite (AzureAD, Google, LinkedIn, Twitter full-pipeline specs all failed the same assertion in tests/specs/base.py's assert_logged_in_cookie_redirect). This migration is meant to be a tooling swap, not a dependency upgrade, so pin back to the 4.x line rather than bundle an investigation into social-auth-core 5.x's behavior change into this PR. Mirrors the existing social-auth-app-django<=5.4.1 constraint, pinned for a related, already-deferred migration in this same dependency family. Follow-up tracked at #38841. Verified: all 46 previously-failing third_party_auth tests pass with social-auth-core==4.9.1 restored via this constraint. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rewrites the Makefile's requirements targets, tox.ini, and ~13 CI workflows to use uv instead of pip-compile/pip-sync for the main app. Deletes requirements/edx/*.in and *.txt (superseded by pyproject.toml + uv.lock, added in PR 1 / #38835). requirements/constraints.txt, common_constraints.txt, and pip-tools.{in,txt} are intentionally kept for now: requirements/edx-sandbox and scripts/* still pip-compile against them and aren't migrated until PR 3/4. requirements/edx/{base,assets,development}.txt are regenerated as `uv export` compatibility artifacts (via the Makefile's compile-requirements target) since external tooling -- notably tutor's Dockerfile -- installs from those exact paths with plain pip, not uv. check_python_dependencies.yml is disabled (workflow_dispatch only, job gated with if: false) since find_python_dependencies can't scan pyproject.toml yet; tracked at openedx/repo-tools#725. User-confirmed before committing since this removes a CI safety net. Part of openedx/public-engineering#543 (2 of 5). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found via real CI runs after opening the PR (unit tests, Quality Others, and the ReadTheDocs build all failed): 1. Makefile's test-requirements used `uv sync --only-group testing`, which EXCLUDES [project.dependencies] entirely (confirmed: `--only-group` replaces the dependency set rather than adding to it, unlike `--group`). This meant Django, XBlock, and the rest of the actual application were never installed for test runs -- ModuleNotFoundError: No module named 'xblock' on every unit test shard. Fixed to `--no-default-groups --group testing`, matching what tox-uv itself generates for the equivalent tox environment. 2. .readthedocs.yaml had the same bug in its doc-requirements.txt export (`--only-group doc`). Since [project.dependencies] were excluded from that export, the subsequent `pip install -e .` resolved the whole dependency tree completely unconstrained by uv.lock's [tool.uv].constraint-dependencies -- picking setuptools==82.0.1 (violates setuptools<82) and Django==6.0.6 (violates Django<6.0). The setuptools violation broke fs/pyfilesystem2's pkg_resources import, crashing the Sphinx build via Django app loading. Fixed to `--group doc` plus `--no-deps` on the `pip install -e .` step, so dependencies only ever come from the properly-constrained export. 3. scripts/xsslint_config.py's SKIP_DIRS didn't exclude .venv. Under the old pip-compile system, dependencies installed into the system Python outside the repo checkout, so this never mattered. Now that `uv sync` creates .venv/ inside the checkout, xsslint's directory walk (which defaults to scanning the whole cwd) swept up thousands of vendored third-party files, inflating violations from 64 (the accepted baseline) to 316. Verified all three: real pytest run (59 passed) plus full Django `manage.py check` for both LMS and CMS against the corrected test-requirements; a local simulation of the RTD build job sequence confirms Django==5.2.15/setuptools==81.0.0 (both constraint-compliant) and a successful Sphinx build. Audited every other `--only-group` usage introduced in this migration (assets.txt compat export, semgrep.yml) -- both are intentionally project-dependency-free, matching the original files' documented behavior, and their CI checks already passed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
requirements/edx/{base,development}.txt were regenerated from the
uv.lock that predated PR 1's social-auth-core<5.0.0 constraint, so
they still referenced social-auth-core==5.0.2 -- caught by
check-consistent-dependencies.yml's re-run of `make compile-requirements`.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Gives requirements/edx-sandbox/ its own standalone pyproject.toml + uv.lock, independent of the main app's dependency graph (codejail intentionally runs untrusted code in a separate, isolated venv). [tool.edx_lint].uv_constraints holds only the subset of the root constraints relevant to this environment's deps (numpy, lxml, setuptools) -- uv/edx-lint have no cross-project constraint chaining equivalent to pip-compile's "-c ../constraints.txt", so root and sandbox constraints are now independently maintained (documented in requirements/edx-sandbox/README.rst). base.txt is regenerated as a `uv export` compatibility artifact (the README documents it as a supported, if unstable, direct pip-install target). releases/*.txt are untouched -- they're frozen historical snapshots, not part of any active compile loop; README now documents cutting future ones via `uv export` instead of pip-compile. Part of openedx/public-engineering#543 (3 of 5). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…on 4/5)
Gives scripts/xblock, scripts/user_retirement, and scripts/structures_pruning
each their own standalone pyproject.toml + uv.lock, mirroring the
codejail sandbox pattern from PR 3. structures_pruning's local
[tool.edx_lint].uv_constraints keeps the pymongo<4.4.1 pin it inherited
via the old "-c ../../../requirements/constraints.txt" chain.
These scripts are documented (in their own READMEs) to support git
sparse-checkout usage -- cloning only e.g. scripts/user_retirement/
without the rest of edx-platform. A self-contained pyproject.toml is
actually an improvement here over the old relative "-c
../../../requirements/constraints.txt" reference, which wouldn't even
resolve in a sparse checkout that excludes the root requirements/ dir.
Compatibility .txt exports are kept at their previously-documented
paths (e.g. scripts/user_retirement/requirements/{base,testing}.txt)
since each script's own README explicitly instructs `pip install -r`
against those exact paths.
Also fixes check-consistent-dependencies.yml's path filter and the two
PR-creating workflows' add-paths, neither of which would have picked
up changes to the new scripts/*/pyproject.toml or uv.lock files.
Part of openedx/public-engineering#543 (4 of 5).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Deletes requirements/constraints.txt, common_constraints.txt, and
pip-tools.{in,txt} -- these were kept alive through PR 2-4 because
requirements/edx-sandbox and scripts/* still pip-compiled against
them, but PR 4 was the last consumer, so they're now fully unused.
Removes the correspondingly-vestigial Makefile machinery: the
pre-requirements target, the COMMON_CONSTRAINTS_TXT curl-fetch-and-sed
target, and the CUSTOM_COMPILE_COMMAND/COMPILE_OPTS variables that only
existed to feed pip-compile invocations which no longer exist anywhere
in this repo.
Finalizes requirements/README.rst for the fully-migrated state and
fixes a couple of remaining stale references (constraints.txt ->
[tool.edx_lint].uv_constraints).
This is the last of 5 PRs migrating openedx-platform from pip-compile
to uv + PEP 621/735 pyproject.toml, tracked in
openedx/public-engineering#543. Two follow-up
items remain outside this repo's control:
- openedx/repo-tools#725: find_python_dependencies needs pyproject.toml
support before check_python_dependencies.yml can be re-enabled.
- Tutor's Dockerfile installs from requirements/edx/{base,assets,development}.txt
with plain pip; those are kept as `uv export` compatibility artifacts
(see PR 2 / #38836) rather than deleted, so no action is required there,
but tutor maintainers should be aware these are now generated files.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI's "Compile requirements" check re-runs `make compile-requirements` and fails if it produces any diff, to catch exactly this kind of inconsistency. The compat-export files in this PR were originally generated with uv 0.11.26; a newer uv (0.11.30, matching what astral-sh/setup-uv installs in CI) resolves grpcio/grpcio-status with an explicit `; platform_python_implementation != 'PyPy'` marker that 0.11.26 omitted. Regenerated with uv 0.11.30 to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The uv migration for scripts/user_retirement dropped the lxml pin that requirements/constraints.txt previously carried, letting the lockfile resolve to lxml 6.1.1. The pin exists to avoid a libxml2 version mismatch at runtime (#36695), and this script transitively depends on lxml via simple-salesforce -> zeep, so the same pin applies here as it does at the repo root.
irfanuddinahmad
changed the title
Resolves conflicts in the generated requirements/edx/*.txt compatibility
exports, per master's 16 new commits since this branch's merge-base:
- requirements/edx/{doc,testing}.txt: modify/delete conflicts (this
migration already deleted both; master's still-pip-compile-based CI kept
regenerating them). Kept deleted, per this migration's actual intent
(matches the documented resolution pattern for this exact case).
- requirements/edx/{base,development}.txt: content conflicts (both sides
regenerated the same file via different toolchains). Regenerated fresh
via `make compile-requirements` against the merged pyproject.toml/uv.lock
rather than hand-merging generated output.
Also bumped openedx-core 1.1.0 -> 1.2.0 in uv.lock to match master's own
bump (feat: remove system-defined taxonomies, CONTENT_TAGGING_AUTO #38895) --
that PR's taxonomy-removal changes in openedx/core/djangoapps/content_tagging/
need the newer openedx-core, and merging its code without also bumping the
dependency would have reproduced the same class of bug as the
openedx-filters/CourseModePriceRequested ImportError from earlier in this
PR's history. Verified the constraint (openedx-core<2) already permits
1.2.0, so this is a straightforward `uv lock --upgrade-package` matching
what master already needed for this code to run.
No actual application-code conflicts -- only the generated compatibility
files conflicted; everything else auto-merged cleanly.
Contributor
Author
@feanil I have reviewed the PR. Please start your review once @salman2013 and @farhan have approved. Thanks. |
added 6 commits
August 11, 2026 21:45
…migration-consolidated
semgrep's dependency chain (via wcmatch) is incompatible with other root-project dependencies (e.g. openedx-authz's pycasbin pin) when resolved together in one shared graph -- master avoided this by compiling semgrep.txt as a fully separate pip-compile run, but our migration merged it into the root project's shared dependency-groups without the same isolation, only django42/django52 got that treatment. Mirrors the existing requirements/edx-sandbox and scripts/xblock pattern: a standalone pyproject.toml + uv.lock, independent of the root project's graph. Unblocks matching openedx-authz/edx-submissions to their current master versions in the next commit.
The original uv lock resolved fresh, silently bundling months of accumulated-but-unreviewed dependency upgrades into what was supposed to be a pure tooling migration -- e.g. openedx-authz 1.20.0 -> master's 1.21.1, edx-submissions 4.0.0 -> 4.0.2, plus ~80 third-party packages. Re-locked by temporarily constraining each package to master's current requirements/edx/*.txt pin, then removing the constraint once resolved (uv lock keeps an already-valid resolution stable, so the pin doesn't need to persist in pyproject.toml). Left three packages unconstrained where forcing master's pin would break the django42 test-matrix fork (django, django-filter, django-simple-history all have a django>=5.2 floor incompatible with django42's django>=4.2,<4.3), since master never had to resolve against an older Django at all.
The inline shell (heredoc-style echo blocks, nested for-loops, escaped line continuations) was cluttering the Makefile the same way the old team-constraints merge logic did before scripts/merge_team_constraints.py extracted it. Verified byte-identical output against the previous inline version for every generated file before switching over. Note: invoke as a plain `python3` script, not via `uv run --no-project` -- wrapping it in an outer uv-managed ephemeral environment caused the script's own nested `uv export` calls to silently drop environment markers (e.g. grpcio's `platform_python_implementation != 'PyPy'`) that a directly-invoked `uv export` preserves correctly.
Per salman2013's review: npm run build-dev/build shells out to scripts/compile_sass.py bare via package.json, resolved through its own #!/usr/bin/env python shebang -- but that doesn't actually require manually splicing the venv's bin/ onto $GITHUB_PATH for the whole job. uv run prepends the venv's bin/ to PATH for the wrapped process and all its descendants, so wrapping the outer npm invocation is sufficient. Verified the PATH-inheritance mechanism directly (nested subprocess shebang resolution through uv run) before applying this.
…migration-consolidated # Conflicts: # requirements/constraints.txt # requirements/edx/base.txt # requirements/edx/development.txt # requirements/edx/doc.txt # requirements/edx/testing.txt
Contributor
Author
|
Verified the manual workflow testing:
|
Contributor
Author
|
CI Testing Compared the two "Upgrade Requirements" runs (#38976 on master, #38974 on this branch) as a cross-check of the uv migration's dependency resolution:
|
This was referenced Aug 12, 2026
Contributor
|
@irfanuddinahmad Upgrade Workflow with master branch Upgrade Workflow with this branch |
added 4 commits
August 12, 2026 16:49
Both steps install an already uv.lock-resolved requirements file (and the local package itself) into RTD's own externally-managed virtualenv -- uv sync has no mode for installing into an arbitrary existing venv from a plain requirements.txt, so uv pip install is the uv-native tool for this, same as the one other confirmed legitimate uv pip usage in this migration (installing into an external consumer's environment). Doesn't change what gets installed or where -- doc-requirements.txt is already fully pinned by uv.lock via the uv export step above it, this only swaps which tool re-installs it.
…e to docs group uv sync --active reads directly from uv.lock/pyproject.toml and syncs into RTD's own already-active virtualenv -- no intermediate uv export step needed, and it installs the local project as an editable install automatically, replacing both of the previous two uv pip install steps with one command. While verifying this against a simulated RTD-style external venv, a real gap surfaced: the docs group never got the equivalent of requirements/edx/doc.in's old `-r base.txt` -- docs/conf.py imports cms.envs.common, whose add_optional_apps() needs bundled XBlocks (e.g. openassessment) importable, not just Sphinx tooling. The prior uv export --group docs command (no --no-default-groups) accidentally worked because it also pulled in the implicit default group, which happens to chain through to bundled -- fragile, and broken by adding --no-default-groups here. Fixed at the source instead: docs now directly includes the bundled group, matching doc.in's real composition regardless of default-groups.
uv sync --active failed on RTD's actual build runner (66s duration -- too fast to have gotten far), for a reason I couldn't confirm without access to RTD's detailed build log (only public build-status API, which doesn't include command output). Most likely --active isn't detecting an active venv there the way it did in a local simulation where I explicitly exported $VIRTUAL_ENV myself -- RTD's actual activation mechanism for post_install jobs may only prepend to PATH. Reverted to the confirmed-working uv export + uv pip install mechanism from before, but keeping the actually-necessary fix from that attempt: the previous uv export --group docs call (this repo's prior working config) had no --no-default-groups, so it only worked by accident via the implicit default-group chain reaching bundled. Now that docs directly includes bundled (previous commit), this export can correctly use --no-default-groups too.
…thedocs Root-caused (via uv sync --active -v) why this failed the first time it was tried: uv sync --active silently falls back to a project-local .venv when $VIRTUAL_ENV isn't set, rather than erroring -- RTD's post_install jobs apparently only prepend to $PATH, never needed to set $VIRTUAL_ENV since bare pip/python don't require it. Deriving it ourselves from `command -v python` (standard PEP 405 venv layout: interpreter always lives at <venv>/bin/python) removes the dependency on RTD having set it. Verified locally by reproducing the exact failure mode (PATH-only activation, no $VIRTUAL_ENV) and confirming this fixes it.
Contributor
Author
|
Tested this branch locally on Tutor 22.0.0 (Verawood, Python 3.12): built both |
farhan
requested changes
Aug 13, 2026
| "libsass", | ||
| "nodeenv", | ||
| ] | ||
| development = [ |
Contributor
There was a problem hiding this comment.
doc.in has been moved to docs altering the name not in unparity with master
Headup on it if it's under consideration
added 4 commits
August 13, 2026 16:02
Per farhan's review feedback.
- Remove uv-run wrapping of npm invocations; move uv into package.json's compile-sass scripts instead, since that's what actually needs the venv's Python for scripts/compile_sass.py's shebang. - Trim/remove several explanatory comments and docstrings added during the migration (check_python_dependencies.yml, quality-checks.yml, static-assets-check.yml, .readthedocs.yaml, scripts/compile_sass.py, requirements/README.rst, docs/references/static-assets.rst) that reviewers flagged as unnecessary. - Fix scripts/watch_sass.sh to point at the `development` dependency-group specifically, not the broader `default` umbrella group. - Add a clarifying note to README.rst about how [project.dependencies] maps to the old base.in's "kernel" packages.
…migration-consolidated # Conflicts: # requirements/constraints.txt
…ption Per feedback -- this belonged in the PR description, not committed docs.
farhan
reviewed
Aug 13, 2026
added 3 commits
August 13, 2026 19:17
…constraints Consolidate the team-governed edx-enterprise pin directly into [tool.edx_lint].uv_constraints instead of maintaining a separate CODEOWNERS-gated file + merge step. Removes requirements/team_constraints.txt, scripts/merge_team_constraints.py, the Makefile/CODEOWNERS/docs references to them, and simplifies scripts/update_uv_constraint_version.py to only edit uv_constraints.
…vious commit) Follow-up to 7ceae62 -- a failed git add silently dropped these from that commit. Removes the remaining Makefile/CODEOWNERS/workflow/docs references to requirements/team_constraints.txt and scripts/merge_team_constraints.py, and moves edx-enterprise's pin into [tool.edx_lint].uv_constraints.
feanil
reviewed
Aug 13, 2026
feanil
left a comment
feanil
left a comment
Contributor
There was a problem hiding this comment.
There's a lot of extra work happening because we're trying to keep the old requirements files around but I think we can drop a lot of that if we can do a fast-track DEPR before we merge this work. As long as operators know what to do instead of what they were doing, they can adapt with sufficient warning.
| if [[ "${{ matrix.django-version }}" == "pinned" ]]; then | ||
| make dev-requirements | ||
| else | ||
| uv sync --group default --group "django${{ matrix.django-version }}" --frozen |
Contributor
There was a problem hiding this comment.
shouldn't the django version group include the default group in it?
| if [[ "${{ matrix.django-version }}" == "pinned" ]]; then | ||
| make test-requirements | ||
| else | ||
| uv sync --no-default-groups --group testing --group "django${{ matrix.django-version }}" --frozen |
Contributor
There was a problem hiding this comment.
Why the --no-default-groups here? Don't we want that?
| os: "ubuntu-lts-latest" | ||
| tools: | ||
| python: "3.12" | ||
| jobs: |
Contributor
There was a problem hiding this comment.
https://about.readthedocs.com/blog/2026/04/uv-native-support/
Let's use the native support to do this.
Comment on lines
+67
to
+68
| local-requirements: ## no-op; kept for backwards compatibility -- uv sync handles this now | ||
| @true |
Contributor
There was a problem hiding this comment.
We should echo out a message to tell folks to stop using this so it's showing up in their logs. And decide when this can be removed.
| # edx-platform installs some Python projects from within the edx-platform repo itself. | ||
| pip install -e . | ||
| dev-requirements: ## install development environment requirements | ||
| uv sync --group default --frozen |
Contributor
There was a problem hiding this comment.
shouldn't this be the development group instead?
| pip-sync requirements/edx/development.txt $(wildcard requirements/edx/private.txt) | ||
| make local-requirements | ||
| base-requirements: ## install only production/runtime dependencies | ||
| uv sync --no-default-groups --group bundled --frozen |
Contributor
There was a problem hiding this comment.
This seems like it should be the default?
| - ``pip install -r requirements/edx/base.txt`` (production) | ||
| - ``pip install -r requirements/edx/development.txt`` (development) | ||
| - ``uv sync --no-default-groups --group bundled`` (production) | ||
| - ``uv sync --group default`` (development) |
Contributor
There was a problem hiding this comment.
Isn't default the default? Why add that here instead of just documenting this as uv sync?
Contributor
There was a problem hiding this comment.
I think we don't need these compatibility files, we just need to communicate the fact that they're going away and that people should use the make targets via a fast-track DEPR. That should reduce the complexity of this PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates edx-platform from pip-compile/pip-tools to
pyproject.toml+uv, tracked in openedx/public-engineering#543.Old file -> new equivalent
requirements/edx/kernel.in[project.dependencies]requirements/edx/bundled.in[dependency-groups].bundledrequirements/edx/testing.in[dependency-groups].testingrequirements/edx/doc.in[dependency-groups].docs(now also pulls inbundleddirectly --doc.indid-r base.txt, i.e. kernel + bundled, anddocs/conf.pyneeds bundled XBlocks likeopenedx-forum/ora2importable to build at all)requirements/edx/assets.in[dependency-groups].assetsrequirements/edx/development.in[dependency-groups].developmentrequirements/edx/coverage.in[dependency-groups].coveragerequirements/edx/semgrep.inscripts/semgrep/{pyproject.toml,uv.lock}-- see below, not a root dependency-grouprequirements/edx/github.in[project.dependencies]requirements/edx/openstack.txt[project.optional-dependencies].openstack(a PEP 621 extra, not a[dependency-groups]entry -- installable viapip install openedx-platform[openstack])requirements/constraints.txt+requirements/common_constraints.txt[tool.edx_lint].uv_constraints(hand-maintained) +[tool.uv].constraint-dependencies(machine-generated from it)requirements/pip-tools.{in,txt}requirements/edx/private.in/private.txtuv sync --inexact-- seerequirements/README.rstfor the exact workflow[dependency-groups].django42/django52: a real, independently-resolved Django-version test matrix, not a runtime patch on an already-resolved venvrequirements/edx/testing.in'stoxentry[dependency-groups].ci, alongside newtox-uv(needed fortox.ini'suv-venv-lock-runner) -- pulled into a bareuv syncviadefault's{include-group = "ci"}sotoxstays available locallyrequirements/edx/{base,assets,development}.txtand the 5 sub-projects' compat.txtfiles stay as machine-generateduv exportoutputs at their historical paths (for external tooling, e.g. Tutor's Dockerfile, that still installs from them directly) -- see the file-by-file breakdown below for exactly what each one exports.File-by-file summary
Root packaging & build config
[project.dependencies]list (previously just["setuptools"]), a new[dependency-groups]tree (coverage,bundled,testing,docs,assets,development,django42,django52,ci,default),[tool.uv].default-groups/constraint-dependencies(machine-managed byedx_lint write_uv_constraints),[tool.edx_lint].uv_constraints(the hand-maintained version pins, each with a dated comment and issue link, carried over from the oldconstraints.txt), and a new[tool.coverage.*]tree.django42/django52: a real, independently-resolved Django-version test matrix, not a runtime patch on an already-resolved venv --[tool.uv].conflictsdeclares the two mutually exclusive souv.lockcarries a genuinely separate resolution for each (confirmed:django42locks Django 4.2.30,django52/testinglocks 5.2.x, with real transitive-dependency divergence between the two, not just a Django-version override).upgrade-one-python-dependency.ymlworkflow uses to bump a version pin in[tool.edx_lint].uv_constraintsvia a propertomlkitTOML round-trip, instead ofsed-patching a text file..txt-compatibility-export logic for the root project and every sub-project, extracted out of the Makefile (previously ~70 lines of inline heredoc-style shell) into a plain Python script.[tool.coverage.*]in pyproject.toml.pre-requirements/pip-synctargets replaced withuv sync --group ...targets;compile-requirementsrewritten to runedx_lint write_uv_constraints->uv lockfor the root project (and the same lock step for each of the 5 uv-managed sub-projects), thenscripts/compile_requirements_exports.pyto re-export compatibility.txtfiles at the old paths for external tools (e.g. Tutor's Dockerfile) that stillpip install -r requirements/edx/base.txtdirectly.runner = uv-venv-lock-runner+dependency_groups = testing; droppedqualityfromenvlist(quality now runs via a dedicated CI workflow/Makefile target, not tox) and the now-redundantusedevelop/commands_pre = make test-requirements.pip install -r requirements/edx/*.txttouv sync --group ....CI workflows
All follow the same mechanical pattern: drop the manual
pip cachesteps in favor ofastral-sh/setup-uv's built-in cache (enable-cache: true), and replace every barepip/pylint/mypy/./manage.py/etc. invocation withuv run <tool>-- not a$GITHUB_PATHsplice. An earlier draft of this PR did splice.venv/binonto$GITHUB_PATHjob-wide instead of fixing individual invocations; that approach is gone everywhere now except one line instatic-assets-check.ymlthat's unrelated to Python (node_modules/.bin, needed by npm-installed JS tooling).check-consistent-dependencies.yml: trigger-detection regex extended to also watchpyproject.toml/uv.lock/sub-project files, not justrequirements/.check_python_dependencies.yml: re-enabled (was temporarily disabled earlier in this PR's history pendingedx-repo-tools[find_dependencies]gaining the ability to scanuv.lockinstead of.in/.txtfiles -- that upstream fix, openedx/repo-tools#735, has since merged). Now runs viauvx --from 'edx-repo-tools[find_dependencies]' find_python_dependencies --req-file uv.lock ..., no persistent install, noactions/setup-python.ci-static-analysis.yml,js-tests.yml,lint-imports.yml,migrations-check.yml,pylint-checks.yml,quality-checks.yml,semgrep.yml,unit-tests.yml: mechanical pip->uv swap as described above.compile-python-requirements.yml,upgrade-one-python-dependency.yml: updated to operate onpyproject.toml/uv.lockinstead of.in/.txt/constraints.txt. The dependency-downgrade script now edits[tool.edx_lint].uv_constraintsvia a propertomlkitTOML round-trip instead ofsed-patching a text file. Both manually triggered and verified against this branch:compile-python-requirementsranmake compile-requirements, detected no diff, and correctly exited without opening a PR;upgrade-one-python-dependency(package=django-filter) opened a PR with exactly the expecteduv.lock+ compat-export changes.static-assets-check.yml: same pip->uv swap, scoped the Python-deps-install step touv sync --no-default-groups --group bundled --group assets --frozen(rather than pulling in the fulldefaultgroup of dev/test/docs/ci tooling this job doesn't need), and wraps the twonpm run build-dev/npm run buildsteps inuv run(instead of a$GITHUB_PATHsplice) so thescripts/compile_sass.pyscript they shell out to -- resolved through its own#!/usr/bin/env pythonshebang -- finds the venv's Python onPATH.units-test-scripts-structures-pruning.yml,units-test-scripts-user-retirement.yml: intentionally untouched -- they stillpip install -r scripts/.../requirements/testing.txt, and those compatibility files continue to be regenerated (viauv export) at the same paths, so these two workflows keep working unmodified.Documentation
.in/.txtworkflow, including how to replace the droppedprivate.inmechanism withuv sync --inexact, and how the 5 standalone sub-project directories (requirements/edx-sandbox,scripts/xblock,scripts/user_retirement,scripts/structures_pruning,scripts/semgrep) each carry their own independentpyproject.toml+uv.lock.[dependency-groups]entry -- semgrep's own dependency chain (viawcmatch) is incompatible with other root-project dependencies (e.g.openedx-authz'spycasbinpin) when resolved together in one shared graph.master's oldpip-compile-based setup avoided this by compilingsemgrep.txtas a fully separate run; the rootuv.lockneeds the same isolation, which only a genuinely separate sub-project (likerequirements/edx-sandbox/scripts/xblockalready were) provides.uv syncequivalents, scoped to the specific dependency-groups each workflow step actually needs.python.installpip-requirements mechanism with a singleuv sync --activecall inpost_install, reading directly fromuv.lockrather than through an intermediate exportedrequirements.txt-- this also installs the local package itself as an editable install automatically, replacing what used to be a separate step.--activeneeds$VIRTUAL_ENVset to know which environment "active" refers to; RTD's build jobs only prepend to$PATH, sopost_installderives it explicitly (dirname "$(dirname "$(command -v python)")", matching the standard PEP 405 venv layout) rather than assuming RTD sets it.--group docs, not--only-group docs, so[project.dependencies](Django/XBlock/etc.) resolves againstuv.lock's pins instead of an unconstrainedsetuptools>=82that breaksfs/pyfilesystem2'spkg_resourcesimport;docsalso now directly includes thebundledgroup (matchingdoc.in's old-r base.txt), sincedocs/conf.pyimportscms.envs.common, which needs bundled XBlocks importable to succeed. Verified against Read the Docs' actual build (not just locally) before landing.Dependency files (requirements/, uv.lock, scripts/*)
requirements/edx/*.in(all deleted): superseded by[project.dependencies]/[dependency-groups]in pyproject.toml.requirements/edx/base.txt,assets.txt,development.txt(kept, regenerated): machine-generateduv exportcompatibility exports at their historical paths for external tooling that still installs from them directly.base.txtexports[project.dependencies]plus thebundledgroup --[project.dependencies](the oldbase.in's "kernel" packages) installs with everyuv syncregardless of which--groupyou pass, so--group bundledalone reproduces the oldbase.txt's kernel + bundled combination.requirements/edx/coverage.{in,txt},doc.{in,txt},testing.{in,txt},semgrep.{in,txt},bundled.in,github.in,kernel.in(all deleted): fully absorbed into pyproject.toml dependency-groups (or, forsemgrep, its own standalone sub-project); no external tooling installed from these paths directly, so no compatibility export was needed.requirements/edx/openstack.txt(deleted): absorbed into[project.optional-dependencies].openstack-- a PEP 621 extra, not a[dependency-groups]entry.requirements/edx/private.readme(deleted): documented theprivate.in/private.txtmechanism, which itself was never checked into git (git-ignored). No direct successor file -- see the mapping table above for theuv sync --inexactreplacement workflow.requirements/common_constraints.txt,requirements/constraints.txt(deleted): absorbed into[tool.uv].constraint-dependencies(machine-managed) and[tool.edx_lint].uv_constraints(hand-maintained) in pyproject.toml, with all original rationale comments and issue links preserved.requirements/pip-tools.{in,txt}(deleted): pip-tools itself is no longer needed.uv.lock,requirements/edx-sandbox/{pyproject.toml,uv.lock},scripts/{xblock,user_retirement,structures_pruning,semgrep}/{pyproject.toml,uv.lock}(new): machine-generated lockfiles for the root project and its 5 independent standalone sub-projects (codejail sandbox, XBlock scripts, user-retirement scripts, structures-pruning scripts, semgrep). Each sub-project sets[tool.uv] package = false(they're script bags, not installable packages) and carries only the subset of root constraints relevant to its own deps.scripts/*/requirements/{base,testing}.txt(kept, regenerated): same compatibility-export treatment asrequirements/edx/*.txt.Misc scripts
uv sync --no-default-groups --only-group assets --no-install-project), verified to actually compile Sass correctly with that reduced environment.pip install -r ...message updated touv sync --group default..venvto the linter's skip-dirs list.What's intentionally NOT done here (tracked externally)
requirements/edx/{base,assets,development}.txtwith plainpip. Those stay asuv exportcompatibility artifacts rather than being deleted, so no action is required on Tutor's side right now -- but Tutor maintainers should be aware these paths are now machine-generated, not hand-compiled. Verifiedtutor images build openedx-devsucceeds against this branch ontutor22.0.0/Verawood (Python 3.12); the previous named release's Python 3.11 base can't satisfy this migration'srequires-python = ">=3.12"at all, independent of anything in this PR.Verification
uv lockresolves cleanly (419 packages, root project); all 5 sub-projects (requirements/edx-sandbox,scripts/xblock,scripts/user_retirement,scripts/structures_pruning,scripts/semgrep) sync cleanly withuv sync --frozen.make compile-requirementsend-to-end for the root project and all 5 uv sub-projects -- regenerated compatibility export files are byte-identical to what's committed, confirming consistency.compile-python-requirementsandupgrade-one-python-dependency(bothworkflow_dispatch-only, so they don't run automatically on this PR) against this branch -- see the CI workflows section above.docs/readthedocs.orgagainst the real Read the Docs build andtutor images build openedx-devagainst a local Verawood (Python 3.12) environment.🤖 Generated with Claude Code