Pin providers in constraints to the versions published in PyPI - #71324
Merged
Conversation
Constraints cut for a release candidate had to land on the wave being voted on, whose providers exist in PyPI only as rc versions. Asking uv for a pre-release strategy left it free to answer with any version satisfying the lower bounds, so the pins were neither the candidate nor the last release, and what a candidate shipped depended on how the resolution happened to go. Retrieving the versions from PyPI and naming them leaves nothing to resolve: the constraints pin what is actually published. A pre-release is only ever considered when the run allows it, and even then it has to sort above every final release, so a provider without a candidate in the wave keeps its release and a released constraints file can never carry an rc pin. A candidate is exempt from the downgrade check - it sorts below the releases the constraints branch already carries, so the comparison says nothing there. Re-running the workflow for the same candidate now replaces that candidate's branch and tag instead of pushing onto them, so the two always describe the run that produced them and a wave can be re-cut.
potiuk
requested review from
amoghrajesh,
ashb,
bugraoz93,
choo121600,
ephraimbuddy,
gopidesupavan,
jason810496,
jedcunningham,
jscheffl and
vatsrahul1001
as code owners
August 8, 2026 08:09
eladkal
approved these changes
Aug 8, 2026
vatsrahul1001
approved these changes
Aug 8, 2026
Contributor
Backport failed to create: v3-3-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker f9b72d8 v3-3-testThis should apply the commit to the v3-3-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
potiuk
added a commit
that referenced
this pull request
Aug 9, 2026
… (#71340) * Pin providers in constraints to the versions published in PyPI Constraints cut for a release candidate had to land on the wave being voted on, whose providers exist in PyPI only as rc versions. Asking uv for a pre-release strategy left it free to answer with any version satisfying the lower bounds, so the pins were neither the candidate nor the last release, and what a candidate shipped depended on how the resolution happened to go. Retrieving the versions from PyPI and naming them leaves nothing to resolve: the constraints pin what is actually published. A pre-release is only ever considered when the run allows it, and even then it has to sort above every final release, so a provider without a candidate in the wave keeps its release and a released constraints file can never carry an rc pin. A candidate is exempt from the downgrade check - it sorts below the releases the constraints branch already carries, so the comparison says nothing there. Re-running the workflow for the same candidate now replaces that candidate's branch and tag instead of pushing onto them, so the two always describe the run that produced them and a wave can be re-cut. * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI (cherry picked from commit f9b72d8)
1 task
potiuk
added a commit
to potiuk/airflow
that referenced
this pull request
Aug 11, 2026
An unbounded requirement lets the resolver answer with any version that has ever been published, so what our constraints pin - and what a user ends up installing - depends on how the resolution went rather than on what the code needs. Four such requirements were found and fixed by hand while pinning providers in constraints (apache#71324); they sat in four different dependency tables, which is why they went unnoticed for so long. Distributions that are members of the uv workspace are exempt - they resolve from the checkout, so a version range would say nothing - as are direct URL requirements, where the URL already names the exact artifact.
potiuk
added a commit
that referenced
this pull request
Aug 11, 2026
An unbounded requirement lets the resolver answer with any version that has ever been published, so what our constraints pin - and what a user ends up installing - depends on how the resolution went rather than on what the code needs. Four such requirements were found and fixed by hand while pinning providers in constraints (#71324); they sat in four different dependency tables, which is why they went unnoticed for so long. Distributions that are members of the uv workspace are exempt - they resolve from the checkout, so a version range would say nothing - as are direct URL requirements, where the URL already names the exact artifact.
potiuk
added a commit
to jason810496/airflow
that referenced
this pull request
Aug 13, 2026
…e#71324) (apache#71340) * Pin providers in constraints to the versions published in PyPI Constraints cut for a release candidate had to land on the wave being voted on, whose providers exist in PyPI only as rc versions. Asking uv for a pre-release strategy left it free to answer with any version satisfying the lower bounds, so the pins were neither the candidate nor the last release, and what a candidate shipped depended on how the resolution happened to go. Retrieving the versions from PyPI and naming them leaves nothing to resolve: the constraints pin what is actually published. A pre-release is only ever considered when the run allows it, and even then it has to sort above every final release, so a provider without a candidate in the wave keeps its release and a released constraints file can never carry an rc pin. A candidate is exempt from the downgrade check - it sorts below the releases the constraints branch already carries, so the comparison says nothing there. Re-running the workflow for the same candidate now replaces that candidate's branch and tag instead of pushing onto them, so the two always describe the run that produced them and a wave can be re-cut. * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI * fixup! Pin providers in constraints to the versions published in PyPI (cherry picked from commit f9b72d8)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Constraints cut for a release candidate have to land on the wave being voted on, whose providers exist in PyPI only as rc versions. Asking
uvfor a pre-release strategy (--prerelease explicitplus a>=0.0.0rc0lower bound per provider) left it free to answer with any version satisfying those bounds, so the pins were neither the candidate nor the last release.Now the versions are retrieved from PyPI and named, so there is nothing left to resolve — the constraints pin what is actually published:
==<version>at the newest version PyPI can actually install for the Python being resolved (fully yanked releases, releases with no files, and releases excluded byrequires_pythonare passed over);--allow-pre-releasesis set, and even then it has to sort above every final release — so a provider without a candidate in the wave keeps its release, and a released constraints file can never carry an rc pin;--prerelease explicitand the>=0.0.0rc0bounds are gone.Checked against live PyPI while the current wave is being voted on:
A release candidate is also exempt from the provider downgrade check: it pins the wave being voted on, which sorts below the releases the constraints branch already carries, so the comparison says nothing there. The check still runs for finals, where it now catches the pinned version disappearing (yanked, or no longer offering a file for that Python).
Finally, re-running
release-constraints.ymlfor the same candidate now deletes and re-creates that candidate's branch and tag instead of pushing onto them — otherwise the second run hits a non-fast-forward push and a tag that already exists. Finals are untouched: they still commit onto the sharedconstraints-X-Y.One consequence worth calling out: with
==pins on the regular run too, two providers whose newest releases need incompatible dependencies now fail generation outright instead of quietly settling on an older provider and tripping the downgrade check. The install-failure message says so and points at theadditional_constraints_for_highest_resolutionescape hatch.Minimum versions added for five unbounded dependencies
The lowest-direct-resolution test suite ("Low dep tests") resolves direct dependencies to the oldest version their specifier permits. A dependency with no lower bound therefore drops all the way to its first-ever release, which for a compiled package means building a decade-old sdist against a current toolchain:
uvwarns about exactly this. Each of these now carries the floor it is actually developed and tested against:providers/samba(kerberosextra)krb5>=0.8.0providers/akeyless(cloud_idextra)akeyless-cloud-id>=0.3.0providers/apache/spark(dev group)pyspark>=4.0.0providers/common/ai(dev group)pydantic-ai-slim[mcp]>=2.0.0dev/registry(dev group)pytest>=9.1.1The akeyless requirement is also spelled with the normalised distribution name (
akeyless-cloud-idrather thanakeyless_cloud_id). The two extras are user-facing, so the provider docs tables were regenerated to match.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines