From b855427d98526f8a774c2e6f6e18b78fb8609e09 Mon Sep 17 00:00:00 2001 From: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Date: Sun, 5 Jul 2026 22:43:17 +0300 Subject: [PATCH 1/2] Add framework for delegating providers release to non-PMC committers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The provider release cycle currently assumes a PMC member drives every step, yet ASF policy only reserves the binding vote and dist/release publication to the PMC. Documenting a clear split lets a non-PMC committer run the bulk of the process while the PMC signs, publishes to PyPI, and casts the binding votes — both spreading the release-manager load and giving prospective PMC members a supervised on-ramp. Generated-by: Claude Code (Opus 4.8 1M context) --- dev/README_RELEASE_PROVIDERS.md | 105 +++++++++++++++++++++++++++++++- 1 file changed, 104 insertions(+), 1 deletion(-) diff --git a/dev/README_RELEASE_PROVIDERS.md b/dev/README_RELEASE_PROVIDERS.md index 7f4c1ab0350b8..7bd40efc29587 100644 --- a/dev/README_RELEASE_PROVIDERS.md +++ b/dev/README_RELEASE_PROVIDERS.md @@ -23,6 +23,7 @@ - [Intro](#intro) - [What the provider distributions are](#what-the-provider-distributions-are) - [Decide when to release](#decide-when-to-release) + - [Delegating release duties to a non-PMC committer](#delegating-release-duties-to-a-non-pmc-committer) - [Collect ambiguities during the release (for a follow-up doc PR)](#collect-ambiguities-during-the-release-for-a-follow-up-doc-pr) - [Special procedures (done very infrequently)](#special-procedures-done-very-infrequently) - [Bump min Airflow version for providers](#bump-min-airflow-version-for-providers) @@ -87,6 +88,48 @@ a given provider needs to be released due to new features or due to bug fixes. package separately, but due to voting and release overhead we try to group releases of Provider distributions together. +## Delegating release duties to a non-PMC committer + +Per the [ASF release policy](http://www.apache.org/legal/release-policy.html), the Release Manager +does not need to be a PMC member, and there is no requirement that only a PMC member may call a +release vote. The policy's own wording: *"If the Release Manager is not a member of the PMC, they +will need to ask a PMC member to do the actual release publication"* — i.e. the one hard boundary +is write access to the `dist/release` SVN area and the binding vote itself; everything else can be +run by any committer. + +This means a non-PMC committer (the **Delegate** below) can run most of the provider release +process end to end, with a PMC member only stepping in for the parts ASF policy reserves to the +PMC. Split of duties: + +| Step | Owner | Notes | +|---|---|---| +| [Convert commits to changelog entries and bump provider versions](#convert-commits-to-changelog-entries-and-bump-provider-versions) | Delegate | Normal PR review/merge process, no PMC involvement needed. | +| [Build Provider distributions for SVN apache upload](#build-provider-distributions-for-svn-apache-upload) | Delegate | Builds are reproducible; anyone can build the unsigned artifacts. | +| [Build and sign](#build-and-sign-the-source-and-convenience-packages) + [commit to `dist/dev`](#commit-the-source-packages-to-apache-svn-repo) + [publish RC to PyPI](#publish-the-regular-distributions-to-pypi-release-candidates) | PMC | The three trust-sensitive steps are kept as **one contiguous PMC block** to avoid handing control back and forth: the PMC member re-builds reproducibly (or takes the Delegate's artifacts as-is), signs with their own key (already in the project's `KEYS` file), commits packages + signatures to `dist/dev`, and uploads the RC to the `apache-airflow-providers-*` PyPI namespace under the PMC's trusted publishing identity. The PMC then hands `files/packages.txt` (the PyPI URLs) back to the Delegate for the vote email. | +| [Push the RC tags](#push-the-rc-tags) | Delegate | Plain git tag push, no elevated access needed. | +| [Prepare documentation in Staging](#prepare-documentation-in-staging) | Delegate | | +| [Prepare issue in GitHub to keep status of testing](#prepare-issue-in-github-to-keep-status-of-testing) | Delegate | Delegate also tracks the issue, the vote thread, and related PRs throughout the release. | +| [Prepare voting email for Providers release candidate](#prepare-voting-email-for-providers-release-candidate) | Delegate | Delegate may send the `[VOTE]` email, but must **not** claim a personal binding `+1` (see note in that section) — only PMC votes are binding. | +| Casting the deciding vote(s) | PMC | At least 3 binding `+1` votes from PMC members are required for the release to pass; this cannot be delegated. | +| [Summarize the voting for the Apache Airflow release](#summarize-the-voting-for-the-apache-airflow-release) (`[RESULT][VOTE]`) | Delegate | Only after at least 3 binding PMC `+1` votes are already visible in the thread — the Delegate is reporting a result the PMC already reached, not deciding it. | +| [Publish release to SVN](#publish-release-to-svn) (`dist/release`) | PMC | Per [ASF Infra policy](https://infra.apache.org/release-publishing), `dist/release` write access is PMC-only by default (a project can request Infra to open it to all committers, but Airflow has not done so). | +| [Publish the packages to PyPI](#publish-the-packages-to-pypi) (final) | PMC | | +| [Add the final release tag in git](#add-the-final-release-tag-in-git) | Either | Not privileged; whoever is running that phase of the process does it. | +| [Publish documentation](#publish-documentation) (live) | Delegate | | +| [Update providers metadata](#update-providers-metadata) | Delegate | | +| [Notify developers of release](#notify-developers-of-release), security announcements, social media, committee report | PMC | Official project communications made under the PMC's authority. | +| [Close the testing status issue](#close-the-testing-status-issue) | Delegate | | + +The PMC continues to oversee the overall process regardless of how many steps are delegated, and +remains the party accountable for the release under ASF policy. + +> [!NOTE] +> Delegation is also a runway toward PMC membership. The first time a committer takes on the +> Delegate role, the overseeing PMC member is encouraged to walk them through the reserved block +> live — sharing their screen (or pairing) through the sign → `dist/dev` → PyPI-RC steps so the +> Delegate sees exactly how it is done. The aim is simply that these steps are familiar rather than +> a surprise if and when the Delegate later becomes a PMC member and runs them for real. + # Collect ambiguities during the release (for a follow-up doc PR) These instructions are imperfect. Every release uncovers at least one command @@ -513,6 +556,13 @@ key you want to use. ## Build and sign the source and convenience packages +> [!NOTE] +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this signing step +> begins the **PMC block** — the PMC member runs `sign.sh` with their own key (already in the +> project's `KEYS` file), then stays on through the `dist/dev` commit and the PyPI RC upload without +> handing control back. They can sign either the Delegate's built artifacts as-is or a local rebuild +> (provider builds are reproducible). The preceding *build* step is the Delegate's. + * Cleanup dist folder: ```shell script @@ -568,6 +618,13 @@ check above steps to install them. ## Commit the source packages to Apache SVN repo +> [!NOTE] +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this is the middle +> of the **PMC block** — the same PMC member who signed commits the packages and signatures here and +> continues to the PyPI RC upload. (`dist/dev` is committer-writable per [ASF Infra +> policy](https://infra.apache.org/release-publishing), so a Delegate *could* do this step, but it is +> kept with the PMC to avoid bouncing control mid-way.) + * Push the artifacts to ASF dev dist repo ```shell script @@ -608,6 +665,14 @@ cd "$AIRFLOW_REPO_ROOT" ## Publish the Regular distributions to PyPI (release candidates) +> [!NOTE] +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this is the end of +> the **PMC block** (sign → `dist/dev` → PyPI RC) — all uploads to the `apache-airflow-providers-*` +> PyPI namespace, RC and final alike, go through the PMC's trusted publishing identity. When done, +> the PMC member hands the generated `files/packages.txt` (the PyPI URLs) back to the Delegate, who +> needs it for the vote email's completeness gate and body, and the Delegate resumes at [Push the RC +> tags](#push-the-rc-tags). + In order to publish release candidate to PyPI you just need to build and release packages. The packages should however contain the rcN suffix in the version file name but not internally in the package, so you need to use `--version-suffix` switch to prepare those packages. @@ -872,6 +937,13 @@ breeze release-management check-release-files providers --release-date "${RELEAS Send out a vote to the dev@airflow.apache.org mailing list. Here you can prepare text of the email. +> [!NOTE] +> If you are a non-PMC Delegate running this step under the [delegated +> process](#delegating-release-duties-to-a-non-pmc-committer), set `IS_RM_VOTE_BINDING=false` below +> — your vote is not binding under ASF policy. Ask a PMC member to reply to the vote thread with +> their own explicit `+1 (binding)` as soon as they've verified the release; the vote is not valid +> until at least 3 such binding replies are posted, regardless of who sent the `[VOTE]` email. + ```shell script export VOTE_DURATION_IN_HOURS=72 export IS_SHORTEN_VOTE=$([ $VOTE_DURATION_IN_HOURS -ge 72 ] && echo "false" || echo "true") @@ -883,6 +955,10 @@ else # Linux fi export RELEASE_MANAGER_NAME="TODO:RELEASE_MANAGER_NAME" export GITHUB_ISSUE_LINK="TODO:ISSUE_LINK" +# true if the PMC itself is running the vote, false for a non-PMC Delegate (see note above) +export IS_RM_VOTE_BINDING=true +export RM_VOTE_BINDING_TEXT=$([ "$IS_RM_VOTE_BINDING" = "true" ] && echo "binding" || echo "non-binding") +export NON_PMC_RM_TEXT=$([ "$IS_RM_VOTE_BINDING" = "true" ] && echo "" || echo "I am a non-PMC committer running this release under Airflow's delegated release process; a PMC member will cast the binding votes needed to pass it.") ``` subject: @@ -901,7 +977,8 @@ I have just cut the new wave Airflow Providers packages with release preparation which will last for $VOTE_DURATION_IN_HOURS hours - which means that it will end on $VOTE_END_TIME UTC and until 3 binding +1 votes have been received. $([ "$IS_SHORTEN_VOTE" = "true" ] && echo "${SHORTEN_VOTE_TEXT}" || echo "") -Consider this my (binding) +1. +Consider this my ($RM_VOTE_BINDING_TEXT) +1. +$([ -n "$NON_PMC_RM_TEXT" ] && echo "$NON_PMC_RM_TEXT" || echo "") @@ -1393,6 +1470,12 @@ echo "prepare release date is ${RELEASE_DATE}" Once the vote has been passed, you will need to send a result vote to dev@airflow.apache.org: +> [!NOTE] +> A Delegate may send this `[RESULT][VOTE]` email under the [delegated +> process](#delegating-release-duties-to-a-non-pmc-committer), but only after confirming at least 3 +> binding `+1` votes from PMC members are already present in the vote thread — the email reports a +> decision the PMC has already made, it does not make that decision. + In both subject and message update DATE OF RELEASE, FIRST/LAST NAMES and numbers). In case some providers were excluded, explain why they were excluded and what is the plan for them (otherwise remove the optional part of the message). There are two options for releasing @@ -1447,6 +1530,13 @@ EOF ## Publish release to SVN +> [!NOTE] +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this step is +> owned by the **PMC member** — moving artifacts into `dist/release` requires PMC write karma that a +> non-PMC Delegate does not have (see the ownership table). The Delegate should have already +> confirmed the vote passed and, if helpful, staged the exact `svn cp`/`clean-old-provider-artifacts` +> commands below for the PMC member to run. + The best way of doing this is to svn cp between the two repos (this avoids having to upload the binaries again, and gives a clearer history in the svn commit logs. @@ -1533,6 +1623,11 @@ This is simply by removing the relevant files locally. ## Publish the packages to PyPI +> [!NOTE] +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this final PyPI +> upload is owned by the **PMC member**, using the same trusted publishing identity as the RC upload +> — it is not delegated. + By that time the packages should be in your dist folder. ```shell script @@ -1717,6 +1812,14 @@ gh pr create --title "Update providers metadata ${current_date}" --web ## Notify developers of release +> [!NOTE] +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) the official +> post-release communications — this announcement, the [security-issue +> announcements](#send-announcements-about-security-issues-fixed-in-the-release), [social +> media](#announce-about-the-release-in-social-media), and the [committee +> report](#add-release-data-to-apache-committee-report-helper) — are made by the **PMC member** +> under the PMC's authority. The Delegate can still draft the text and hand it over. + Notify users@airflow.apache.org (cc'ing dev@airflow.apache.org) that the artifacts have been published. From 9b5247f043b3f4a6197346c94a62e320c821ae1c Mon Sep 17 00:00:00 2001 From: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Date: Tue, 7 Jul 2026 19:16:41 +0300 Subject: [PATCH 2/2] Merge the provider build step into the PMC-owned release block A PMC member who only signs artifacts built by someone else does not actually know what they are signing, per ASF release policy guidance on owned/controlled hardware. Move the build step into the PMC block so the same PMC member builds, signs, commits to dist/dev, and publishes the PyPI RC without handing control back to the Delegate. --- dev/README_RELEASE_PROVIDERS.md | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/dev/README_RELEASE_PROVIDERS.md b/dev/README_RELEASE_PROVIDERS.md index 7bd40efc29587..1fbe3ec05baef 100644 --- a/dev/README_RELEASE_PROVIDERS.md +++ b/dev/README_RELEASE_PROVIDERS.md @@ -104,8 +104,7 @@ PMC. Split of duties: | Step | Owner | Notes | |---|---|---| | [Convert commits to changelog entries and bump provider versions](#convert-commits-to-changelog-entries-and-bump-provider-versions) | Delegate | Normal PR review/merge process, no PMC involvement needed. | -| [Build Provider distributions for SVN apache upload](#build-provider-distributions-for-svn-apache-upload) | Delegate | Builds are reproducible; anyone can build the unsigned artifacts. | -| [Build and sign](#build-and-sign-the-source-and-convenience-packages) + [commit to `dist/dev`](#commit-the-source-packages-to-apache-svn-repo) + [publish RC to PyPI](#publish-the-regular-distributions-to-pypi-release-candidates) | PMC | The three trust-sensitive steps are kept as **one contiguous PMC block** to avoid handing control back and forth: the PMC member re-builds reproducibly (or takes the Delegate's artifacts as-is), signs with their own key (already in the project's `KEYS` file), commits packages + signatures to `dist/dev`, and uploads the RC to the `apache-airflow-providers-*` PyPI namespace under the PMC's trusted publishing identity. The PMC then hands `files/packages.txt` (the PyPI URLs) back to the Delegate for the vote email. | +| [Build](#build-provider-distributions-for-svn-apache-upload) + [sign](#build-and-sign-the-source-and-convenience-packages) + [commit to `dist/dev`](#commit-the-source-packages-to-apache-svn-repo) + [publish RC to PyPI](#publish-the-regular-distributions-to-pypi-release-candidates) | PMC | Kept as **one contiguous PMC block**: per [ASF release policy](https://www.apache.org/legal/release-policy.html#owned-controlled-hardware), a PMC member signing a release should build it themselves from source rather than sign artifacts someone else built, so they know what they're actually signing. The PMC member builds, signs with their own key (already in the project's `KEYS` file), commits packages + signatures to `dist/dev`, and uploads the RC to the `apache-airflow-providers-*` PyPI namespace under the PMC's trusted publishing identity. The PMC then hands `files/packages.txt` (the PyPI URLs) back to the Delegate for the vote email. | | [Push the RC tags](#push-the-rc-tags) | Delegate | Plain git tag push, no elevated access needed. | | [Prepare documentation in Staging](#prepare-documentation-in-staging) | Delegate | | | [Prepare issue in GitHub to keep status of testing](#prepare-issue-in-github-to-keep-status-of-testing) | Delegate | Delegate also tracks the issue, the vote thread, and related PRs throughout the release. | @@ -126,7 +125,7 @@ remains the party accountable for the release under ASF policy. > [!NOTE] > Delegation is also a runway toward PMC membership. The first time a committer takes on the > Delegate role, the overseeing PMC member is encouraged to walk them through the reserved block -> live — sharing their screen (or pairing) through the sign → `dist/dev` → PyPI-RC steps so the +> live — sharing their screen (or pairing) through the build → sign → `dist/dev` → PyPI-RC steps so the > Delegate sees exactly how it is done. The aim is simply that these steps are familiar rather than > a surprise if and when the Delegate later becomes a PMC member and runs them for real. @@ -500,6 +499,14 @@ breeze release-management prepare-provider-documentation --include-removed-provi ## Build Provider distributions for SVN apache upload +> [!NOTE] +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this build step +> begins the **PMC block**, together with signing, the `dist/dev` commit, and the PyPI RC upload. +> Per [ASF release policy](https://www.apache.org/legal/release-policy.html#owned-controlled-hardware), +> a PMC member should build the release themselves before signing it, rather than sign artifacts +> built by someone else — otherwise they don't actually know what they're signing. So the PMC member +> who signs also runs the build below, instead of the Delegate handing over pre-built artifacts. + Those packages might get promoted to "final" packages by just renaming the files, so internally they should keep the final version number without the rc suffix, even if they are rc1/rc2/... candidates. @@ -557,11 +564,11 @@ key you want to use. ## Build and sign the source and convenience packages > [!NOTE] -> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this signing step -> begins the **PMC block** — the PMC member runs `sign.sh` with their own key (already in the -> project's `KEYS` file), then stays on through the `dist/dev` commit and the PyPI RC upload without -> handing control back. They can sign either the Delegate's built artifacts as-is or a local rebuild -> (provider builds are reproducible). The preceding *build* step is the Delegate's. +> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this step +> continues the **PMC block** started at [Build Provider distributions](#build-provider-distributions-for-svn-apache-upload) +> — the same PMC member builds and signs with their own key (already in the project's `KEYS` +> file), then stays on through the `dist/dev` commit and the PyPI RC upload without handing control +> back. * Cleanup dist folder: @@ -667,7 +674,7 @@ cd "$AIRFLOW_REPO_ROOT" > [!NOTE] > Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this is the end of -> the **PMC block** (sign → `dist/dev` → PyPI RC) — all uploads to the `apache-airflow-providers-*` +> the **PMC block** (build → sign → `dist/dev` → PyPI RC) — all uploads to the `apache-airflow-providers-*` > PyPI namespace, RC and final alike, go through the PMC's trusted publishing identity. When done, > the PMC member hands the generated `files/packages.txt` (the PyPI URLs) back to the Delegate, who > needs it for the vote email's completeness gate and body, and the Delegate resumes at [Push the RC