Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 111 additions & 1 deletion dev/README_RELEASE_PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
- [Intro](#intro)
- [What the provider distributions are](#what-the-provider-distributions-are)
- [Decide when to release](#decide-when-to-release)
- [Delegating release duties to a non-PMC committer](#delegating-release-duties-to-a-non-pmc-committer)
- [Collect ambiguities during the release (for a follow-up doc PR)](#collect-ambiguities-during-the-release-for-a-follow-up-doc-pr)
- [Special procedures (done very infrequently)](#special-procedures-done-very-infrequently)
- [Bump min Airflow version for providers](#bump-min-airflow-version-for-providers)
Expand Down Expand Up @@ -87,6 +88,47 @@ a given provider needs to be released due to new features or due to bug fixes.
package separately, but due to voting and release overhead we try to group releases of Provider
distributions together.

## Delegating release duties to a non-PMC committer

Per the [ASF release policy](http://www.apache.org/legal/release-policy.html), the Release Manager
does not need to be a PMC member, and there is no requirement that only a PMC member may call a
release vote. The policy's own wording: *"If the Release Manager is not a member of the PMC, they
will need to ask a PMC member to do the actual release publication"* — i.e. the one hard boundary
is write access to the `dist/release` SVN area and the binding vote itself; everything else can be
run by any committer.

This means a non-PMC committer (the **Delegate** below) can run most of the provider release
process end to end, with a PMC member only stepping in for the parts ASF policy reserves to the
PMC. Split of duties:

| Step | Owner | Notes |
|---|---|---|
| [Convert commits to changelog entries and bump provider versions](#convert-commits-to-changelog-entries-and-bump-provider-versions) | Delegate | Normal PR review/merge process, no PMC involvement needed. |
| [Build](#build-provider-distributions-for-svn-apache-upload) + [sign](#build-and-sign-the-source-and-convenience-packages) + [commit to `dist/dev`](#commit-the-source-packages-to-apache-svn-repo) + [publish RC to PyPI](#publish-the-regular-distributions-to-pypi-release-candidates) | PMC | Kept as **one contiguous PMC block**: per [ASF release policy](https://www.apache.org/legal/release-policy.html#owned-controlled-hardware), a PMC member signing a release should build it themselves from source rather than sign artifacts someone else built, so they know what they're actually signing. The PMC member builds, signs with their own key (already in the project's `KEYS` file), commits packages + signatures to `dist/dev`, and uploads the RC to the `apache-airflow-providers-*` PyPI namespace under the PMC's trusted publishing identity. The PMC then hands `files/packages.txt` (the PyPI URLs) back to the Delegate for the vote email. |
| [Push the RC tags](#push-the-rc-tags) | Delegate | Plain git tag push, no elevated access needed. |
| [Prepare documentation in Staging](#prepare-documentation-in-staging) | Delegate | |
| [Prepare issue in GitHub to keep status of testing](#prepare-issue-in-github-to-keep-status-of-testing) | Delegate | Delegate also tracks the issue, the vote thread, and related PRs throughout the release. |
| [Prepare voting email for Providers release candidate](#prepare-voting-email-for-providers-release-candidate) | Delegate | Delegate may send the `[VOTE]` email, but must **not** claim a personal binding `+1` (see note in that section) — only PMC votes are binding. |
| Casting the deciding vote(s) | PMC | At least 3 binding `+1` votes from PMC members are required for the release to pass; this cannot be delegated. |
| [Summarize the voting for the Apache Airflow release](#summarize-the-voting-for-the-apache-airflow-release) (`[RESULT][VOTE]`) | Delegate | Only after at least 3 binding PMC `+1` votes are already visible in the thread — the Delegate is reporting a result the PMC already reached, not deciding it. |
| [Publish release to SVN](#publish-release-to-svn) (`dist/release`) | PMC | Per [ASF Infra policy](https://infra.apache.org/release-publishing), `dist/release` write access is PMC-only by default (a project can request Infra to open it to all committers, but Airflow has not done so). |
| [Publish the packages to PyPI](#publish-the-packages-to-pypi) (final) | PMC | |
| [Add the final release tag in git](#add-the-final-release-tag-in-git) | Either | Not privileged; whoever is running that phase of the process does it. |
| [Publish documentation](#publish-documentation) (live) | Delegate | |
| [Update providers metadata](#update-providers-metadata) | Delegate | |
| [Notify developers of release](#notify-developers-of-release), security announcements, social media, committee report | PMC | Official project communications made under the PMC's authority. |
| [Close the testing status issue](#close-the-testing-status-issue) | Delegate | |

The PMC continues to oversee the overall process regardless of how many steps are delegated, and
remains the party accountable for the release under ASF policy.

> [!NOTE]
> Delegation is also a runway toward PMC membership. The first time a committer takes on the
> Delegate role, the overseeing PMC member is encouraged to walk them through the reserved block
> live — sharing their screen (or pairing) through the build → sign → `dist/dev` → PyPI-RC steps so the
> Delegate sees exactly how it is done. The aim is simply that these steps are familiar rather than
> a surprise if and when the Delegate later becomes a PMC member and runs them for real.

# Collect ambiguities during the release (for a follow-up doc PR)

These instructions are imperfect. Every release uncovers at least one command
Expand Down Expand Up @@ -457,6 +499,14 @@ breeze release-management prepare-provider-documentation --include-removed-provi

## Build Provider distributions for SVN apache upload

> [!NOTE]
> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this build step
> begins the **PMC block**, together with signing, the `dist/dev` commit, and the PyPI RC upload.
> Per [ASF release policy](https://www.apache.org/legal/release-policy.html#owned-controlled-hardware),
> a PMC member should build the release themselves before signing it, rather than sign artifacts
> built by someone else — otherwise they don't actually know what they're signing. So the PMC member
> who signs also runs the build below, instead of the Delegate handing over pre-built artifacts.

Those packages might get promoted to "final" packages by just renaming the files, so internally they
should keep the final version number without the rc suffix, even if they are rc1/rc2/... candidates.

Expand Down Expand Up @@ -513,6 +563,13 @@ key you want to use.

## Build and sign the source and convenience packages

> [!NOTE]
> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this step
> continues the **PMC block** started at [Build Provider distributions](#build-provider-distributions-for-svn-apache-upload)
> — the same PMC member builds and signs with their own key (already in the project's `KEYS`
> file), then stays on through the `dist/dev` commit and the PyPI RC upload without handing control
> back.

* Cleanup dist folder:

```shell script
Expand Down Expand Up @@ -568,6 +625,13 @@ check above steps to install them.

## Commit the source packages to Apache SVN repo

> [!NOTE]
> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this is the middle
> of the **PMC block** — the same PMC member who signed commits the packages and signatures here and
> continues to the PyPI RC upload. (`dist/dev` is committer-writable per [ASF Infra
> policy](https://infra.apache.org/release-publishing), so a Delegate *could* do this step, but it is
> kept with the PMC to avoid bouncing control mid-way.)

* Push the artifacts to ASF dev dist repo

```shell script
Expand Down Expand Up @@ -608,6 +672,14 @@ cd "$AIRFLOW_REPO_ROOT"

## Publish the Regular distributions to PyPI (release candidates)

> [!NOTE]
> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this is the end of
> the **PMC block** (build → sign → `dist/dev` → PyPI RC) — all uploads to the `apache-airflow-providers-*`
> PyPI namespace, RC and final alike, go through the PMC's trusted publishing identity. When done,
> the PMC member hands the generated `files/packages.txt` (the PyPI URLs) back to the Delegate, who
> needs it for the vote email's completeness gate and body, and the Delegate resumes at [Push the RC
> tags](#push-the-rc-tags).

In order to publish release candidate to PyPI you just need to build and release packages.
The packages should however contain the rcN suffix in the version file name but not internally in the package,
so you need to use `--version-suffix` switch to prepare those packages.
Expand Down Expand Up @@ -872,6 +944,13 @@ breeze release-management check-release-files providers --release-date "${RELEAS
Send out a vote to the dev@airflow.apache.org mailing list. Here you can prepare text of the
email.

> [!NOTE]
> If you are a non-PMC Delegate running this step under the [delegated
> process](#delegating-release-duties-to-a-non-pmc-committer), set `IS_RM_VOTE_BINDING=false` below
> — your vote is not binding under ASF policy. Ask a PMC member to reply to the vote thread with
> their own explicit `+1 (binding)` as soon as they've verified the release; the vote is not valid
> until at least 3 such binding replies are posted, regardless of who sent the `[VOTE]` email.

```shell script
export VOTE_DURATION_IN_HOURS=72
export IS_SHORTEN_VOTE=$([ $VOTE_DURATION_IN_HOURS -ge 72 ] && echo "false" || echo "true")
Expand All @@ -883,6 +962,10 @@ else # Linux
fi
export RELEASE_MANAGER_NAME="TODO:RELEASE_MANAGER_NAME"
export GITHUB_ISSUE_LINK="TODO:ISSUE_LINK"
# true if the PMC itself is running the vote, false for a non-PMC Delegate (see note above)
export IS_RM_VOTE_BINDING=true
export RM_VOTE_BINDING_TEXT=$([ "$IS_RM_VOTE_BINDING" = "true" ] && echo "binding" || echo "non-binding")
export NON_PMC_RM_TEXT=$([ "$IS_RM_VOTE_BINDING" = "true" ] && echo "" || echo "I am a non-PMC committer running this release under Airflow's delegated release process; a PMC member will cast the binding votes needed to pass it.")
```

subject:
Expand All @@ -901,7 +984,8 @@ I have just cut the new wave Airflow Providers packages with release preparation
which will last for $VOTE_DURATION_IN_HOURS hours - which means that it will end on $VOTE_END_TIME UTC and until 3 binding +1 votes have been received.
$([ "$IS_SHORTEN_VOTE" = "true" ] && echo "${SHORTEN_VOTE_TEXT}" || echo "")

Consider this my (binding) +1.
Consider this my ($RM_VOTE_BINDING_TEXT) +1.
$([ -n "$NON_PMC_RM_TEXT" ] && echo "$NON_PMC_RM_TEXT" || echo "")

<ADD ANY HIGH-LEVEL DESCRIPTION OF THE CHANGES HERE!>

Expand Down Expand Up @@ -1393,6 +1477,12 @@ echo "prepare release date is ${RELEASE_DATE}"

Once the vote has been passed, you will need to send a result vote to dev@airflow.apache.org:

> [!NOTE]
> A Delegate may send this `[RESULT][VOTE]` email under the [delegated
> process](#delegating-release-duties-to-a-non-pmc-committer), but only after confirming at least 3
> binding `+1` votes from PMC members are already present in the vote thread — the email reports a
> decision the PMC has already made, it does not make that decision.

In both subject and message update DATE OF RELEASE, FIRST/LAST NAMES and numbers). In case
some providers were excluded, explain why they were excluded and what is the plan for them
(otherwise remove the optional part of the message). There are two options for releasing
Expand Down Expand Up @@ -1447,6 +1537,13 @@ EOF

## Publish release to SVN

> [!NOTE]
> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this step is
> owned by the **PMC member** — moving artifacts into `dist/release` requires PMC write karma that a
> non-PMC Delegate does not have (see the ownership table). The Delegate should have already
> confirmed the vote passed and, if helpful, staged the exact `svn cp`/`clean-old-provider-artifacts`
> commands below for the PMC member to run.

The best way of doing this is to svn cp between the two repos (this avoids having to upload the binaries
again, and gives a clearer history in the svn commit logs.

Expand Down Expand Up @@ -1533,6 +1630,11 @@ This is simply by removing the relevant files locally.

## Publish the packages to PyPI

> [!NOTE]
> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) this final PyPI
> upload is owned by the **PMC member**, using the same trusted publishing identity as the RC upload
> — it is not delegated.

By that time the packages should be in your dist folder.

```shell script
Expand Down Expand Up @@ -1717,6 +1819,14 @@ gh pr create --title "Update providers metadata ${current_date}" --web

## Notify developers of release

> [!NOTE]
> Under the [delegated process](#delegating-release-duties-to-a-non-pmc-committer) the official
> post-release communications — this announcement, the [security-issue
> announcements](#send-announcements-about-security-issues-fixed-in-the-release), [social
> media](#announce-about-the-release-in-social-media), and the [committee
> report](#add-release-data-to-apache-committee-report-helper) — are made by the **PMC member**
> under the PMC's authority. The Delegate can still draft the text and hand it over.

Notify users@airflow.apache.org (cc'ing dev@airflow.apache.org) that
the artifacts have been published.

Expand Down