Use trusted publishing

[mattip]

Now that all the builds are on github CI, we can move to trusted publishing. Help welcome. I guess the publishing should be triggered by merging a PR to main and/or maintenance/* branches.

[mattip]

@andyfaff, could you help out with trusted publishing? Can we do it when we still have 3 workflows: posix, win-x86_64, win-arm64?

[andyfaff]

I can help. Only one workflow (e.g. publish.yml) can work with trusted publishing. That workflow can then use other workflows from the repo, in this case the ones that build the artefacts for each platform.

[andyfaff]

Out of interest where do the mac wheels get built?

[mattip]

The macos ones are built in the posix ci run. A bit of a misnomer 🤷‍♂️

[andyfaff]

Underway. Can you see if you can grab the testpypi name? It'd be good to practice on there before th real thing.

[mattip]

Done,
https://test.pypi.org/project/scipy-openblas64/
https://test.pypi.org/project/scipy-openblas32/

[mattip]

Let me know what to fill in for trusted publishing

[andyfaff]

If that image is for testpypi then the environment name is 'testpypi'. The workflow will be called 'publish.yml'.

What's the typical workflow for a release? I want to make a check in the workflow to ensure what has been built is definitely a release, and not a nightly. Do you normally push a tag and then do a manual build from the tag?

THere's a few ways to do that, e.g:

• check the name of a wheel
• if [ -n "$(git tag --points-at HEAD)" ]; then. This checks if the current HEAD has a tag, but is less useful if there have been commits since.

[andyfaff]

You also need to configure the settings for the MacPython/openblas-libs repo.

1. Navigate to Settings --> Environments.
2. Make two new environments. One called 'testpypi' and the other called 'pypi'.
3. If necessary you can specify protection rules that limit the branches that you can deploy from. e.g. on scipy we could limit deployments to branches called maintenance/**.

If you have access to numpy/numpy-release then you can see what's done there.

[andyfaff]

When you configure each environment you also need to add 'required reviewers', i.e. "Specify people or teams that may approve workflow runs when they access this environment."

I suggest that you add whoever deals with releases. For scipy and numpy there are numpy-release and scipy-release teams. You can of course add individual user handles.

These reviewers get a notification for when the artefacts are ready to deploy. At that point the release manager looks at all the wheels then does a manual approval for the deployment to pypi.

[mattip]

What's the typical workflow for a release?

I am the only committer here, rarely someone else will pitch in. So my workflow has been

• create a feature branch, bump the pyproject.toml version number manually following the OpenBLAS commit in github_commit.txt. If the github_commit.txt is unchanged, then increment the build number on the version, so 0.3.31.126.0 would become 0.3.31.126.1
• Once I am happy with CI I merge to main
• pull main, create a tag from the pyproject.toml version, and push to main.
• Download the wheels from anaconda.org's storage with tools/download-wheels.py
• use twine to upload the wheels to PyPI

So I think it makes sense to push to PyPI with trusted publishing on every merge to main. Is there a way to automate the tag creation on merging to main?

Since I am the only committer, I would also be the 'required reviewer'.

[mattip]

There is still a need for the cron job to run once a week to build wheels off OpenBLAS/develop and upload them to anaconda.org, as well as run test CI against PRs with no uploading.

[andyfaff]

There is still a need for the cron job to run once a week to build wheels off OpenBLAS/develop and upload them to anaconda.org, as well as run test CI against PRs with no uploading.

That will still occur on the individual workflows.

It seems to me that when you want to release you push a tag, which at that moment in time is pointing to the HEAD commit.
How about one of these two possibilities:

• You could start a release run manually, selecting the pypi or testpypi environment, then approve deployment when wheels are built and CI passes. A check is done to see if HEAD corresponds to a tag.
• This option is more automated. As soon as you push a tag to main the wheels+CI are run, then deployment automatically occurs. It's already know at this point that it should be a release because the run was activated by a tag. You could still have a deployment approval if you wanted to.

[mattip]

I have created the testpypi and pypi environments and configured testpypi.org/project/scipy-openblas64 and testpypi.org/project/scipy-openblas32 to use the testpypi environment, publish.yml workflow.

Once we get that going I will configure pypi.org/project/* as well.

[mattip]

#256 was merged, thanks. I triggered a run with testpypi.

What happens if i trigger a run with “none” as the environment?