Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -770,13 +770,33 @@ jobs:
python3 "$AUTOBUILD_PATH/generate.py" ${{ matrix.workspace.name }}
git add *.ipynb
git commit -m "Release ${{ needs.version_number.outputs.version_number }}: update notebooks and version" || true
- name: Write workspace version.txt
- name: Write workspace version
if: "${{ github.event.inputs.skip_release != 'true' }}"
run: |
VERSION="${{ needs.version_number.outputs.version_number }}"
pushd workspace
echo "$VERSION" > version.txt
git add version.txt
python3 - "$VERSION" <<'PY'
import re, sys, pathlib
version = sys.argv[1]
path = pathlib.Path("config/general.yaml")
if not path.exists():
sys.exit(0)
text = path.read_text()
line_re = re.compile(r"^(?P<indent>\s*)workspace_version:\s*.*$", re.MULTILINE)
if line_re.search(text):
text = line_re.sub(lambda m: f"{m.group('indent')}workspace_version: {version}", text)
else:
block_re = re.compile(r"^version:\s*\n", re.MULTILINE)
if block_re.search(text):
text = block_re.sub(f"version:\n workspace_version: {version}\n", text, count=1)
else:
if not text.endswith("\n"):
text += "\n"
text += f"version:\n workspace_version: {version}\n"
path.write_text(text)
PY
git add version.txt config/general.yaml 2>/dev/null || git add version.txt
git commit -m "Release $VERSION: pin workspace version" || true
- name: Bump Colab URL tag refs
if: "${{ github.event.inputs.skip_release != 'true' && matrix.workspace.bump_colab_urls == true }}"
Expand Down
9 changes: 8 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,14 @@ This script does the following for each repo:

Before the per-repo loop, `pre_build.sh` invokes `admin_jammy/software/ensure_workspace_labels.sh` to assert the canonical `pending-release` label across every release-window repo (idempotent — a no-op when nothing has drifted).

After the loop, before dispatching the workflow, `pre_build.sh` invokes `verify_workspace_versions.sh` — a fail-fast check that blocks the release if any workspace's `version.txt` is ahead of the currently-installed library version. This guards against bootstrap commits that set an aspirational `version.txt`, which would otherwise crash every `welcome.py` / `start_here.py` run with `WorkspaceVersionMismatchError` until the next release lands.
After the loop, before dispatching the workflow, `pre_build.sh` invokes `verify_workspace_versions.sh` — a fail-fast check that blocks the release if any workspace's pinned version is ahead of the currently-installed library version. This guards against bootstrap commits that set an aspirational version, which would otherwise crash every script run with `WorkspaceVersionMismatchError` until the next release lands.

The pinned workspace version is resolved with the same precedence as `autoconf.workspace.check_version`:

1. `config/general.yaml` — `version.workspace_version` key (canonical, written by `release.yml`).
2. `version.txt` at the workspace root (legacy fallback, also written by `release.yml`).

Both sources are written atomically by the `Write workspace version` step in `release.yml`; if they ever disagree on a checked-out workspace, `verify_workspace_versions.sh` fails before dispatch. The runtime check in `autoconf.workspace.check_version` (called from every library's `__init__.py`) reads the same precedence, so workspace/library mismatches surface on every script run, not just `welcome.py`.

`generate.py` is run from the workspace root with `PYTHONPATH` pointing at `PyAutoBuild/autobuild/`. Only specific safe directories are committed — never `output/`, `output_model/`, or run-generated artefacts. After all workspaces are done, PyAutoBuild itself is committed and pushed, then `gh workflow run release.yml` dispatches the GitHub Actions release.

Expand Down
90 changes: 74 additions & 16 deletions verify_workspace_versions.sh
Original file line number Diff line number Diff line change
@@ -1,18 +1,24 @@
#!/usr/bin/env bash
# verify_workspace_versions.sh — fail fast if any workspace's version.txt is
# verify_workspace_versions.sh — fail fast if any workspace's pinned version is
# AHEAD of the currently-installed library version.
#
# Usage: bash PyAutoBuild/verify_workspace_versions.sh
#
# Background: the bootstrap commit for a new workspace-style repo (HowToLens,
# 2026-04-21) set version.txt to today's date as an aspirational tag. Until
# the next release dispatch wrote a real version.txt, every welcome.py run
# the next release dispatch wrote a real version, every welcome.py run
# crashed with WorkspaceVersionMismatchError. This check blocks pre_build.sh
# from dispatching a release that would be invalidated by such a mismatch.
#
# Compares each of the 7 workspaces with a version.txt (3 main + 3 HowTo +
# euclid_pipeline) against its installed library version, parsed as a
# YYYY.M.D.B 4-tuple of ints. Exits 1 if any workspace.version > library.version.
# Source-of-truth precedence (mirrors autoconf.workspace.check_version):
# 1. config/general.yaml's `version.workspace_version` key
# 2. version.txt at the workspace root (legacy fallback)
# If both exist and disagree, the script fails — they must be kept in sync
# by the release pipeline.
#
# Compares each of the 7 workspaces (3 main + 3 HowTo + euclid_pipeline)
# against its installed library version, parsed as a YYYY.M.D.B 4-tuple of
# ints. Exits 1 if any workspace.version > library.version.
#
# Workspace → library mapping mirrors the release_workspaces matrix in
# .github/workflows/release.yml.
Expand Down Expand Up @@ -55,24 +61,75 @@ compare_versions() {
echo "MATCH"
}

# Resolve the workspace's pinned version using the same precedence as
# autoconf.workspace.check_version. Echoes the version string on stdout, or
# prints "MISSING" if neither source is present, "MISMATCH:<yaml>:<txt>" if
# both exist and disagree.
resolve_workspace_version() {
local ws_root="$1"
python3 - "$ws_root" <<'PY'
import pathlib, sys
root = pathlib.Path(sys.argv[1])
yaml_v = None
try:
import yaml
cfg = root / "config" / "general.yaml"
if cfg.exists():
with cfg.open() as f:
data = yaml.safe_load(f) or {}
v = (data.get("version") or {}).get("workspace_version")
if v is not None:
yaml_v = str(v).strip()
except Exception:
pass
txt_v = None
txt_path = root / "version.txt"
if txt_path.exists():
txt_v = txt_path.read_text().strip() or None
if yaml_v and txt_v and yaml_v != txt_v:
print(f"MISMATCH:{yaml_v}:{txt_v}")
elif yaml_v:
print(yaml_v)
elif txt_v:
print(txt_v)
else:
print("MISSING")
PY
}

failed=0

for entry in "${WORKSPACES[@]}"; do
ws="${entry%%|*}"
pkg="${entry#*|}"
version_file="$PYAUTOBASE/$ws/version.txt"
ws_root="$PYAUTOBASE/$ws"

if [ ! -f "$version_file" ]; then
printf " %-45s SKIP (no version.txt)\n" "$ws"
if [ ! -d "$ws_root" ]; then
printf " %-45s SKIP (workspace dir missing)\n" "$ws"
continue
fi

ws_version=$(tr -d '[:space:]' < "$version_file")
if [ -z "$ws_version" ]; then
printf " %-45s FAIL (version.txt is empty)\n" "$ws" >&2
failed=1
continue
fi
ws_version=$(resolve_workspace_version "$ws_root")

case "$ws_version" in
MISSING)
printf " %-45s SKIP (no general.yaml workspace_version and no version.txt)\n" "$ws"
continue
;;
MISMATCH:*)
yaml_v="${ws_version#MISMATCH:}"; yaml_v="${yaml_v%%:*}"
txt_v="${ws_version##*:}"
printf " %-45s FAIL (general.yaml workspace_version=%s disagrees with version.txt=%s)\n" \
"$ws" "$yaml_v" "$txt_v" >&2
failed=1
continue
;;
"")
printf " %-45s FAIL (could not resolve workspace version)\n" "$ws" >&2
failed=1
continue
;;
esac

if ! lib_version=$(python3 -c "import $pkg; print($pkg.__version__)" 2>/dev/null); then
printf " %-45s SKIP (cannot import %s)\n" "$ws" "$pkg"
Expand Down Expand Up @@ -102,8 +159,9 @@ done

if [ "$failed" -ne 0 ]; then
echo >&2
echo "verify_workspace_versions: at least one workspace is AHEAD of its installed library." >&2
echo " Release dispatch blocked. Patch the offending version.txt(s)" >&2
echo "verify_workspace_versions: at least one workspace is AHEAD of its installed library" >&2
echo " or has a config/general.yaml ↔ version.txt disagreement." >&2
echo " Release dispatch blocked. Patch the offending workspace(s)" >&2
echo " to match the installed library, then re-run." >&2
exit 1
fi