Skip to content

Pin lean container's science stack to the Anaconda 2025.12 baseline (#28) - #84

Merged
mmcky merged 1 commit into
mainfrom
fix/28-pin-lean-container-numpy-scipy
Jun 22, 2026
Merged

Pin lean container's science stack to the Anaconda 2025.12 baseline (#28)#84
mmcky merged 1 commit into
mainfrom
fix/28-pin-lean-container-numpy-scipy

Conversation

@mmcky

@mmcky mmcky commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Closes #28.

Root cause

The quantecon-build (lean) image listed its scientific stack unpinned, so numpy/scipy/pandas/etc. resolved to latest at build time and drifted ahead of the Anaconda 2025.12 set that every lecture repo pins via anaconda=2025.12 (and that the full quantecon image uses). That drift produced non-reproducible images that diverged from what the lectures are tested against, and surfaced downstream as a CellExecutionError in un_insure.md's sp.optimize.fsolve(Vu_error_Λ, 15000, ...) path for repos that build on the lean image (e.g. lecture-dp). The full image already pins anaconda=2025.12 and was unaffected.

Fix

Pin the lean image's core scientific stack to the exact Anaconda 2025.12 versions, so the lean image stays reproducible and consistent with the full image and the source repos.

package was now (= Anaconda 2025.12)
numpy unpinned 2.3.5
scipy unpinned 1.16.3
pandas unpinned 2.3.3
matplotlib unpinned 3.10.6
seaborn unpinned 0.13.2
sympy unpinned 1.14.0
numba unpinned 0.62.1
networkx unpinned 3.5
statsmodels unpinned 0.14.5
scikit-learn unpinned 1.7.2

Pinning the whole core stack (not just numpy/scipy) eliminates the drift class entirely and keeps numba/numpy in their tested pairing. Dependabot (conda ecosystem) will surface future bumps for review; bump the set when the lecture repos move to a newer Anaconda release.

On the exact breakage — honest caveat

I could not reproduce the original un_insure.md failure on current packages. The actual calibration cell runs cleanly on four combinations I tested — numpy 2.3.5+scipy 1.16.3 (2025.12), numpy 2.4.0+scipy 1.15.3, numpy 2.4.0+scipy 1.16.3, and numpy 2.4.6+scipy 1.17.1 (2026.06). numpy 2.4.0 did make float()/int() of a 1-d array a hard TypeError, which is the most likely trigger class, but the specific combination that broke it in March 2026 appears to have been a transient regression that's since been patched upstream. The fix here stands regardless: it addresses the root cause (unpinned drift / non-reproducible builds that diverge from the source-repo baseline), not the transient trigger.

Why not Anaconda 2026.06

2026.06 exists and is installable for py313, and un_insure.md does run on its stack (numpy 2.4.6 / scipy 1.17.1). But it jumps to pandas 3.0 (a major release no lecture repo is validated against) and would make the containers diverge from the source repos (all still on 2025.12), recreating the drift this PR fixes. The newer baseline should be a separate, coordinated bump of the lecture repos + both containers.

Verification

  • Confirmed the pinned set solves on linux-64 (the container platform) via conda env create --dry-run with CONDA_OVERRIDE_GLIBC=2.39; numpy resolves to 2.3.5, numba to 0.62.1, nodejs stays at 20.x (within >=20,<25).
  • Version data taken directly from conda search anaconda=2025.12=py313_mkl_0 --info against pkgs/main.
  • Ran the actual un_insure.md calibration cell (extracted helpers + brentq→fsolve) on the pinned baseline and on the current-latest stack.

Notes for reviewers

  • Testing does cover this. test-containers-lectures.yml runs after each container build (workflow_run) and its Stage 1 "Build HTML (Execute Notebooks)" executes notebooks for lecture-python-advanced.myst (which contains un_insure.md) on both images. Caveat: that workflow is non-blocking (runs after :latest is pushed) and has been red for months.
  • Separate pre-existing breakage (not addressed here): the container tests are currently failing on a ChromeNotFoundError from kaleido v1 (unpinned) on Plotly static-image export (BCG_complete_mkts.md, BCG_incomplete_mkts.md, knowing_forecasts_of_others.md) — a distinct "unpinned dep bumped a major" problem worth its own issue/PR. The container test signal won't go fully green until that's fixed too.

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings June 22, 2026 00:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes reproducibility and downstream notebook execution failures in the lean quantecon-build container by pinning its core scientific Python stack to the Anaconda 2025.12 baseline, aligning it with the full quantecon image and with lecture repositories that pin anaconda=2025.12.

Changes:

  • Pin the lean container’s core scientific stack (numpy/scipy/pandas/matplotlib/etc.) to the Anaconda 2025.12 versions to prevent “latest at build time” drift.
  • Update the lean container README to document that the science stack is pinned to the 2025.12 baseline.
  • Add a changelog entry describing the fix and the downstream failure it addresses.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
containers/quantecon-build/environment.yml Pins the core scientific stack to Anaconda 2025.12 versions to prevent dependency drift in the lean image.
containers/quantecon-build/README.md Documents that the lean image’s science stack is pinned to the Anaconda 2025.12 baseline.
CHANGELOG.md Records the fix and the rationale (preventing numpy drift causing downstream notebook execution errors).

)

The quantecon-build (lean) image listed numpy/scipy/pandas/etc. unpinned, so
they resolved to "latest at build time" and drifted ahead of the Anaconda
2025.12 set that every lecture repo pins via `anaconda=2025.12` (and that the
full `quantecon` image uses). That drift produced non-reproducible images that
diverged from what the lectures are tested against, and surfaced downstream as a
CellExecutionError in un_insure.md's `fsolve(..., 15000)` call on repos building
on the lean image (see #28).

Pin the core scientific stack (numpy, scipy, pandas, matplotlib, seaborn,
sympy, numba, networkx, statsmodels, scikit-learn) to the exact Anaconda
2025.12 versions so the lean image stays reproducible and consistent with the
full image and the source repos. Verified the set solves on linux-64 (numpy
2.3.5, scipy 1.16.3, ...) and that un_insure.md's calibration cell runs on it.

Note: the precise dependency combination that broke un_insure.md in March 2026
is no longer reproducible — the calibration cell runs on both the 2025.12 and
current-latest (numpy 2.4.6 / scipy 1.17.1) stacks today; the likely trigger was
numpy 2.4.0's stricter array-to-scalar conversion. Pinning to the source-repo
baseline fixes the root cause (unpinned drift / non-reproducible builds)
regardless of the transient trigger.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mmcky
mmcky force-pushed the fix/28-pin-lean-container-numpy-scipy branch from 0907591 to df0ead8 Compare June 22, 2026 00:43
@mmcky
mmcky merged commit 81b7047 into main Jun 22, 2026
@mmcky
mmcky deleted the fix/28-pin-lean-container-numpy-scipy branch June 22, 2026 01:18
mmcky added a commit that referenced this pull request Jul 8, 2026
All flagship lecture repos now pin `anaconda=2026.06`, so move the
containers to match — keeping them reproducible and consistent with the
source repos and with each other (the #28 / PR #84 rationale, now
retargeted from 2025.12 to 2026.06):

- Full `quantecon` image: anaconda 2025.12 -> 2026.06.
- Lean `quantecon-build` image: repin the individual scientific stack to
  the exact anaconda=2026.06 linux-64 py313 (MKL) versions — numpy 2.4.6,
  scipy 1.17.1, pandas 3.0.3, matplotlib 3.10.9, numba 0.65.1, networkx
  3.6.1, statsmodels 0.14.6, scikit-learn 1.8.0 (seaborn 0.13.2 / sympy
  1.14.0 unchanged). Resolved via `conda search anaconda=2026.06 --info`;
  linux-64 keeps MKL as the default BLAS, so the existing mkl pins stand.
  These are the deliberate coordinated-bump versions — not Dependabot's
  latest, which overshoot the baseline (numpy 2.5.0, scipy 1.18.0, etc.).

Also add the pinned stack + `anaconda` to the Dependabot conda `ignore`
list so it holds at 2026.06 until the next coordinated bump, instead of
being drifted piecemeal (the #86 / #87 PRs this supersedes).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mmcky added a commit that referenced this pull request Jul 8, 2026
All flagship lecture repos now pin `anaconda=2026.06`, so move the
containers to match — keeping them reproducible and consistent with the
source repos and with each other (the #28 / PR #84 rationale, now
retargeted from 2025.12 to 2026.06):

- Full `quantecon` image: anaconda 2025.12 -> 2026.06.
- Lean `quantecon-build` image: repin the individual scientific stack to
  the exact anaconda=2026.06 linux-64 py313 (MKL) versions — numpy 2.4.6,
  scipy 1.17.1, pandas 3.0.3, matplotlib 3.10.9, numba 0.65.1, networkx
  3.6.1, statsmodels 0.14.6, scikit-learn 1.8.0 (seaborn 0.13.2 / sympy
  1.14.0 unchanged). Resolved via `conda search anaconda=2026.06 --info`;
  linux-64 keeps MKL as the default BLAS, so the existing mkl pins stand.
  These are the deliberate coordinated-bump versions — not Dependabot's
  latest, which overshoot the baseline (numpy 2.5.0, scipy 1.18.0, etc.).

Also add the pinned stack + `anaconda` to the Dependabot conda `ignore`
list so it holds at 2026.06 until the next coordinated bump, instead of
being drifted piecemeal (the #86 / #87 PRs this supersedes).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

quantecon-build container: unpinned scipy/numpy causes CellExecutionError in un_insure.md

2 participants