Skip to content

Migrate the production build chain (publish, cache, execution tests) to QuantEcon/mystmd after #363 #593

Description

@mmcky

Goal

Finish what #363 starts: once the JB2 branch lands, the repo should be 100% QuantEcon/mystmd — no jupyter-book 1.x anywhere in the build chain. #363 deliberately migrates only the PR preview (ci.yml) and ipynb builds (build-ipynb.yml); everything production-facing still runs the JB1/Sphinx toolchain from main, driven by lectures/_config.yml + _toc.yml, which #363 replaces with myst.yml. Mixing the two long-term means every lecture edit is validated against one engine and published with another.

What still runs jupyter-book 1.x

surface current mystmd path
publish.yml HTML deploy jb build lectures myst build --html — already proven in ci.yml / build-ipynb.yml
publish.yml PDF jb build --builder pdflatex typst export — a book-pdf export is already stubbed (commented) in lectures/myst.yml; needs building out and visual review
publish.yml notebook downloads sphinx-tojupyter custom builder myst build --ipynb — already prototyped in build-ipynb.yml, including the MyST-syntax-leak audit step
cache.yml weekly execution cache jb build mirror of ci.yml's build, or retire in favour of ci.yml's cache (whose key is now engine-aware)
execution-*.yml (linux/osx/win) pip install jupyter-book + Sphinx stack decide: port to mystmd, or retire — their purpose (execution regression) is partly covered by ci.yml now being a genuine cold-execution test
environment.yml pins jupyter-book>=1.0.4post1,<2.0 replace with myst_requirements.txt as the single environment
linkcheck_ignore in _config.yml Sphinx-only config already duplicated into linkcheck.yml ignore-patterns (#592); retires with the file

Housekeeping in the same sweep: cache.yml is still named "Build Cache [using jupyter-book]", and .github/workflows/_github_actions/ci.yml is a stale stashed copy of the old preview workflow.

Constraints and lessons to carry over (from the #363 incident, 2026-08-03)

Execution-affecting configuration must be inside the execution cache key — hash the workflow file and requirements alongside the lectures, as ci.yml now does, or an engine/env change ships green against stale outputs. Pin the mystmd fork by SHA and JAX by version. Keep XLA_PYTHON_CLIENT_PREALLOCATE=false (or successor policy from QuantEcon/meta#350) on any step that executes the JAX lectures concurrently on a single-GPU runner; the JB1 chain never needed it only because Sphinx executes sequentially. Keep the memory/dmesg telemetry pattern on GPU execution jobs.

Sequencing

This is follow-up work to #363, not part of it — it should land as its own PR(s) after #363 merges, with the PDF export likely the long pole (needs typst template work and visual signoff). Related: QuantEcon/meta#348 (shared build-lectures action) is the natural place for the standardised pieces to land once, and QuantEcon/meta#350 will supply the JAX-on-CI configuration defaults.

Carried over from #344 (consolidated 2026-08-04)

Two mystmd build warnings remain from the #342/#344 tracking pair (both closed in favour of this issue):

  • python_oop.md:12 — index directive parse failure: single: OOP II: Building Classes (the second colon breaks parsing). Fix by rewording the entry, e.g. single: OOP II -- Building Classes.
  • status.md:16unknown directive: nb-exec-table (sphinx-tojupyter only). Needs a mystmd equivalent or a redesign of the execution-status page.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions