feat: Phase 4 — instrument-framed README dashboard

[Jammy2211]

Overview

Phase 4 of the autolens_profiling z_feature roadmap — build the public-facing READMEs (top-level + per-section) so that anyone landing on https://github.com/PyAutoLabs/autolens_profiling immediately sees the latest run-times framed by astronomy instrument (HST, Euclid, JWST, ALMA, …) rather than pixel counts or grid sizes.

The user has been most explicit about this deliverable: "Framing it in terms of Astronomy instruments and telescopes, instead of just 'number of pixels' or other such metrics is way more intuitive."

Depends on Phase 1 (likelihood mirror, shipped) and Phase 2 (simulators mirror, issue #4, in flight) — needs their result artifacts to exist before the auto-generation has anything to populate. Phase 3 (Nautilus, issue #5) is independent of the dashboard skeleton but its results round out the section coverage.

Plan

• Add a small scripts/build_readme.py (or similar) that scans results/**/*_summary_v<version>.{json,png}, picks the latest version per axis, and regenerates the markdown tables in every relevant README. Idempotent — re-running on unchanged results produces no diff. Phase 5's CI workflow will call this.
• Refresh README.md (top-level) with a headline instrument × model-composition table.
• Refresh likelihood/README.md and each likelihood/<type>/README.md with instrument tables for that section.
• Refresh simulators/README.md with per-simulator run-time tables broken down by instrument where applicable.
• Refresh searches/README.md and searches/nautilus/README.md with sampler run-time tables (instrument framing is less natural here — n_free_parameters is a reasonable alternative row axis).
• Generate version-history PNG line plots per instrument × model combination using the versioned JSON artifacts under results/ (reuse _developer/jax_profiling/results/jit/.../*_v<version>.png generation logic).
• "How to reproduce" footer on each table cell pointing at the script that produced it.

Detailed implementation plan

Affected Repositories

• PyAutoLabs/autolens_profiling (primary, target).
• PyAutoLabs/autolens_workspace_developer (read-only — reference for the existing version-history PNG generation logic).

Work Classification

Docs + scripts work (workspace-style). Manual worktree setup.

Branch Survey

Repository	Current Branch	Dirty?
./autolens_profiling	main	clean

Suggested branch: feature/instrument-dashboard
Worktree root: ~/Code/PyAutoLabs-wt/instrument-dashboard/

Parallel-execution note

Phases 2 (#4) + 3 (#5) are queued on the same repo; Phase 4 touches READMEs + a new scripts/build_readme.py, which is disjoint from Phase 2's simulators/ and Phase 3's searches/ script work. The READMEs themselves overlap with Phases 2/3 (which both update the placeholder section READMEs they own). Run Phase 4 after Phases 1, 2, and ideally 3 ship so the auto-generation has all results to populate, and the section READMEs Phase 4 refreshes are not displaced mid-flight.

Headline table shape

Rows: astronomy instrument (e.g. HST imaging, Euclid imaging, JWST imaging, ALMA interferometer, SMA interferometer). Columns: supported model compositions (MGE, pixelization, Delaunay) plus a "JIT speedup vs NumPy" column. Cells: latest single-likelihood evaluation time (steady-state per-call, post-warmup), in milliseconds. Plus a footer link per cell to the producing script.

CPU / laptop GPU / HPC GPU split — open decision: either three columns per cell, or three stacked sub-tables. Implementer picks what reads best on GitHub.

Implementation Steps

1. Set up worktree manually:

   source admin_jammy/software/worktree.sh
   worktree_create instrument-dashboard autolens_profiling
   cd ~/Code/PyAutoLabs-wt/instrument-dashboard/autolens_profiling
   git checkout -b feature/instrument-dashboard

2. Author scripts/build_readme.py (new directory scripts/). Capabilities:

   • Scan results/**/*_summary_v<version>.json and group by section + instrument + model + hardware.
   • For each group, pick the latest version (semver-ish: 2026.5.14.2 > 2026.5.1.4).
   • Render markdown tables into the right README files between sentinel comments (<!-- BEGIN auto-table:section --> / <!-- END auto-table:section -->).
   • Render per-instrument version-history PNGs to results/dashboard/<section>/<instrument>_<model>_history.png.
   • Idempotent: same inputs → no diff.
   • CLI: python scripts/build_readme.py [--check] (the --check mode used by CI to fail if the working tree would diff after running).

3. Insert sentinel comments into every README that will host an auto-table. Hand-write the surrounding prose; let build_readme.py own the table region.

4. Smoke build_readme.py locally against whatever results/likelihood/ artifacts exist (Phase 1 + smoke runs).

5. "Cool extras" — the user prompt explicitly asks for: "Do a bit of deep research and thinking about other cool ways we can enhance a repo thats all about profiling and run times." Pick one or two to land in this PR, document the rest as future follow-ups in a "Roadmap" section of the top-level README. Candidates:

   • Regression-watch badge that turns red if any cell worsens by >X% vs the previous release.
   • Plotly-rendered interactive per-version timeline (embed via GitHub Pages if needed).
   • Flamegraph captures alongside the headline timing numbers.

6. Versioning policy — open decision. Default: keep the last 6 minor PyAutoLens releases in the graphs. Older *_summary_v<old>.json artifacts get moved to results/archive/ so they stay accessible but don't clutter the latest views. Implementer can adjust.

7. Open PR against main of PyAutoLabs/autolens_profiling. PR body: link this issue + summary of which READMEs got auto-tables + which "cool extras" landed + which were deferred.

Key Files

• New: autolens_profiling/scripts/build_readme.py.
• Edit: autolens_profiling/README.md (top-level — add headline auto-table region).
• Edit: autolens_profiling/likelihood/README.md + each likelihood/<type>/README.md (auto-table regions, hand-written surrounding prose).
• Edit: autolens_profiling/simulators/README.md (auto-table region; refresh after Phase 2 ships).
• Edit: autolens_profiling/searches/README.md + searches/nautilus/README.md (auto-table region; refresh after Phase 3 ships).
• New: autolens_profiling/results/dashboard/<section>/<instrument>_<model>_history.png (generated artifacts).

Open design questions for the implementer

1. CPU / laptop-GPU / HPC-GPU split: 3 columns per cell, or 3 stacked sub-tables? Open.
2. Versioning policy: keep last 6 minor releases, archive older? Default yes.
3. Which "cool extras" land in this PR? Pick 1–2, defer the rest.
4. Where do the version-history PNGs live? Default: results/dashboard/ (not under any section, since they aggregate).

F1 lesson applied

The auto-generation script reads results/**/*.json artifacts which are produced by the mirrored scripts. These mirror was re-mirrored in PR #3 to faithfully reflect clean origin/main of _developer. Do not regenerate the Phase 1 input datasets — those are pinned snapshots.

Out of scope

• CI / GitHub Actions automation (that's Phase 5, [issue to-be-created]). Phase 4 commits build_readme.py so Phase 5 can call it.
• Cross-instrument simulator parity (i.e. running every simulator at every instrument resolution). Phase 4 surfaces what's already produced, doesn't expand coverage.
• Implementing missing JIT paths for simulators (out of scope per Phase 2).
• Resolving the F1 upstream issues (PyAutoLens#514, _developer#68, _developer#69) — but show their cells as ERR or pending upstream in the dashboard rather than blanking them.

Original Prompt

Click to expand starting prompt

The full prompt content is preserved at PyAutoPrompt/issued/instrument_readme_dashboard.md after this issue is created.