Overview
Phase 2 of the autolens_profiling z_feature roadmap — mirror the 6 simulator-profiling scripts at autolens_workspace_developer/jax_profiling/simulators/ (~2,040 LOC across imaging, interferometer, point_source, cluster, group, multi) into the new repo at simulators/. _developer stays the source of truth — the originals do not move. This phase also writes the section README that Phase 4's dashboard will read from. Datasets are produced by these scripts, not consumed, so there's no dataset-mirroring decision (in contrast to Phase 1).
Plan
- Mirror 6 simulator scripts from
_developer/jax_profiling/simulators/ into autolens_profiling/simulators/, preserving filenames and the run-time narrative each script prints.
- Rewrite path references:
_workspace_root / "jax_profiling" / "dataset" → _workspace_root / "dataset" (matches Phase 1's top-level layout), and _workspace_root / "jax_profiling" / "results" / "simulators" → _workspace_root / "results" / "simulators".
- Rewrite
_script_dir.parents[2] → _script_dir.parents[1] (one fewer level up because new layout is simulators/<script>.py vs jax_profiling/simulators/<script>.py).
- Update docstring example commands (
python jax_profiling/simulators/imaging.py → python simulators/imaging.py).
- Write
simulators/README.md (replacing the Phase 0 placeholder) — what each simulator profiles (construction time, dataset write time, NumPy vs JIT where applicable), how to read the timing JSON, the versioned artifact convention, and a headline-run-times table placeholder that Phase 4 auto-fills.
- Where a simulator's JIT path isn't implemented yet (the prompt explicitly flags this): emit
jit: not-yet-implemented in the result JSON and link a TODO to the upstream issue in the README. Out of scope: actually implementing the missing JIT paths — that's a PyAutoLens-library task.
- Smoke-check: run each of the 6 scripts end-to-end and confirm both the simulated dataset and the timing JSON+PNG land at the expected paths.
Detailed implementation plan
Affected Repositories
PyAutoLabs/autolens_profiling (primary, target) — currently at aa131a7 on main after F1 re-mirror.
PyAutoLabs/autolens_workspace_developer (read-only source-of-truth, no changes).
Work Classification
Scripts mirroring (workspace-style). Same routing pattern as Phase 1 — no /start_workspace template fit, manual worktree setup.
Branch Survey
| Repository |
Current Branch |
Dirty? |
./autolens_profiling |
main |
clean |
Suggested branch: feature/simulators-mirror
Worktree root: ~/Code/PyAutoLabs-wt/simulators-mirror/
Path-rewrite patterns (uniform across all 6 scripts)
| Pattern |
Before |
After |
| Dataset write path |
_workspace_root / "jax_profiling" / "dataset" / <type> / <name> |
_workspace_root / "dataset" / <type> / <name> |
| Results write path |
_workspace_root / "jax_profiling" / "results" / "simulators" |
_workspace_root / "results" / "simulators" |
| Workspace-root walk-up |
_script_dir.parents[2] |
_script_dir.parents[1] |
| Docstring example |
python jax_profiling/simulators/<name>.py |
python simulators/<name>.py |
The artifact filename convention is unchanged: <simulator_name>_summary_v<al.__version__>.{json,png} (flat under results/simulators/, no subfolders).
Implementation Steps
-
Set up worktree manually (autolens_profiling has no /start_workspace template):
source admin_jammy/software/worktree.sh
worktree_create simulators-mirror autolens_profiling
cd ~/Code/PyAutoLabs-wt/simulators-mirror/autolens_profiling
git checkout -b feature/simulators-mirror
-
Create the section folder skeleton: simulators/ already exists from Phase 0 with the placeholder README — nothing new to create at the folder level.
-
Copy the 6 scripts from ~/Code/PyAutoLabs-wt/simulators-mirror/autolens_workspace_developer/jax_profiling/simulators/{imaging,interferometer,point_source,cluster,group,multi}.py (worktree's clean origin/main copy, NOT the canonical — F1 taught us why) into simulators/.
-
Apply the 4 path-rewrite patterns above to each script.
-
Replace simulators/README.md placeholder with the real section README — what each simulator profiles, how to read the timing artifacts, versioned artifact convention, headline-run-times table placeholder.
-
JIT-placeholder audit: for each script, check whether the JIT path is exercised end-to-end. If not, ensure the result JSON includes a jit: not-yet-implemented entry and the README has a TODO: upstream issue #... line (filing the upstream issue is part of this task if not already open).
-
Smoke each of the 6 scripts from the repo root and verify:
- Simulated datasets land at
dataset/<type>/<name>/.
- Timing JSON+PNG land at
results/simulators/<simulator>_summary_v<al.__version__>.{json,png}.
- Internal regression assertions (if any) pass; if any drift in the way F1 surfaced, file or update the upstream issue rather than refreshing the constant.
-
Open PR against main of PyAutoLabs/autolens_profiling. PR body: link this issue + summary of which simulators moved + which JIT paths are stubbed.
Key Files
- Source (read-only):
autolens_workspace_developer/jax_profiling/simulators/{imaging,interferometer,point_source,cluster,group,multi}.py (~2,040 LOC).
- New:
autolens_profiling/simulators/{imaging,interferometer,point_source,cluster,group,multi}.py — mirrored scripts.
- Edit:
autolens_profiling/simulators/README.md — replace Phase 0 placeholder.
- Edit:
autolens_profiling/.gitignore — already covers dataset/cluster/, dataset/group/, dataset/multi/ as regenerable variants from Phase 1; no changes expected here.
Decisions inherited from Phase 1
- Datasets live at top-level
dataset/<type>/<name>/, not under simulators/dataset/. Phase 1 locked this in for likelihood-input data; Phase 2 simulator outputs reuse the same folder for the regenerable variants (cluster/, group/, multi/, and any new simple variants the simulators produce). Already covered by .gitignore rules in Phase 1.
F1 lesson applied: copy from clean origin/main, not the canonical
Phase 1 inadvertently copied from the dirty _developer canonical checkout, which carried in-progress modifications. F1 (autolens_workspace_developer#67) surfaced and fixed it via autolens_profiling#3. Phase 2 must copy from the worktree's _developer (clean origin/main) directly.
Out of scope
- Phase 3+ of the autolens_profiling z_feature.
- Actual JAX JIT implementation for simulators where it's missing — that's PyAutoLens / PyAutoArray library work.
- Re-running or refreshing Phase 1's input datasets (those are pinned to the snapshot the user prepared).
- Triaging the 3 open F1 upstream issues (PyAutoLens#514, _developer#68, _developer#69).
Original Prompt
Click to expand starting prompt
The full prompt content is preserved at PyAutoPrompt/issued/simulators_mirror.md after this issue is created. See also the z_features tracker at PyAutoPrompt/z_features/autolens_profiling.md.
Overview
Phase 2 of the autolens_profiling z_feature roadmap — mirror the 6 simulator-profiling scripts at
autolens_workspace_developer/jax_profiling/simulators/(~2,040 LOC across imaging, interferometer, point_source, cluster, group, multi) into the new repo atsimulators/._developerstays the source of truth — the originals do not move. This phase also writes the section README that Phase 4's dashboard will read from. Datasets are produced by these scripts, not consumed, so there's no dataset-mirroring decision (in contrast to Phase 1).Plan
_developer/jax_profiling/simulators/intoautolens_profiling/simulators/, preserving filenames and the run-time narrative each script prints._workspace_root / "jax_profiling" / "dataset"→_workspace_root / "dataset"(matches Phase 1's top-level layout), and_workspace_root / "jax_profiling" / "results" / "simulators"→_workspace_root / "results" / "simulators"._script_dir.parents[2]→_script_dir.parents[1](one fewer level up because new layout issimulators/<script>.pyvsjax_profiling/simulators/<script>.py).python jax_profiling/simulators/imaging.py→python simulators/imaging.py).simulators/README.md(replacing the Phase 0 placeholder) — what each simulator profiles (construction time, dataset write time, NumPy vs JIT where applicable), how to read the timing JSON, the versioned artifact convention, and a headline-run-times table placeholder that Phase 4 auto-fills.jit: not-yet-implementedin the result JSON and link aTODOto the upstream issue in the README. Out of scope: actually implementing the missing JIT paths — that's a PyAutoLens-library task.Detailed implementation plan
Affected Repositories
PyAutoLabs/autolens_profiling(primary, target) — currently ataa131a7on main after F1 re-mirror.PyAutoLabs/autolens_workspace_developer(read-only source-of-truth, no changes).Work Classification
Scripts mirroring (workspace-style). Same routing pattern as Phase 1 — no
/start_workspacetemplate fit, manual worktree setup.Branch Survey
./autolens_profilingSuggested branch:
feature/simulators-mirrorWorktree root:
~/Code/PyAutoLabs-wt/simulators-mirror/Path-rewrite patterns (uniform across all 6 scripts)
_workspace_root / "jax_profiling" / "dataset" / <type> / <name>_workspace_root / "dataset" / <type> / <name>_workspace_root / "jax_profiling" / "results" / "simulators"_workspace_root / "results" / "simulators"_script_dir.parents[2]_script_dir.parents[1]python jax_profiling/simulators/<name>.pypython simulators/<name>.pyThe artifact filename convention is unchanged:
<simulator_name>_summary_v<al.__version__>.{json,png}(flat underresults/simulators/, no subfolders).Implementation Steps
Set up worktree manually (autolens_profiling has no
/start_workspacetemplate):Create the section folder skeleton:
simulators/already exists from Phase 0 with the placeholder README — nothing new to create at the folder level.Copy the 6 scripts from
~/Code/PyAutoLabs-wt/simulators-mirror/autolens_workspace_developer/jax_profiling/simulators/{imaging,interferometer,point_source,cluster,group,multi}.py(worktree's clean origin/main copy, NOT the canonical — F1 taught us why) intosimulators/.Apply the 4 path-rewrite patterns above to each script.
Replace
simulators/README.mdplaceholder with the real section README — what each simulator profiles, how to read the timing artifacts, versioned artifact convention, headline-run-times table placeholder.JIT-placeholder audit: for each script, check whether the JIT path is exercised end-to-end. If not, ensure the result JSON includes a
jit: not-yet-implementedentry and the README has aTODO: upstream issue #...line (filing the upstream issue is part of this task if not already open).Smoke each of the 6 scripts from the repo root and verify:
dataset/<type>/<name>/.results/simulators/<simulator>_summary_v<al.__version__>.{json,png}.Open PR against
mainofPyAutoLabs/autolens_profiling. PR body: link this issue + summary of which simulators moved + which JIT paths are stubbed.Key Files
autolens_workspace_developer/jax_profiling/simulators/{imaging,interferometer,point_source,cluster,group,multi}.py(~2,040 LOC).autolens_profiling/simulators/{imaging,interferometer,point_source,cluster,group,multi}.py— mirrored scripts.autolens_profiling/simulators/README.md— replace Phase 0 placeholder.autolens_profiling/.gitignore— already coversdataset/cluster/,dataset/group/,dataset/multi/as regenerable variants from Phase 1; no changes expected here.Decisions inherited from Phase 1
dataset/<type>/<name>/, not undersimulators/dataset/. Phase 1 locked this in for likelihood-input data; Phase 2 simulator outputs reuse the same folder for the regenerable variants (cluster/,group/,multi/, and any new simple variants the simulators produce). Already covered by.gitignorerules in Phase 1.F1 lesson applied: copy from clean origin/main, not the canonical
Phase 1 inadvertently copied from the dirty
_developercanonical checkout, which carried in-progress modifications. F1 (autolens_workspace_developer#67) surfaced and fixed it via autolens_profiling#3. Phase 2 must copy from the worktree's_developer(cleanorigin/main) directly.Out of scope
Original Prompt
Click to expand starting prompt
The full prompt content is preserved at
PyAutoPrompt/issued/simulators_mirror.mdafter this issue is created. See also the z_features tracker atPyAutoPrompt/z_features/autolens_profiling.md.