fix: point_source priors aligned to simulator truth + refresh constants (#514)#14
Merged
Merged
Conversation
…ts (#514)
Root cause
----------
The `image_plane.py` / `source_plane.py` scripts build their lens model
via `af.GaussianPrior(mean=X, sigma=Y)` with the comment "centres prior-
median at the simulator truth". But the mean values for `mass.centre`
and `mass.ell_comps` had drifted away from the simulator's actual
truth values:
Script (pre-fix) | Simulator truth
-----------------------------------+-------------------------------
mass.centre.centre_* = 0.01 | (0.0, 0.0)
mass.ell_comps.ell_comps_0 = 0.01 | 0.05263158
mass.ell_comps.ell_comps_1 = 0.01 | 0.0
The simulator
(`autolens_workspace_developer/jax_profiling/dataset_setup/point_source.py`)
uses `al.mp.Isothermal(centre=(0, 0), einstein_radius=1.6,
ell_comps=al.convert.ell_comps_from(axis_ratio=0.9, angle=45°))` which
resolves to ell_comps ≈ (0.05263158, 0).
With drifted means, the PointSolver only finds 2-3 image positions
because the lens model has effectively zero ellipticity (the eager
`solver.solve` produced a doubly-imaged config rather than a quad),
while the data was simulated from a quad lens with 4 positions. The
2-position model + 4-position data → 2 unmatched data positions each
contributing ~3 arcsec residuals → chi² blows up → log_likelihood
crashes to -362 / -3599 (vs constants set at 0.07 / -294).
Diagnosed via direct instrumentation:
truth tracer model_positions: [[-1.04, -1.04], [0.44, 1.61],
[ 1.61, 0.44], [1.18, 1.18]]
data positions: [[-1.03, -1.09], [0.35, 1.63],
[ 1.57, 0.43], [1.25, 1.22]]
residuals at truth: ~0.05 arcsec per pair (consistent
with noise_map = 0.05)
pre-fix script model_positions: [[1.37, 0.99], [-0.95, -1.16], [inf, inf]]
(only 2 finite — quad collapses to double)
Fix
---
1. Update `GaussianPrior(mean=...)` values in both scripts to match the
simulator truth:
- `mass.centre.centre_*` → 0.0
- `mass.ell_comps.ell_comps_0` → 0.05263158
- `mass.ell_comps.ell_comps_1` → 0.0
Sigmas unchanged.
2. Refresh `EXPECTED_LOG_LIKELIHOOD_*` constants to match the post-fix
evaluation. The previous values were set on 2026-04-24 against an
earlier dataset+priors combination that has since been regenerated
(dataset committed in autolens_workspace_developer@f8a5cef on
2026-05-05); the new values reflect the current truth-aligned
evaluation:
- `EXPECTED_LOG_LIKELIHOOD_IMAGE_PLANE`: 0.0748 → 7.196577317761017
- `EXPECTED_LOG_LIKELIHOOD_SOURCE_PLANE`: -294 → -33788.35731127962
The source-plane number is dramatically more negative than the
image-plane one because the source-plane chi² formula weights
residuals by `magnifications² / noise²`. The simulator truth puts
the source close to a caustic (quad config near the threshold for
image creation), where magnifications can swing by 10-100x with
small parameter perturbations. This is documented as a sensitivity
characteristic in the refreshed constant's comment block.
Validation
----------
Both scripts exit 0 with all three-tier regression assertions PASSING:
eager ≡ JIT, JIT ≡ vmap, and the refreshed regression constants
match. The bug filed at PyAutoLabs/PyAutoLens#514 can be closed
(the drift is fully explained by stale prior means in the SCRIPT, not
an upstream behaviour regression).
Refs
----
- Filed at PyAutoLabs/PyAutoLens#514
- Upstream source-of-truth scripts at
`autolens_workspace_developer/jax_profiling/jit/point_source/{image_plane,source_plane}.py`
have the same bug; follow-up PR there to mirror.
- Surfaced by today's full-script run against the canonical
autolens_profiling repo (Phase 5 ship validation).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes PyAutoLens#514 —
point_source/{image_plane,source_plane}.pyreturning massively-drifted log-likelihoods (-362vs expected0.07;-3599vs expected-294). The drift was not an upstream PyAutoLens regression — it was stale prior means in the SCRIPTS that no longer matched the simulator truth.Both scripts now exit 0 with all regression assertions passing.
Root cause
The scripts build their model with
af.GaussianPrior(mean=X, sigma=Y)and a comment claiming "mean = simulator truth". The means had drifted away from the actual simulator truth values:mass.centre.centre_* = 0.01(0.0, 0.0)mass.ell_comps.ell_comps_0 = 0.010.05263158mass.ell_comps.ell_comps_1 = 0.010.0The simulator (
autolens_workspace_developer/jax_profiling/dataset_setup/point_source.py) usesal.mp.Isothermal(centre=(0, 0), einstein_radius=1.6, ell_comps=al.convert.ell_comps_from(axis_ratio=0.9, angle=45°))which resolves toell_comps ≈ (0.05263158, 0).With drifted means, the
PointSolverfinds only 2 image positions (effectively zero-ellipticity → doubly-imaged config) while the data has 4 positions (quad-image config from the truth lens). The 2 unmatched data positions each contribute ~3 arcsec residuals → chi² explodes → log_likelihood crashes.Diagnosed via direct instrumentation:
Fix
EXPECTED_LOG_LIKELIHOOD_*constants to match the post-fix evaluation:IMAGE_PLANE:0.0748→7.196577317761017SOURCE_PLANE:-294→-33788.35731127962The source-plane number is dramatically more negative because that chi² formula weights residuals by
magnifications² / noise²— the simulator truth puts the source near a caustic where magnifications can swing by 10-100× with small parameter perturbations. Documented as a sensitivity characteristic in the refreshed constant's comment block.Validation
Both scripts exit 0; all three-tier regression assertions (eager ≡ JIT, JIT ≡ vmap, regression-constant) PASS.
Refs
autolens_workspace_developer/jax_profiling/jit/point_source/{image_plane,source_plane}.pyhave the same bug; follow-up PR there should mirror this fix.autolens_profilingrepo (Phase 5 ship validation).🤖 Generated with Claude Code