Repository navigation
Description
Overview
Multi-plane ray tracing (tracer_util.traced_grid_2d_list_from and its final-plane-normalised scaling factors) is validated only against itself or pinned literals produced by the same code. PyAutoLens#480 survived four months in exactly this machinery and was finally settled by an independent ray-traced Jacobian; the control arm exposed a second bug (NumPy Richardson Hessian step). This task builds permanent, genuinely independent cross-validation arms and asserts the implementation agrees with the published formalism. Phase 1 = library test module (this issue); phase 2 = workspace guide extension, filed separately once the conventions are settled.
Bundle: ci-smoke — bundle 2 (orchestration only — this issue and its PR stand alone).
Plan
- Fix the formalism from the literature (SEF 1992 §9.1 eqs 9.6/9.7b; Narayan & Bartelmann inspirehep 419263 eqs 55/60; McCully+2014 Jacobian recursion) and state it in the module docstring, with the two convention traps:
deflections_between_planes_fromis the final-plane-scaled difference θ_i − θ_j (not the physical deflection at plane j), and truncating a tracer to planes ≤ j changes every β (the 1.86-vs-27.9 example from bug: PointSolver magnification filter uses last-plane magnification instead of plane_redshift #480). - Build independent oracles sharing no code with
tracer_util: astropyPlanck15angular-diameter distances for β_ij; the SEF recursion written from the paper using each galaxy's single-planedeflections_yx_2d_from; a central-difference Jacobian θ→β_j with μ=1/det J (h-stable 1e-4…1e-7); the McCully Jacobian recursion. - Degenerate reductions first: all mass in one plane reduces to the single-plane analytic result; β=1 at z_j=z_final, β=0 at z_i=z_j; a deflector at plane j leaves the mapping to plane j unchanged; explicit two-plane formula.
- Analytic multi-plane arm: two aligned SIS on the axis (closed-form positions, double Einstein ring radii).
- General 3–4 plane elliptical case: traced positions,
deflections_between_planes_from, magnification / convergence / shear at every plane, across all arms. - Any disagreement is a finding posted here — never a loosened tolerance. The known NumPy Richardson-step bug (
draft/bug/autogalaxy/lenscalc_numpy_hessian_step_is_too_coarse.md) is recorded as a strict xfail, not widened. - File the phase-2 prompt (workspace guide) with the settled conventions.
Detailed implementation plan
Work Classification
Library (phase 1). Workspace guide follows as phase 2 (separate prompt/issue).
Affected Repositories
- PyAutoLens (primary)
- PyAutoGalaxy (claimed in the worktree; no edits expected)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./PyAutoLens | main | clean |
| ./PyAutoGalaxy | main | clean |
Suggested branch: feature/multi-plane-cross-validation
Worktree root: ~/Code/PyAutoLabs-wt/ci-smoke-bundle-2/ (shared bundle worktree)
Implementation Steps
- New module
test_autolens/lens/test_multi_plane_cross_validation.py— numpy only, no jax import (unit-test contract; the JAXjacfwdarm moves to phase 2 on the workspace side). - Module docstring: the formalism and conventions (above), with citations.
- Oracle helpers (module-level):
_astropy_beta(z_i, z_j, z_final)viaastropy.cosmology.Planck15.angular_diameter_distance_z1z2— first test asserts it matchesag.cosmo.Planck15().scaling_factor_between_redshifts_fromto rtol 1e-6 (check the hand-rolled cosmology's parameters atautogalaxy/cosmology/model.py:439-478match astropy's Planck15 incl. Tcmb0/radiation; if they disagree beyond 1e-5, stop and report — do not tune)._trace_from_paper(planes, grid, betas)— SEF recursion._jacobian_via_central_difference/_magnification_via_jacobian(pattern:test_autolens/point/triangles/test_solver_multi_plane.py:206)._jacobian_via_recursion— McCully recursion with U_i from central differences of the profile deflections. - Degenerate-reduction tests, then the SIS analytic arm, then the general 3–4 plane case: for every plane j assert
traced_grid_2d_list_from(...)[j]== oracle (rtol 1e-10);Tracer.deflections_between_planes_from(plane_i=0, plane_j=j)== θ − θ_j;ag.LensCalc.from_tracer(tracer, plane_j=j).magnification_2d_via_hessian_fromvs Jacobian oracles (rtol 1e-3); convergence/shear at plane j from the recursion Jacobian (κ = 1 − tr A/2, γ from A). - Where the NumPy Richardson Hessian fails (expected):
pytest.xfail(strict=True, reason=...)on that single assertion referencing the draft bug. - No library source edits expected. Any other disagreement → stop, report verbatim on this issue.
- File
PyAutoMind/draft/docs/autolens_workspace/multi_plane_cross_validation_guide.md(extend the existingscripts/guides/advanced/multi_plane.pywith a__Cross Validation__section, the 1.86-vs-27.9 worked example, the JAX arm underENV: jax).
Key Files
test_autolens/lens/test_multi_plane_cross_validation.py— newautolens/lens/tracer_util.py:174,autolens/lens/tracer.py:921— implementation under test (read-only)PyAutoGalaxy/autogalaxy/cosmology/model.py:311(scaling factor),autogalaxy/operate/lens_calc.py:167,382,417,624— read-onlytest_autolens/point/triangles/test_solver_multi_plane.py:206— Jacobian oracle pattern (untouched)PyAutoMind/complete/2026/08/point-solver-magnification-plane-redshift.md— bug: PointSolver magnification filter uses last-plane magnification instead of plane_redshift #480 evidence
Verification
pytest test_autolens/lens/test_multi_plane_cross_validation.py -qgreen bar the strict xfail;pytest test_autolens/lens test_autolens/point -qcounts unchanged.
Original Prompt
Click to expand starting prompt
Cross-validate multi-plane ray tracing
Type: test
Target: PyAutoLens
Repos:
- PyAutoGalaxy
- PyAutoLens
- autolens_workspace
Themes: - ci-smoke
- notebooks
Difficulty: large
Autonomy: supervised
Priority: high
Status: formalised
Model: Fable (research-grade — paper reading + derivation + narrative guide prose)
Filed: 2026-08-27
Single-plane lensing quantities in this stack are cross-validated two independent ways.
Multi-plane ray tracing is not validated against anything. Build that, then publish it as
a @autolens_workspace guide.
The analogue that already exists
@PyAutoGalaxy test_autogalaxy/profiles/mass/total/test_isothermal.py::test__shear_yx_2d_from__matches_via_hessian
is the pattern to copy:
shear_analytic = mp.shear_yx_2d_from(grid=grid)
shear_via_hessian = LensCalc.from_mass_obj(mp).shear_yx_2d_via_hessian_from(grid=grid)
np.testing.assert_allclose(shear_analytic, shear_via_hessian, rtol=1e-3, atol=1e-6)Its docstring states the value plainly: the two paths "must agree to within
finite-difference accuracy", and the cross-check "guards against either path silently
drifting (e.g. a sign flip, a column swap, or a rotation-frame mistake)". A closed-form
formula and a numerical derivative of the deflections are wrong in different ways, so
agreement is evidence, where either alone is only self-consistency.
Multi-plane has no such second opinion. tracer_util.traced_grid_2d_list_from and
its scaling factors are the only implementation of the multi-plane recursion, and every
downstream quantity — deflections_between_planes_from, plane-bound magnification,
plane_index_via_redshift_from — is tested against itself or against pinned literals
produced by the same code.
Why now: this cost us four months
PyAutoLens#480 (fixed 2026-08-27, complete/2026/08/point-solver-magnification-plane-redshift.md)
was a plane-mismatch in exactly this machinery that survived four months in a core path.
Two things about how it was finally settled motivate this task:
- The ray-traced Jacobian was the oracle that resolved it. Central-differencing
theta -> beta_jthroughtraced_grid_2d_list_fromand takingmu = 1/det(J)shares no
code with the Hessian path, and it agreed with exact JAX autodiff to 1.8e-09. That
one-off cross-check is the seed of this task; it should be a permanent, general
fixture rather than something rebuilt by hand each time. - The control arm found a second, unrelated bug. Running the same comparison at the
last plane exposedLensCalc._hessian_via_richardson's hardcoded 0.01" step being
off by 122% with the wrong sign on three of four points
(draft/bug/autogalaxy/lenscalc_numpy_hessian_step_is_too_coarse.md). A systematic
cross-validation would have caught that years ago.
What to build
Compute the same multi-plane quantities by several genuinely independent routes and
assert they agree. Independence is the whole point: a route that reuses
traced_grid_2d_list_from is not a second opinion.
Candidate routes, to be settled during the task:
- The implementation —
tracer.traced_grid_2d_list_from/deflections_between_planes_from. - A from-scratch multi-plane lens equation, written directly from a paper's equations
against explicit angular-diameter distances, deliberately NOT sharing the codebase's
scaling-factor helper. This is the primary independent arm. - Numerical Jacobian of
theta -> beta_j,mu = 1/det(J)(validated on bug: PointSolver magnification filter uses last-plane magnification instead of plane_redshift #480; stable
across h from 1e-4 to 1e-7). - Exact JAX autodiff (
jacfwd), which needsjax_enable_x64to be meaningful —
float32 gave 1.6e-02 residuals where float64 gave 1.8e-09. - Degenerate-case reductions, the cheapest and sharpest checks:
- a multi-plane tracer with all mass in one plane must reduce to the single-plane
result, where the existing analytic closed forms apply; - a two-plane tracer must match the two-plane formulas exactly;
- a deflector sitting at plane j must not affect the mapping to plane j (verified
incidentally during bug: PointSolver magnification filter uses last-plane magnification instead of plane_redshift #480: mu at z=1.0 was identical with and without source_0's mass).
- a multi-plane tracer with all mass in one plane must reduce to the single-plane
- Analytic multi-plane configurations where a closed form exists (e.g. aligned
isothermal spheres giving a double Einstein ring) — the strongest arm where available.
Quantities to cover: traced positions per plane, deflections_between_planes_from,
magnification at each plane, convergence/shear at each plane, and time delays if
draft/feature/autolens/multi_plane_time_delays.md has landed by then.
Papers — pair against the literature, do not re-derive from the code
The point is to check the implementation against the published formalism, so the
equations must come from the papers, not from reading our source and transcribing it.
The one certain starting point: the codebase already cites
https://inspirehep.net/literature/419263 for its equations 55 and 60 (Hessian and
magnification) in autogalaxy/operate/lens_calc.py. Start there, since it is the
reference the current code claims to implement.
Then survey the standard multi-plane formalism. Candidate references to locate and
verify — treat this list as leads to confirm, not as established citations:
Blandford & Narayan (1986); Schneider, Ehlers & Falco, Gravitational Lenses;
Schneider (2014) on multi-plane lensing; McCully et al. (2014) on multi-plane algorithms;
Petkova, Metcalf & Giocoli (2014) (GLAMER). Cross-reading lenstronomy's multi-plane
implementation is also worthwhile as a second independent codebase.
Record which convention each source uses. The likely source of confusion — and worth
settling explicitly early — is deflection normalisation: our
deflections_between_planes_from returns traced_grids[i] - traced_grids[j], i.e.
scaled against the final plane, so it is NOT the physical deflection at plane j in the
convention some papers use. Concretely, during #480 the obvious cross-check of truncating
the tracer to planes <= j gave 1.86 where the correct answer was 27.9. That is not a bug,
it is a convention mismatch — and it is exactly the kind of thing this task must
document rather than trip over.
Deliverable: a workspace guide
This ends as a @autolens_workspace guide, not only a test module. Multi-plane
ray-tracing conventions are exactly the thing users get wrong and we have nothing
explaining them. Route the authorship through /workspace for placement, audience
register and the sibling guide to mirror.
The guide should carry: the multi-plane lens equation as the papers state it, how our
scaling factors map onto it, the convention traps above (with the 1.86-vs-27.9 worked
example), and the cross-validation itself as runnable, inspectable code — a reader
should be able to check the library themselves.
Split at start_dev time: the library test module and the workspace guide are one prompt
here but likely two PRs, library first.
Suggested scope
- Read the papers, fix the notation, write down the multi-plane lens equation and the
magnification/Jacobian at an arbitrary plane in the paper's convention. - Map our implementation onto it and document every convention difference explicitly.
- Implement the independent arms as reusable fixtures.
- Add the cross-validation tests, covering the degenerate reductions first (cheap, sharp)
and the general multi-plane case second. - Report any disagreement as a finding, not as a test to loosen. bug: PointSolver magnification filter uses last-plane magnification instead of plane_redshift #480 and the Richardson
step bug both say the implementation is the more likely party to be wrong. - Author the workspace guide via
/workspace.
Activity
Jammy2211 commentedon Aug 29, 2026
Library PR Created: #715
Tests: full
test_autolens/suite 569 passed, 1 xfailed (strict, the recordedLensCalc._hessian_via_richardsonstep-size finding); the new module alone is 16 passed, 1 xfailed. Cross-validation findings are in the PR body.Merge pull request #715 from PyAutoLabs/feature/multi-plane-cross-val…
Jammy2211 commentedon Aug 29, 2026
Shipped — 2026-08-29
PRs
Summary
Added
test_autolens/lens/test_multi_plane_cross_validation.py: independent oracles (astropy Planck15 distances, the SEF §9.1 recursion written from the paper, a central-difference ray-traced Jacobian, the McCully+14 Jacobian recursion) and cross-validation of traced positions,deflections_between_planes_from, magnification, convergence and shear at every plane, plus degenerate reductions and an analytic two-SIS arm. 16 passed / 1 strict xfail; fulltest_autolens/569 passed. CI green on py3.12, py3.13 and no-jax legs.Session Notes
LensCalc._hessian_via_richardsonhardcoded 0.01" step reproduces the bug: PointSolver magnification filter uses last-plane magnification instead of plane_redshift #480 control-arm defect exactly (102–122% off, sign flipped, on the bug: PointSolver magnification filter uses last-plane magnification instead of plane_redshift #480 fixture) while agreeing to 1.3e-8 at the intermediate plane. Pinned asxfail(strict=True); tracked bydraft/bug/autogalaxy/lenscalc_numpy_hessian_step_is_too_coarse.md.guides/advanced/multi_plane.pywith a cross-validation section + the JAXjacfwdarm) is filed asPyAutoMind/draft/docs/autolens_workspace/multi_plane_cross_validation_guide.md.Jammy2211 commentedon Aug 29, 2026
Phase 2 (workspace guide) PR: PyAutoLabs/autolens_workspace#517
test: cross-validate multi-plane ray tracing against independent orac…