Repository navigation
Description
Overview
Phase 2 of the mass-field epic (PyAutoMind draft/feature/autogalaxy/mass_field_epic.md). Phase 1 shipped ag.MassField in PyAutoGalaxy (PyAutoGalaxy#621, merged 2026-09-17): a standalone, MassProfile-only container for external shear, mass sheets and external potentials that a Galaxies plane can hold beside galaxies. This task teaches PyAutoLens about it: Tracer(galaxies, fields=None) with its own fields argument merged into the planes only, the analysis fields model slot, COOLEST 1:1 both ways (legacy peel kept), the line-of-sight sampler emitting sheets as fields, and al.model_util.mass_field_from with the ExternalPotential centre tied to the lens mass centre. Hard invariants: tracer.galaxies never contains a field; legacy galaxy-attached sheets keep working with no warning and byte-identical COOLEST export; no prior-config change; a galaxy-attached model's PyAutoFit identifier is pinned.
Plan
- Add
fields=toTracer: stored astracer.fields, validated onredshift, merged with galaxies only where planes are built (galaxies_ascending_redshiftstays galaxies-only; a new members-ascending helper feedsplane_redshifts/planes);total_planescounts field-only planes;sliced_tracer_fromand the dict round trip carry fields; the JAX walker registers fields. - Keep per-galaxy surfaces galaxy-only:
galaxy_image_2d_dict_fromand every consumer that walks plane members (fit plane images, subplots, aggregator tables) skipMassFieldmembers. - Fold
instance.fieldsinto the tracer inAnalysisLens.tracer_via_instance_fromand the aggregator tracer builder, beside theextra_galaxies/scaling_galaxiesfolds. - COOLEST:
tracer.fieldsexport asMassFieldentities;from_coolestreturns them asal.MassFieldinfields; legacy peel of galaxy-attached sheets unchanged and regression-tested. - LOS sampler: sheets become
MassFields returned separately from halo galaxies, with a shim for the old single-list return. model_util.mass_field_from(lens, shear=True, mass_sheet=False, potential=False, redshift=None)with shared-prior centre ties.- Tests for each bullet, the identifier pin, and API docs; re-export
al.MassField.
Detailed implementation plan
Work Classification
Library (PyAutoLens only). Feature Agent sized it large and suggested splitting; the epic's human ruling (2026-09-17) sized phase 2 as one prompt, so it stays whole. Workspaces follow as phases 3–5.
Affected Repositories
- PyAutoLens (primary). Depends on PyAutoGalaxy
main≥ 33714b80 (ag.MassField); CI runs the same-named library branch, so no release is needed.
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
PyAutoLens (main @ 223132c) |
main | clean |
Remote branches: main, dev_Q, two archive/condemned/* — none touch lens/, analysis/ or interop/. No active.md task claims PyAutoLens.
Suggested branch: feature/mass-field-integration
Worktree root (local-dev): ~/Code/PyAutoLabs-wt/mass-field-integration/ (a web session works on its clone instead)
Facts established on main
- Identifier pin, captured before any edit:
af.Collection(galaxies=af.Collection(lens=af.Model(al.Galaxy, redshift=0.5, mass=af.Model(al.mp.Isothermal), shear=af.Model(al.mp.ExternalShear)), source=af.Model(al.Galaxy, redshift=1.0, bulge=af.Model(al.lp.Sersic))))→fef2697b5c32ba56bb18a7baecb7b0f6. Must hold on the branch. Tracer.__init__(galaxies, cosmology=None)runs_validate_galaxies(duck-typed onredshift) and_warn_if_no_light_is_behind_any_mass, then storesself.galaxies.galaxies_ascending_redshiftsorts unless any redshift is traced;plane_redshifts/planescalltracer_util.plane_redshifts_from/planes_fromon that list — both are duck-typed on.redshiftand.has, so aMassFieldmember works there unchanged.Tracerhas no customto_dict;autonerves.dictableserialises__init__arguments, so afieldsargument stored asself.fieldsround-trips once it exists (test_tracer.py already hasfrom_json/output_to_jsontests to mirror).Tracer.galaxy_image_2d_dict_fromiterates planes (for galaxies in self.planes: for galaxy in galaxies), so without a filter a field becomes a dict key with a zeros image and reachesFitImaging.galaxy_model_image_dict, the galaxies subplots andmodel_images_of_planes_list. The prompt's assumption that per-galaxy surfaces only walktracer.galaxiesis false here — handle it (step 1e).autolens/jax/registration.pywalkstracer.galaxiesand registersGalaxywithno_flatten=("redshift",)(unlike the autogalaxy walker); it is a deliberate copy, so patch it, do not share.AnalysisLens.tracer_via_instance_from(autolens/analysis/analysis/lens.py:83) foldsextra_galaxies/scaling_galaxies;AnalysisImaging,AnalysisInterferometer(viaAnalysisDataset) andAnalysisPointall inherit it, so one edit covers the three.autolens/aggregator/tracer.py:~90andautolens/analysis/result.py:254build tracers independently.- COOLEST (
autolens/interop/coolest.py):to_coolestaccepts aTraceror a list, peelsExternalShear/MassSheetoff each galaxy intolazy.MassField(f"mass_field_{i}", z, mass_model=...);from_coolestturns every entity,MassFieldincluded, into aGalaxy.ExternalPotentialis not in the peel today and has no COOLEST profile. - LOS:
LOSSampler.galaxies_fromreturns one list:ag.Galaxy(redshift=z_cen, mass=halo)per halo (line ~761) plusag.Galaxy(redshift=..., mass_sheet=ag.mp.MassSheet(kappa=kappa_neg))per plane (line ~789). Onlytest_autolens/lens/test_los.pycalls it in-repo; the workspacelos_halosscripts call it too (outside this repo — keep the shim). al.model_utilisautolens/analysis/model_util.py, importingmge_model_frometc. from autogalaxy;al.MassFieldis not yet re-exported.
Implementation Steps
-
Tracer(galaxies, fields=None, cosmology=None)—autolens/lens/tracer.py
a. Signature gainsfields: Optional[List[ag.MassField]] = None;self.fields = list(fields) if fields is not None else []. Validate with a_validate_fieldsmirroring_validate_galaxies(list/tuple oraf.ModelInstance; each entry hasredshift; messages nameMassField, e.g.fields must be an iterable of MassField objects, e.g. Tracer(galaxies=[lens, source], fields=[field])). Extend_warn_if_no_light_is_behind_any_massto take galaxies + fields (fields are mass-bearing, never light-bearing).
b. Add@property members_ascending_redshift(name it clearly;galaxies_ascending_redshiftkeeps its meaning and return value — galaxies only):sorted(self.galaxies + self.fields, key=redshift)with the same traced-redshift bypass (tracer_util._any_traced).plane_redshiftsandplanesuse it, so a plane (ag.Galaxies) may hold aMassField. Audit every other use ofgalaxies_ascending_redshiftin the file (grid_2d_at_redshift_from~582,time_delays_from~1034,plane_index_via_redshift_from) and switch those that compute lensing quantities to the members list; leave galaxy-only surfaces on galaxies.
c.total_planesfollows fromplane_redshifts, so a field-only plane counts.upper_plane_index_with_light_profile,image_2d_list_from,traced_grid_2d_list_from,deflections_*,convergence_2d_from,potential_2d_from,time_delays_from,has,cls_list_from,perform_inversion,plane_indexes_with_pixelizations, the positions solver, magnification and critical curves all flow through planes — do not change them, but write one test per surface proving a field contributes (mass) or is invisible (light).
d.sliced_tracer_from(lens_galaxies, line_of_sight_galaxies, source_galaxies, planes_between_lenses, cosmology=None, fields=None): snap each field's redshift to the nearest sliced plane exactly like the line-of-sight galaxies, and passfields=through to the constructor.
e. Galaxy-only per-galaxy surfaces. Ingalaxy_image_2d_dict_from, skip members that are notag.Galaxy(isinstance(galaxy, ag.MassField)→ continue) so fields never become dict keys. Then audit every consumer that walks plane members and indexes that dict:autolens/imaging/fit_imaging.py~306–330 (model_images_of_planes_listloopsfor galaxy in galaxiesover planes),galaxy_model_image_dict,subplot_galaxies_images/ the fit plotters,lens/to_inversion.py(usestracer.galaxies, fine), the aggregator galaxy tables,extract_attributes_of_galaxies. Each one either skipsMassFieldmembers or already only readstracer.galaxies. Write the assertion test: fit aTracer(galaxies=[lens, source], fields=[field])throughFitImagingand checkgalaxy_model_image_dict.keys() == {lens, source}and the plane images equal those of the galaxy-attached equivalent.
f. Dict round trip:output_to_json/from_jsonof a tracer with fields rebuildstracer.fieldsasal.MassFieldinstances (mirror the existing test).
g. JAX —autolens/jax/registration.py: registerMassFieldwithno_flatten=("redshift",)besideGalaxy(same reasoning: plane bookkeeping compares redshifts), walktracer.fieldsaftertracer.galaxies, and extend the docstring. Probe: the three-way check (NumPy /xp=jnp/jax.jitwith the tracer traced and the grid closed over, as in PyAutoGalaxy#621's probe) ontraced_grid_2d_list_fromwith a field present; unit tests stay NumPy-only.
h.MultiPlaneRedshiftWarning: a tracer whose only mass is a field behind all light must warn exactly as a mass galaxy would; a field at the lens redshift with light behind must not. -
Analysis
fieldsslot —autolens/analysis/analysis/lens.pytracer_via_instance_from: after thescaling_galaxiesfold,fields = list(instance.fields) if getattr(instance, "fields", None) is not None else None;Tracer(galaxies=galaxy_list, fields=fields, cosmology=cosmology). Same fold inautolens/aggregator/tracer.py(_tracer_from/ the list builder ~line 90) and inautolens/analysis/result.py:254only ifmax_log_likelihood_galaxiessits beside amax_log_likelihood_fields(add that property onResultif the instance carriesfields, else leave the positions helper galaxy-only and say so). The subhaloperturbpath is unchanged. Test: anAnalysisImagingfit ofaf.Collection(galaxies=..., fields=af.Collection(field=af.Model(al.MassField, redshift=0.5, shear=af.Model(al.mp.ExternalShear))))underPYAUTO_TEST_MODE=2(or the conftest's test-mode fixture) produces aTracerwhosefieldshas oneMassFieldand whose log likelihood equals the galaxy-attached model's at the same parameter values. -
COOLEST —
autolens/interop/coolest.py:to_coolest: when given aTracer, also takefields = list(galaxies.fields); after the galaxy loop, each field becomes onelazy.MassField(f"field_{j}", float(field.redshift), mass_model=...)carrying all its profiles (ExternalShear→ExternalShear,MassSheet→ConvergenceSheetviacoolest_dict_from_mass); anExternalPotentialor any other unsupported profile goes through the existingon_unsupportedbranch naming the profile (_record_skippedwith the field's name). The legacy peel of galaxy-attached sheets is untouched — write a byte-identity test: export a galaxy-attached tracer on this branch and compare to a fixture JSON captured onmain(or the existing round-trip tests plus a stored expected file), and keepautolens_workspace_test'scoolest_round_trip.pyas the downstream regression.from_coolest: a COOLESTMassFieldentity becomesal.MassField(redshift=..., **{f"mass_{i}": ...})appended to afieldslist, and the returned tracer isTracer(galaxies=galaxies, fields=fields, cosmology=cosmology). Update the docstring (it currently says everyMassFieldentity becomes a galaxy). Test: round trip of a tracer withfields=[MassField(shear, mass_sheet)]returns aMassFieldwith parameters equal to 1e-12; a legacy galaxy-attached export now imports as a field too (document this as the one behaviour change on import, and say so in the PR body).
-
LOS sampler —
autolens/lens/los.py: keepgalaxies_from()returning the halo galaxies and, for compatibility, addfields_from()returning oneag.MassField(redshift=z_cen, mass_sheet=ag.mp.MassSheet(kappa=kappa_neg))per plane; refactor the shared sampling into a private helper so the two do not sample twice (cache the sampled population on the instance, or return both from onegalaxies_and_fields_from() -> Tuple[List[Galaxy], List[MassField]]). Shim:galaxies_from()keeps returning halos + sheet galaxies exactly as today unless called withsheets_as_fields=True(default False) — name the workspacelos_halosscripts as the callers the shim protects, and note in the PR that phase 3 flips them. Test intest_los.py: the new return's sheets areMassFields with the same kappa per plane as the legacy sheet galaxies; the test-mode halo cap still applies. -
model_util.mass_field_from—autolens/analysis/model_util.py:mass_field_from(lens: af.Model, shear: bool = True, mass_sheet: bool = False, potential: bool = False, redshift: Optional[float] = None) -> af.Model. RaiseValueErrornaming the galaxy iflenshas nomass;redshiftdefaults tolens.redshift; buildaf.Model(al.MassField, redshift=..., shear=af.Model(al.mp.ExternalShear))etc.; tiefield.potential.centre = lens.mass.centreandfield.mass_sheet.centre = lens.mass.centreas shared priors (assignment of the same prior object, soisholds). Docstring: multi-deflector callers name the primary galaxy; the result goes infields=af.Collection(field=...). Tests intest_autolens/analysis/test_model_util.py(create it if absent): each flag, theisidentity of the tied centres, the no-mass error, the redshift override, and that the composed collection'sprior_countmatches expectation. -
Exports + docs —
autolens/__init__.py:from autogalaxy.galaxy.mass_field import MassFieldbesideGalaxy;docs/api/galaxy.rstlistsMassFieldand documentsTracer.fields;docs/api/modeling.rst(or wherevermodel_utillives) gainsmass_field_from. A short "External fields" paragraph in the overview page that introduces the tracer, stating the galaxy-attached form remains supported. No PyAutoMemory citations. -
Tests + gate —
pytest test_autolens -q -n autogreen; the identifier pin as a regression test;git diff --name-onlyshows nothing underautolens/config/; nowarnings.warnadded for legacy forms.
Key Files
autolens/lens/tracer.py— constructor, validation, ascending members, planes,sliced_tracer_from,galaxy_image_2d_dict_from.autolens/lens/tracer_util.py— read-only; duck-typed plane builders.autolens/imaging/fit_imaging.py— plane-image loop must skip fields.autolens/analysis/analysis/lens.py,autolens/aggregator/tracer.py,autolens/analysis/result.py— tracer builders.autolens/interop/coolest.py— export/import.autolens/lens/los.py— sampler return shape.autolens/analysis/model_util.py— helper.autolens/jax/registration.py— walker.autolens/__init__.py,docs/api/galaxy.rst,docs/api/modeling.rst.- Tests:
test_autolens/lens/test_tracer.py,test_tracer_validation.py,test_los.py,test_autolens/interop/test_coolest.py,test_autolens/analysis/analysis/test_analysis_lens.py, newtest_autolens/analysis/test_model_util.py.
Hard constraints
- Legacy galaxy-attached sheets and shear-only galaxies in
galaxies: no warning, no behaviour change, COOLEST export byte-identical. tracer.galaxiesnever contains a field; per-galaxy surfaces never see one.- No prior config changes; identifier pin holds.
Out of scope
Workspace adoption (guides/profiles/mass.py, multi_galaxy/, group/, siblings) is phases 3–5 behind a release.
Original Prompt
Click to expand starting prompt
MassField in PyAutoLens: Tracer(fields=), the analysis fields slot, COOLEST 1:1, LOS sheets, model_util helper
Type: feature
Target: PyAutoLens
Repos:
- PyAutoLens
Themes: - cluster
Difficulty: large
Autonomy: supervised
Priority: normal
Consequence: judge
Witness:al.Tracer(galaxies=[lens, source], fields=[al.MassField(redshift=0.5, shear=...)]).traced_grid_2d_list_from(grid)isnp.allcloseto the same shear onlens;tracer.galaxiesis[lens, source](no field in it); a model withfields=af.Collection(field=af.Model(al.MassField, ...))fits throughAnalysisImagingunderPYAUTO_TEST_MODE=2; COOLEST round trip of that tracer returns anal.MassFieldwith the same parameters to 1e-12 and a legacyal.Galaxy(..., shear=...)still exports byte-identically;al.model_util.mass_field_from(lens=lens_model, potential=True).potential.centre is lens_model.mass.centre; identifier pin (galaxy-attached model, equal onmainand branch);pytest test_autolens -q -n autogreen.
Review-minutes: 10
Unattended: ready
Epic: mass-field
Phase: 2
Filed: 2026-09-17
Gate cleared 2026-09-17: phase 1 (complete/2026/09/mass-field-class.md,
PyAutoGalaxy#621) is merged on PyAutoGalaxy main; CI runs the same-named
library branch, so main is enough and a release is not required for this
phase.
Teach PyAutoLens about ag.MassField (phase 1 of the epic
draft/feature/autogalaxy/mass_field_epic.md; read it first). A field is its
own thing: it gets its own tracer argument and its own model slot, and only the
tracer's planes merge it with the galaxies. tracer.galaxies never holds a
field, so every script that indexes galaxies positionally keeps working.
What to build
Tracer(galaxies, fields=None, cosmology=...)—@PyAutoLens/autolens/lens/tracer.py:- store
self.fields(a list;[]when None); validate members on
redshiftthe way_validate_galaxiesdoes, with a message naming
MassField. galaxies_ascending_redshift,plane_redshiftsandplanesare built
over galaxies and fields; a plane (anag.Galaxies) may therefore
hold aMassField, which phase 1 made safe. Everything downstream
(traced_grid_2d_list_from,deflections_yx_2d_from,
convergence_2d_from,potential_2d_from,has,cls_list_from, the
positions solver, magnification, critical curves) flows through planes
and needs no change — verify each with a test rather than assume.tracer.galaxiesis unchanged;tracer.fieldsis the new accessor;
total_planescounts field-only planes (a LOS sheet plane is a plane).sliced_tracer_fromplaces fields in the planes nearest their redshift
likeline_of_sight_galaxies;to_dict/from_dictcarriesfields;
the JAX pytree flatten ofTracerincludesfields
(@PyAutoLens/autolens/jax/registration.pywalkstracer.galaxies— walk
tracer.fieldstoo and registerMassField; the walker is a copy of the
autogalaxy one by a recorded human decision, so patch both, do not share).MultiPlaneRedshiftWarning: fields are mass-bearing for the light/mass
ordering check.- The per-galaxy surfaces (
FitImaging.galaxy_model_image_dict,
subplot_galaxies_images, the aggregator's galaxy tables) iterate
tracer.galaxiesand therefore never see a field — assert that in a test.
- store
- Analysis
fieldsslot —@PyAutoLens/autolens/analysis/analysis/lens.py
tracer_via_instance_from: beside theextra_galaxies/scaling_galaxies
folds,fields = list(instance.fields) if getattr(instance, "fields", None) is not None else None
and passTracer(galaxies=galaxy_list, fields=fields, cosmology=cosmology).
Same in the interferometer / point-source analyses if they build their own
tracer. The subhaloperturbpath is unchanged. - COOLEST —
@PyAutoLens/autolens/interop/coolest.py:to_coolest: eachtracer.fieldsentry becomes one COOLESTMassField
entity carrying all its profiles (ExternalShear→ExternalShear,
MassSheet→ConvergenceSheet;ExternalPotentialhas no COOLEST
profile — route it through the existingon_unsupportedpath with the
profile named). Keep the legacy peel: aGalaxycarrying sheets still
exports them as a separateMassFieldentity, byte-identical to today's
output (the workspace_test round-trip script is the regression).from_coolest: a COOLESTMassFieldentity becomes anal.MassField
in the returned tracer'sfields, not aGalaxy. Update the docstring.
- Line-of-sight sampler —
@PyAutoLens/autolens/lens/los.pybuilds
ag.Galaxy(redshift=z_cen, mass_sheet=...)per plane; buildag.MassField
and return the sheets separately (galaxies_from→ halos as galaxies,
fields_from→ sheets, or one call returning both) so the caller writes
Tracer(galaxies=[lens, source] + halos, fields=sheets). Keep a shim for
the old single-list return if any caller in the tests or the workspace
los_halosscripts depends on it, and say which in the PR. model_utilhelper — besidemge_model_from(the module the workspace
reaches asal.model_util):mass_field_from(lens, shear=True, mass_sheet=False, potential=False, redshift=None) -> af.Model. Builds
af.Model(al.MassField, redshift=<lens.redshift unless given>, ...)with
af.Model(al.mp.ExternalShear)/af.Model(al.mp.MassSheet)/
af.Model(al.mp.ExternalPotential)as requested, and ties
potential.centre(andmass_sheet.centre) tolens.mass.centreas a shared
prior — the human's convention for where an external potential is centred.
lensis anaf.Modelof aGalaxywith amass; if it has none, raise
naming the galaxy. Document that for a multi-deflector system the caller
picks the primary galaxy explicitly, and that the result goes in
fields=af.Collection(field=...).- Tests + docs — tracer tests for each bullet of step 1; an analysis
test fitting a model withfieldsunder test mode; round-trip and
legacy-peel COOLEST tests; atest_model_utilcase for each helper flag and
the shared-prior identity;lostests for the new return; the identifier
pin; API docs forMassField(re-export in@PyAutoLens/autolens/__init__.py
besideGalaxy),Tracer.fieldsandmodel_util.mass_field_from.
Hard constraints
- Legacy galaxy-attached sheets and shear-only galaxies in
galaxies: no
warning, no behaviour change, COOLEST export unchanged (regression-test it). tracer.galaxiesnever contains a field.- No prior config changes.
Activity
Jammy2211 commentedon Sep 17, 2026
Library PR Created
feature/mass-field-integration, labelledpending-release)API changes: added
Tracer(fields=)/tracer.fields/members, the analysisfieldsfold,Result.max_log_likelihood_fields, COOLEST export of fields and import ofMassFieldentities intotracer.fields,LOSSampler.fields_from()(legacy return kept as default),model_util.mass_field_from, and theal.MassFieldre-export. One behaviour change on COOLEST import only (aMassFieldentity now lands infields, notgalaxies). Legacy galaxy-attached sheets unchanged; identifier pinfef2697b5c32ba56bb18a7baecb7b0f6holds; export byte-identical.Gate:
pytest test_autolens -n auto710 passed, 1 pre-existing xfail (57 new tests); JAX probe green with a field in the tracer; nothing underautolens/config/; no deprecation warning.Workspace impact: Added-only API, no migration needed. Adoption in
autolens_workspace(guides/profiles/mass.py,multi_galaxy/,group/, thelos_halosscripts) is the epic's phases 3–5, gated on a release.Next:
/prmonce CI is green on #742 (merge is human).Generated by Claude Code
Jammy2211 commentedon Sep 17, 2026
Shipped
Merged #742 (merge commit 7197380) into
mainon 2026-09-17 — CI green on Python 3.12, 3.13, nojax and the docs build;pending-releaselabel carried.What changed:
Tracer(galaxies, cosmology=None, fields=None)withtracer.fieldsmerged into the planes only (Tracer.members), sotracer.galaxiesnever holds a field and per-galaxy surfaces skip fields explicitly; the analysisfieldsmodel slot (AnalysisLens, aggregator,Result.max_log_likelihood_fields); COOLEST export of fields asMassFieldentities and import ofMassFieldentities intotracer.fields(legacy galaxy-attached export pinned byte-identical);LOSSampler.fields_from()/galaxies_and_fields_from()with the legacy return kept as default;al.model_util.mass_field_fromwith theMassSheet/ExternalPotentialcentre tied tolens.mass.centre;al.MassFieldre-exported; API docs and overview. Galaxy-attached shear/sheets: no warning, no behaviour change, identifier pinfef2697b5c32ba56bb18a7baecb7b0f6held.Mind: phase 2 of the
mass-fieldepic — recordPyAutoMind/complete/2026/09/mass-field-integration.md. Phases 3–5 (workspace adoption) wait on a release of PyAutoGalaxy#621 and PyAutoLens#742.Generated by Claude Code