Skip to content

feat: accept a bare MassField in the fields slot #743

Closed

Description

@Jammy2211

Overview

Accept a bare MassField in the fields model slot and Tracer constructor, alongside the existing collection/list form. This additive PyAutoLens-only change preserves existing collection prior paths, prior counts and identifiers. Workspace examples retain the collection form.

Plan

  • Capture shipped-configuration identifiers, prior counts and paths before edits.
  • Add the flat-model witness and record its failure on unchanged library source.
  • Normalize bare fields at all five entry points, preserving validation and warning ordering.
  • Add regression tests and document both forms with collections remaining primary.
  • Run the full NumPy suite, downstream smoke and review before shipping.
Detailed implementation plan

Affected Repositories

PyAutoLens (library); PyAutoMind task bookkeeping only.

Branch Survey

PyAutoLens: clean main at 7197380; no conflicting claim; only local branch main.
Suggested branch: feature/mass-field-bare-fields
Worktree root: ~/Code/PyAutoLabs-wt/mass-field-bare-fields/

Implementation Steps

  1. Capture baseline from repository root with PYAUTO_SKIP_WORKSPACE_VERSION_CHECK=1. Expect collection identifier c5cf98ea7733689dc8c0ede6939c1d83 and flat 36a0be37c9667958bafca2f01e487f80; reproduce rather than assume. Capture prior_count 14 and paths.
  2. Add witness in test_autolens/analysis/analysis/test_analysis_lens.py; run red before source edits and retain traceback.
  3. Add internal tracer_util.fields_list_from: None -> [], isinstance(ag.MassField) -> [fields], otherwise list(fields). Never test the shared MassProfileAggregate base.
  4. In tracer._validate_fields wrap bare MassField after ModelInstance escape hatch, before string trap. Preserve every message. Bare Galaxy continues to get generic container rejection; [Galaxy] retains the targeted message. Direct generators remain rejected without consumption.
  5. Tracer.init: validate first, normalize before _warn_if_no_light_is_behind_any_mass, store normalized list. Replace folds in sliced_tracer_from, analysis/analysis/lens.py, aggregator/tracer.py, analysis/result.py. Update constructor and slicing annotations.
  6. Tests: bare/list storage and list ownership, deflections and nonzero MassSheet convergence, members/redshifts, geometry warning, JSON parity, slicing redshift snapping, bare Galaxy rejection, flat/collection paths/count/collection identifier; analysis witness and likelihood agreement to 1e-8; Result fields for flat/collection/absent/None/empty; aggregator regression for flat/collection/absent. Existing validation and identifier tests stay untouched. NumPy only.
  7. Docs: helper module bullet; Tracer and validation docstrings; docs/api/galaxy.rst and modeling.rst; model_util.mass_field_from. Keep collection primary; internal helper stays out of public exports/autosummary.
  8. Full suite: NUMBA_CACHE_DIR=/tmp/numba_cache MPLCONFIGDIR=/tmp/matplotlib python -m pytest test_autolens/. PR states additive API, witness traceback, unchanged collection evidence, and different identifiers by design.

Review conclusions

Five normalization sites exhaust autolens source. Serialization reads self.fields; JAX reconstructs dict without constructor re-entry. No PyAutoGalaxy changes. No standalone slam package in checkout. Human approved reviewed plan with 'proceed'.

Original Prompt

Starting prompt

Accept a bare MassField in fields= so the flat model form works

Type: feature
Target: PyAutoLens
Repos:

  • PyAutoLens
    Themes:
  • cluster
    Difficulty: small
    Autonomy: human-required
    Priority: normal
    Consequence: judge
    Epic: mass-field
    Filed: 2026-09-18
    Witness: with the branch installed, af.Collection(galaxies=..., fields=af.Model(al.MassField, redshift=0.5, shear=af.Model(al.mp.ExternalShear))) yields prior paths fields.shear.gamma_1/gamma_2 (not fields.field.shear.…), AnalysisImaging's tracer construction succeeds on its instance, and tracer.fields == [MassField(redshift=0.5, shear)]. On main the same model raises TypeError: 'MassField' object is not iterable. The collection form fields=af.Collection(field=field) is unchanged in paths, prior count and result identifier.

Original request

The following API: field = af.Model(al.MassField, redshift=0.5, shear=af.Model(al.mp.ExternalShear)) and model = af.Collection(
galaxies=af.Collection(lens=lens, source=source),
fields=af.Collection(field=field),
) produces model.fields.field.shear, is there a way to avoid the double API of field.shear?

and, after the options were reported back:

do this If you want the flat form to work, it's a small library change in PyAutoLens, in two places:

  1. PyAutoLens/autolens/analysis/analysis/lens.py:138 — wrap a bare MassField in a list rather than calling list() on it.
  2. PyAutoLens/autolens/lens/tracer.py:240 (_validate_fields) plus the self.fields = list(fields) line in Tracer.init — accept a single MassField.

Adding iter to MassField in PyAutoGalaxy would also satisfy list(), but it would yield the field's profiles, not the field — wrong for Tracer(fields=...). So the lens-side change is the right one.

Why

The shipped composition is fields=af.Collection(field=field), giving parameter
paths fields.field.shear.gamma_1. The middle level is a name the user must
invent for a collection that, in every workspace example today, holds exactly
one entry — so it reads as a stutter (fields.field) rather than as
information. The galaxy-attached form it replaced was one level shallower
(galaxies.lens.shear.gamma_1).

Measured against the installed stack 2026-09-18:

Form Paths Result
fields=af.Collection(field=field) fields.field.shear.gamma_1 works (current convention)
fields=af.Collection(los=field) fields.los.shear.gamma_1 works — rename only
fields=af.Collection([field]) fields.0.shear.gamma_1 works — index middle level
fields=field fields.shear.gamma_1 builds, then dies at tracer construction

prior_count is 14 in all four. Only the flat form removes the level, and it is
the one that breaks: MassField and its base MassProfileAggregate define
neither __iter__ nor __getitem__, so list(instance.fields) raises, and
_validate_fields independently rejects a non-list/tuple non-ModelInstance.

Scope

Three edits, all in PyAutoLens:

  1. autolens/analysis/analysis/lens.py (~line 138, _tracer_via_instance_from)
    — the fields = list(instance.fields) if getattr(instance, "fields", None) is not None else None block wraps a bare MassField in a list instead of
    calling list() on it.
  2. autolens/lens/tracer.py _validate_fields (line 240) — accept a single
    MassField alongside the list/tuple/ModelInstance forms, keeping the
    string trap and the Galaxy-in-fields message intact.
  3. autolens/lens/tracer.py Tracer.__init__ (~line 352) — self.fields = list(fields) normalises a single MassField to [field], preserving the
    "plain list, never None" invariant the members-walking properties rely on.

Explicitly not in scope: adding __iter__/__getitem__ to MassField in
PyAutoGalaxy. It would satisfy list() but yield the field's profiles, not
the field, which is wrong for Tracer(fields=...) and would silently produce a
tracer holding ExternalShear objects with no redshift.

Open questions for the plan

  • Epic decision conflict. The mass-field epic's Decisions section reads
    "Model slot fields=, a collection; tracer argument fields=, a list. The
    analysis folds list(instance.fields) into Tracer(fields=...) exactly as it
    folds extra_galaxies / scaling_galaxies into galaxies." Accepting a bare
    field relaxes that and breaks symmetry with extra_galaxies /
    scaling_galaxies, which stay collection-only. Decide whether the relaxation
    is accepted as a second supported form (both work) and record it in the epic
    ledger.
  • Which form the workspaces teach. Phase 3 (autolens_workspace +
    autolens_workspace_test, 217 + 87 files) has shipped as draft PRs using the
    collection form, and phase 5 (HowToLens, autolens_assistant) is unstarted. If
    the flat form becomes the taught idiom, that is a follow-up sweep, not this
    task. This task adds the capability only.
  • Multi-field systems. The collection form stays mandatory for more than one
    field (per-plane LOS sheets, foreground/background tidal planes); the flat
    form is a single-field convenience. Error messages should say so.
  • Whether Tracer(fields=<bare MassField>) should also be accepted directly, or
    only the model slot. Scope above assumes both, since _validate_fields is the
    shared door.

Notes

  • Backwards compatibility is the epic's hard invariant: the collection form's
    paths, prior count and PyAutoFit result identifier must be untouched, and the
    galaxy-attached form stays unwarned.
  • Diagnostic script behind the table:
    /tmp/claude-1000/-home-jammy-Code-PyAutoLabs/65d55a6f-8729-4857-be92-b098eeb4031d/scratchpad/field_variants.py
    (scratch, not committed).

Activity

  1. Jammy2211 commented on Sep 18, 2026

    @Jammy2211
    CollaboratorAuthor

    Implementation complete in feature/mass-field-bare-fields at the task worktree (not yet committed or pushed; Heart development-shipping gate still pending).

    • Baseline reproduced before source edits: collection c5cf98ea7733689dc8c0ede6939c1d83, flat 36a0be37c9667958bafca2f01e487f80, both prior_count 14.
    • Witness ran red on unchanged library source at 7197380: TypeError: 'MassField' object is not iterable, autolens/analysis/analysis/lens.py:139. Traceback retained for PR body.
    • Five normalization sites implemented; validation diagnostics preserved; normalization precedes geometry warning; docs retain collections as primary.
    • Focused regression tests: 84 passed.
    • Full suite: 739 passed, 1 xfailed, 34 warnings in 156.77s.
    • Independent Sol review: CLEAN, with independent probes for five folds, both identifiers and the geometry warning.
    • Downstream smoke checks are running in disposable workspace snapshots; AutoFit 8 scripts + 2 notebooks and AutoGalaxy 14 scripts + 2 notebooks passed so far.

    No source changes outside PyAutoLens. Original validation and identifier pins remain unmodified.

  2. Jammy2211 commented on Sep 18, 2026

    @Jammy2211
    CollaboratorAuthor

    Implementation and verification complete; awaiting the live human development-shipping override for Heart RED. No commit, push or PR-open yet.

    • Full library suite: 739 passed, 1 xfailed, 34 warnings in 156.77s.
    • Independent Sol review: CLEAN, with end-to-end probes of all five folds, both identifiers and geometry warning.
    • Downstream smoke: all passed in disposable workspace snapshots using the feature library source.
    Workspace Passed Failed Skipped
    autofit_workspace 10 0 0
    autogalaxy_workspace 16 0 0
    autolens_workspace 40 0 0
    autolens_workspace_test 32 0 0
    euclid_strong_lens_modeling_pipeline 9 0 0
    HowToLens 50 0 0

    Current Heart RED reason (verbatim):

    • release validation FAILED (stage integrate)

    Current YELLOW reason (verbatim):

    • manifest drift: remote-session blocks (generated) — 2 mismatch(es) vs PyAutoMind/repos.yaml

    The earlier install-verification failure has cleared. This feature does not claim to repair the release integration failure. Shipping remains gated by PyAutoBrain/AUTONOMY.md "Human override for Heart RED (development only)"; merge and release remain separate.

    Review dispositions:
    CLEAN — independent Sol review, 2026-09-18.
    No must-fix findings.
    Claim unchanged collection identity: test_tracer_fields.py identifier subprocess plus independent probe reproduced collection c5cf98ea7733689dc8c0ede6939c1d83, flat 36a0be37c9667958bafca2f01e487f80, prior_count 14.
    Claim correct flat folding: fields_list_from used at all five entry points; independent probe exercised constructor, slicing, analysis, Result and aggregator; bare/list NumPy deflections agree.
    Claim preserved warning: normalization precedes warning; independent probe observed exactly one MultiPlaneRedshiftWarning.
    Bare Galaxy generic diagnostic and [Galaxy] targeted placement diagnostic preserved. List copying and collection/empty semantics preserved. No upstream or workspace source edits. git diff --check clean.

    Artifacts under /home/jammy/Code/PyAutoLabs-wt/mass-field-bare-fields/: pr-body.md, witness-red.txt, full-suite.txt, smoke-tests.txt, smoke-summary.json, review.txt, heart-readiness.json.

  3. Jammy2211 commented on Sep 18, 2026

    @Jammy2211
    CollaboratorAuthor

    Human development-shipping override recorded for PyAutoLens #743 on 2026-09-18.

    I authorize, raise pr and merge when CI is green. Then give me a prompt to apply this to euclid strong lens modeling pipeline (if not already done) and update the euclid_dr1 science project, including HPYPyAutoPull, whihc ill put in another codex chat

    Current Heart RED: release validation FAILED (stage integrate).
    Current YELLOW: manifest drift: remote-session blocks (generated) — 2 mismatch(es) vs PyAutoMind/repos.yaml.

    Passed branch gates: 739 passed / 1 xfailed (full suite); 157 downstream smoke checks; independent Sol review CLEAN. Collection identifier reproduced and pinned unchanged; red witness captured before source edits.

    Authorized: commit, push, pending-release PR; merge in this turn only if every CI workflow/matrix leg is green and other merge gates pass. No release or CI bypass; this feature does not repair Heart.

  4. Jammy2211 commented on Sep 18, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped in PR #744, merged at 478213e. All four CI legs passed (docs, Python 3.12, Python 3.13, no-JAX). Local validation: 739 passed / 1 xfailed; 157 downstream smoke checks; independent review CLEAN. Bare MassField support is additive; collection identifiers remain unchanged. Pending release; Heart RED was overridden for development only. Task worktree cleanup and Mind close-out follow via prm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions