Skip to content

feat: model figures phase 3 — lens solved/missing semantics + cookbook figures #736

Closed

Description

@Jammy2211

Overview

Phase 3 of the model-figures epic (PyAutoMind ledger draft/feature/autofit/model_figures_epic.md; phases 1–2 shipped as PyAutoFit#1606 and PyAutoFit#1614 + autofit_workspace#152). The renderer draws any model truthfully but cannot know which lens parameters a fit solves rather than samples. This phase adds a class-declared __solved_parameters__ protocol consumed by PyAutoFit's graph_spec (no lens class names inside PyAutoFit), declares it on LightProfileLinear, PointSolved and Pixelization, renders the three lens acceptance models with real classes, and puts model figures into the lens and galaxy cookbooks and docs pages. Two prompt statements were stale against the installed stack and the plan follows the code: Pixelization is two-deep and areas_factor is a Delaunay mesh parameter (genuinely missing from the null priors config).

Plan

  • PyAutoFit: __solved_parameters__ protocol read from obj.cls in graph_spec, additive solved_paths, rule R7 widened to float subclasses (free Redshift is a pill on the galaxy card), solved count/text in the presentation; tests.
  • PyAutoArray: Pixelization.__solved_parameters__ = ("reconstruction",). PyAutoGalaxy: LightProfileLinear (intensity), PointSolved (centre); Basis declares nothing (members carry the solved amplitudes); real-class semantics tests.
  • PyAutoLens: real-class acceptance tests for the simple lens, MGE 2×30 + Delaunay pixelization and the 8-extra-galaxy group model; committed renders; docs/general/model_cookbook.md re-synced to the script with figures; overview_3_features.md MGE/pixelization/point figures. PyAutoGalaxy docs likewise.
  • Ship four library PRs on one same-named branch, merged PyAutoFit → PyAutoArray → PyAutoGalaxy → PyAutoLens.
  • Workspaces: figures at every model.info stage of both modeling cookbooks, a fixed Redshift Free stage and a new Solved Parameters stage in the lens cookbook, notebooks regenerated, model_figure: false in both config/output.yaml.
Detailed implementation plan

Scope note (widening the prompt's repo list)

The prompt lists PyAutoLens, PyAutoGalaxy and the two workspaces. The survey shows the protocol needs PyAutoFit (extractor consumes it) and PyAutoArray (Pixelization is autoarray's class). Both changes are small (≈40 and 1 lines) and are the only honest way to meet the prompt's "registry or protocol, not class-name strings inside PyAutoFit" rule. Six repos, one same-named branch, library PRs merged in dependency order.

Branch survey (step 4)

All six repos are on main, clean, level with origin. worktree_check_conflict fires only for PyAutoFit (still remove-parallel-ep-optimiser, PR #1612 open, no local worktree, EP-optimiser files only) — same disjoint-file parallel-claim waiver as phase 2, to be recorded on the active.md row. PyAutoArray, PyAutoGalaxy, PyAutoLens, autolens_workspace, autogalaxy_workspace: no claims.

  • Task model-figures-lens · branch feature/model-figures-lens (identical name across all six repos so lib-tests.yml checks out the same-named dependency branches) · worktree ~/Code/PyAutoLabs-wt/model-figures-lens/.
  • Issue: feat: model figures phase 3 — lens solved/missing semantics + cookbook figures on PyAutoLens (the prompt's target).

Decisions (recommendations)

  1. Protocol = class attribute __solved_parameters__ (a tuple of attribute names), read from obj.cls by the extractor exactly as it already reads __exclude_identifier_fields__ (graph_spec.py:610-625). Declared on: LightProfileLinear (("intensity",)), Basis (nothing — it has no model-level amplitude; the solved amplitudes are one per member Gaussian, and the docs say so), PointSolved (("centre",)), aa.Pixelization (("reconstruction",) — the name of the solved quantity on inversion.reconstruction). Subclasses inherit it (linear Gaussian/Sersic/… get intensity from the base).
  2. solved_paths widened to synthesise a row when unmatched (never silently ignored); a warning is not needed — the row appears, in_model_info=False.
  3. Free redshift: extend rule R7 so af.Model(cls) with cls a subclass of int/float (al.Redshift(float)) is a row on its owner, not a one-pill child card (_is_dropped_model, graph_spec.py:581-586 → issubclass). A free redshift then renders as the ordinary sampled redshift pill on the Galaxy card; a fixed one stays the redshift = 0.5 subtitle.
  4. Presentation: solved pills read name · solved; footer gains N solved during fitting; counts["solved"] added; legend wording unchanged ("solved during fitting").
  5. Tint: documented as nesting depth, decorative, in both model_cookbook.md pages and the legend note. No code change.
  6. Images follow each repo's live convention docs/<section>/images/<page>/… referenced by raw URL, not a new docs/images/.
  7. Acceptance renders with real classes are built in test_autolens (autofit may not import autolens): (a) imaging/modeling.py:269-285 shape, (b) the MGE composition multi_gaussian_expansion/modeling.py:209-261 with a Pixelization(Delaunay(pixels=…, zeroed_pixels=…), ConstantSplit) source (assembled — no workspace script has both), (e) group/modeling.py:285-348 with eight synthetic extra-galaxy centres (the shipped dataset has one; the epic's count is eight and phase 1's doubles already use eight). Counts are pinned as measured with the epic targets in comments; the structural invariants (two 30 components plates split on ell_comps, intensity · solved inside the plate, areas_factor · missing, per-galaxy fixed centres) are the assertions.
  8. Lens cookbook: the prompt's "6 stages" are 7 print(model.info) calls (:82, :117, :166, :200, :256, :319, :373); the last re-prints the JSON reload. Figures go after the six distinct stages; the JSON stage gets one sentence ("the reloaded model draws identically"). Two additions so the page actually shows the semantics this phase exists for: the __Redshift Free__ stage builds and prints its model (today it builds nothing) and gets a figure; a new __Solved Parameters__ stage after it composes lp_linear bulge + Pixelization(Delaunay, ConstantSplit) source + ps.PointSolved and shows the solved / missing encodings with the many-to-many model.info note. The galaxy cookbook gets figures at its 8 stages (7 distinct + the JSON reload sentence) — no new stage.

High-level plan

  1. PyAutoFit: __solved_parameters__ protocol + additive solved_paths + R7 for float subclasses + solved counts/text; tests.
  2. PyAutoArray + PyAutoGalaxy: declare the protocol on Pixelization, LightProfileLinear, PointSolved; PyAutoGalaxy tests over real classes (linear intensity solved, Basis no amplitude, PointSolved centre solved, Delaunay areas_factor missing, free redshift pill, fixed redshift subtitle).
  3. PyAutoLens: real-class acceptance tests for (a)/(b)/(e), the make_figures.py + committed PNGs, docs/general/model_cookbook.md refreshed to the script (it has drifted to al.lp.*) with figures, docs/overview/overview_3_features.md MGE + pixelization sections each gain the model figure. PyAutoGalaxy docs model_cookbook.md likewise.
  4. Ship four library PRs (pending-release), merged in order PyAutoFit → PyAutoArray → PyAutoGalaxy → PyAutoLens.
  5. Workspaces: autolens_workspace/scripts/guides/modeling/cookbook.py (figures at 6 stages + Redshift Free fix + Solved Parameters stage), autogalaxy_workspace/scripts/guides/modeling/cookbook.py (figures at 7 stages), notebooks regenerated, model_figure: false added to both config/output.yaml (they carry the full key list, so this is the phase-6 rollout for these two repos done early and cheaply). Two workspace PRs behind the library-first gate.

Detailed plan — libraries

PyAutoFit (autofit/graph_spec.py, autofit/model_figure/presentation.py)

  • _solved_parameter_names(obj): getattr(getattr(obj, "cls", None), "__solved_parameters__", ()), tuple of str; documented in the module docstring next to the other protocol probes. In _node, after the __dict__ walk: for each declared name not already a row/child at that path, append ParamRow(name, path+(name,), sampling="solved", provenance=Provenance("solved-by-fit"), in_model_info=False); a declared name that IS a slot (e.g. a subclass that re-exposes intensity) is re-tagged, not duplicated. Provenance.kind docstring gains solved-by-fit.
  • solved_paths: keep the re-tag; after the tree is built, any remaining unmatched path is attached as a synthesised solved row on its owner node (owner missing → root), reusing the attach_latents shape (graph_spec.py:1226-1264; factor the shared "attach rows by owner path" helper).
  • R7: _is_dropped_model → cls in (int, float) or (isinstance(cls, type) and issubclass(cls, (int, float))); the row keeps the prior/constant state and the class name is not shown (documented).
  • _counts: add "solved" (tuple components counted per scalar like missing). path_index entries for solved rows: empty tuple (already the in_model_info=False rule).
  • Presentation: _text → f"{name} · solved"; footer _plural(counts["solved"], "parameter solved during fitting", "parameters solved during fitting"); legend line unchanged. Plate repeats line: a solved member row reads intensity solved.
  • Tests (test_autofit/graph_spec/ + test_autofit/model_figure/): a double class with __solved_parameters__ = ("intensity",) and a subclass; a pass class with ("centre",); unmatched solved_paths synthesises; declared name that is a slot re-tags; Model(FloatSubclass) is a row; determinism and to_dict stability; footer/text. Update lens_doubles.py docstring lines 26-30 to say areas_factor now lives on the Delaunay mesh (the double keeps the old shape deliberately).

PyAutoArray — autoarray/inversion/pixelization.py

  • class Pixelization: __solved_parameters__ = ("reconstruction",) with a 3-line comment naming the protocol and PyAutoFit's graph_spec.

PyAutoGalaxy

  • autogalaxy/profiles/light/linear/abstract.py LightProfileLinear.__solved_parameters__ = ("intensity",); autogalaxy/profiles/point_sources.py PointSolved.__solved_parameters__ = ("centre",); Basis untouched (comment in basis.py docstring: no amplitude of its own — members carry the solved intensities).
  • test_autogalaxy/profiles/test_model_figure_semantics.py (Agg): the six real-class assertions in decision 2 via af.GraphSpec.from_model + af.ModelPlotter(...).presentation(); af.Model(ag.mesh.Delaunay) yields three missing rows (pixels, zeroed_pixels, areas_factor) and Delaunay(pixels=…, zeroed_pixels=…) leaves exactly areas_factor missing.
  • docs/general/model_cookbook.md: figures after the stages that mirror the galaxy cookbook, PNGs under docs/general/images/model_cookbook/ via make_figures.py there; tint note; many-to-many note where the MGE appears in docs/overview/overview_3_features.md (:46 region) — add the MGE model figure there.

PyAutoLens

  • test_autolens/model_figure/test_acceptance_lens.py (Agg): builds (a)/(b)/(e) with real classes (helpers in test_autolens/model_figure/lens_models.py), asserts the invariants + pinned counts, renders to tmp_path, checks widths ≤ 1400 px and determinism of the layout dict.
  • docs/general/images/model_cookbook/make_figures.py + PNGs: simple_lens, complex_lens, four_galaxies, concise, prior_custom, model_custom, redshift_free, solved_parameters (cookbook stages) and the acceptance trio acceptance_simple_lens, acceptance_mge_pixelized, acceptance_group_scale; docs/overview/images/overview_3/ gains mge_model.png, pixelization_model.png, point_model.png.
  • docs/general/model_cookbook.md: re-synced to the script (lp_linear, the Redshift Free and Solved Parameters stages, drop-or-mark the script-less "Multi Galaxy, Group and Cluster Models" section as pointing at group/modeling.py with the group acceptance figure), figures embedded after each model.info block, plus a "Reading the figure" subsection: map vs legend, solved/missing are additional information absent from model.info, the plate-vs-model.info grouping paragraph verbatim from the prompt, tint = nesting depth.
  • docs/overview/overview_3_features.md: model figure beside the MGE (:128), pixelization and point-source sections.
  • Sphinx warning counts ≤ each repo's baseline.

Detailed plan — workspaces (after the four library PRs are open)

  • autolens_workspace/scripts/guides/modeling/cookbook.py: prose cell + af.ModelPlotter(model).figure() after :82, :117, :166, :200, :256, :319; one sentence at :373; Redshift Free builds model, prints info, figure; new __Solved Parameters__ section (lp_linear bulge, Pixelization(Delaunay(pixels=500, zeroed_pixels=0), ConstantSplit) source, a ps.PointSolved galaxy) with the many-to-many/solved/missing prose. Runs headless under profile_smoke.yaml defaults (explicit .figure() calls are unaffected by PYAUTO_SKIP_VISUALIZATION; no dataset, so PYAUTO_SMALL_DATASETS is moot).
  • autogalaxy_workspace/scripts/guides/modeling/cookbook.py: figures after :50, :76, :107, :127, :149, :194; sentence at :61 (repeat) and :227 (JSON).
  • Both config/output.yaml: model_figure: false block after model_graph.
  • Regenerate only the two cookbook notebooks (generate.py autolens / autogalaxy, run from inside each workspace); revert any drift twins.
  • Prose is tutorial prose → Opus; architect reviews it.

Delegation (Opus, one subagent per leg, heartbeat files + Monitor)

A. PyAutoFit protocol + tests. B. PyAutoArray + PyAutoGalaxy + PyAutoLens library edits, tests, acceptance renders, docs pages (starts after A's API is in the worktree; same worktree, disjoint repos). C. Test gates + ship_library for the four repos (four PRs, cross-referenced). D. Two workspace cookbooks + notebooks + config. E. ship_workspace (two PRs). Architect reviews the acceptance PNGs after B and the prose after D.

Verification

  • Serial pytest: test_autofit/graph_spec test_autofit/model_figure, test_autogalaxy/profiles/test_model_figure_semantics.py, test_autolens/model_figure; full suites of the four libraries before ship; black/pyflakes on touched files.
  • Committed PNGs ≤ 1400 px; MGE render shows two 30 components plates with intensity · solved pills and areas_factor · missing; group render shows 8-member plate with centre · fixed, varies by member.
  • Both cookbooks run headless from their workspace worktree roots with the smoke env; notebooks regenerated.
  • CI: lib-tests on all four libraries (same-named dependency branches resolve automatically), docs builds, workspace smoke on the same-named library branches. Merge order PyAutoFit → PyAutoArray → PyAutoGalaxy → PyAutoLens → workspaces via /prm.

Out of scope

Flipping model_figure to default-true (separate human decision, ledger), phases 4–6, dotted lens latent keys (analysis= placement of magnification etc. on the root card — noted as a follow-up prompt), the autofit>= floor bump in PyAutoGalaxy (only needed once a release depends on the protocol; noted in the PR).

Original Prompt

Click to expand starting prompt

Model figures phase 3 — lens semantics and cookbooks

Type: feature
Target: PyAutoLens
Repos:

  • PyAutoLens
  • PyAutoGalaxy
  • autolens_workspace
  • autogalaxy_workspace
    Themes:
  • visualization
  • mge
  • pixelization
    Difficulty: medium
    Autonomy: supervised
    Priority: normal
    Status: formalised
    Consequence: judge
    Review-minutes: 25
    Unattended: ready
    Epic: model-figures
    Phase: 3
    Filed: 2026-09-10

Phase 2 shipped 2026-09-11 — complete/2026/09/model-figures-renderer.md (PyAutoFit#1614 merged, autofit_workspace#152 merged); this phase is unblocked.
(Deliberately not in a Blocked-by: header: that key is graded against GitHub
refs and cannot name a Mind prompt path.)

Phase 3 of the model-figures epic. Ledger (brief, full design record, full
independent review): draft/feature/autofit/model_figures_epic.md. By this
phase the renderer is already correct on the acceptance models; phase 3 supplies
the lens domain semantics it renders and puts the figures into the lens
documentation.

Domain semantics hooks

PyAutoFit cannot know which parameters a lens fit solves rather than samples.
Supply that from PyAutoGalaxy / PyAutoLens as data the phase-1 spec consumes —
a registry or a small protocol, not hard-coded class-name strings inside
PyAutoFit:

  • Linear light profiles (al.lp_linear.*) and al.lp_basis.Basis —
    intensity is absent from the model entirely; it is solved by the
    inversion. It must be emitted as a solved pill with the legend wording
    "solved during fitting", or the figure silently misreports the profile as
    having no amplitude. Beware the review's caveat: a Basis.intensity drawn
    alongside member intensities risks implying an additional solved amplitude —
    decide and document where it belongs.
  • Pixelization — the source pixels are solved; al.Pixelization is a
    3-deep chain (image_mesh / mesh / regularization) whose only free
    parameters are the regularization coefficients. mesh_shape and pixels are
    plain scalar attributes, not priors: rows on their owner, never boxes.
  • al.ps.PointSolved — zero parameters; the centre is solved analytically.
    al.ps.Point has two. Without the annotation the box looks empty and wrong.
  • areas_factor — the missing state. It is unset configuration, not a
    solved quantity and not an absent one. This is an explicit review finding
    (the prototype labelled a Delaunay.pixels pill "solved" when the real gap
    was areas_factor) and it is an epic acceptance case.
  • Redshift — fixed redshift renders as redshift = 0.5 under the galaxy
    header (a presentation convenience, not a different semantic rule); a free
    redshift (af.Model(al.Redshift)) keeps the ordinary sampled pill.

Class-family header tint — decide it here

The review's nit: either explain class-family header colours (light / mass /
pixelization) in the legend, or keep them strictly decorative and say so.
Do not ship an unexplained semantic-looking encoding. Whichever way it goes,
record the decision in the docs page.

Documentation deliverables

  • autolens_workspace/scripts/guides/modeling/cookbook.py — 442 lines with 6
    model.info stages
    ; each gains its figure.
  • @PyAutoLens/docs/general/model_cookbook.md and
    @PyAutoLens/docs/overview/overview_3_features.md — PNGs committed under
    docs/images/ and referenced by raw URL, matching the existing pattern.
  • The equivalent PyAutoGalaxy / autogalaxy_workspace cookbook stages where
    they mirror the lens ones.

Document the plate-vs-model.info grouping difference

This is the one place the promised correspondence is genuinely many-to-many and
it must be written down for readers, not papered over:

The figure partitions by component: the MGE model shows two 30 components
plates, split because each basis holds its own ell_comps pair. model.info
groups per parameter: it prints a single 0 - 59 block for centre
spanning both figure plates, two separate ellipticity blocks, and individual
sigma blocks.

The contract that does hold, and which the docs should state: every displayed
element resolves to a path or grouped paths in model.info; every omission and
every added annotation (solved, missing) is explicit. Solved annotations
have no counterpart in model.info at all and must be identified as additional
information.

Acceptance

  • The three lens acceptance figures from the ledger render truthfully with lens
    semantics attached, and the renders are committed.
  • All 6 cookbook stages carry a figure; the docs pages build with the images.
  • Once these pass, the per-search output.yaml key model_figure (phase 2) may
    be proposed for default-true — a separate human decision, recorded in
    the ledger, not taken inside this phase.

🤖 Generated with Claude Code

https://claude.ai/code/session_0178yLU9v19GtACcRPFggjGa

Activity

  1. Jammy2211 commented on Sep 11, 2026

    @Jammy2211
    CollaboratorAuthor

    Workspace PRs Created

    Close-out: /prm — merge order PyAutoFit#1615 → PyAutoArray#550 → PyAutoGalaxy#616 → PyAutoLens#737 → autogalaxy_workspace → autolens_workspace (library-first gate; workspace CI runs on the same-named library branches).

  2. added a commit that references this issue on Sep 11, 2026
  3. Jammy2211 commented on Sep 11, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped

    Merged, in order: PyAutoFit#1615 · PyAutoArray#550 · PyAutoGalaxy#616 · PyAutoLens#737 · autogalaxy_workspace#240 · autolens_workspace#541 (all pending-release).

    Class-declared __solved_parameters__ protocol in PyAutoFit's graph_spec (additive solved rows; R7 for float subclasses; Model no longer leaks cls from a custom __new__), declared on LightProfileLinear, PointSolved and Pixelization; real-class lens acceptance tests + committed renders; model figures in both model cookbooks (docs pages and workspace scripts + notebooks), model_figure: false added to both workspace configs. Follow-ups filed in the Mind: af.Model.from_json drops zero-free-parameter components; tuple-component relations not yet annotated on pills.

    Epic model-figures: phase 3 of 6 done; phase 4 (graphical plates) unblocked. Flipping model_figure to default-true stays a separate human decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions