Skip to content

refactor: share the colour-norm helper; fix inversion log10 floor #488

New issue
New issue
Closed
#489
Closed
refactor: share the colour-norm helper; fix inversion log10 floor#488
#489

Description

@Jammy2211
Jammy2211
opened on Aug 24, 2026

Overview

Building the matplotlib colour norm from use_log10 / vmin / vmax is written
out three times — autoarray/plot/array.py, autoarray/plot/inversion.py, and
autogalaxy/util/plot_utils.py::norm_from (added by PyAutoGalaxy#586). The copies
have already diverged, and one divergence is a live behaviour bug: inversion.py
hardcodes the 1e-4 log floor and never consults autonerves config, so a user
who changes visualize.general.general.log10_min_value gets it honoured on array
plots and silently ignored on inversion plots.

Deduplicating is how the divergence stops recurring; fixing the config floor is
the point. Split out of aplt-output-drift-remaining-repos (PyAutoGalaxy#585).

Plan

  • Add one colour-norm helper to autoarray/plot/utils.py — the lowest repo that
    owns this logic, alongside auto_mask_edge and the other shared plot helpers.
  • Decide each of the three divergences deliberately rather than assuming
    array.py wins on all of them, and write the decision into the helper's
    docstring.
  • Call the helper from array.py and inversion.py.
  • Make autogalaxy.util.plot_utils.norm_from delegate to it rather than
    reimplement it, keeping the autogalaxy-facing name so the Clicker /
    Scribbler callers added by PyAutoGalaxy#586 keep working.
  • Cover the config-floor path (both call sites), explicit limits, derived vmax
    and the degenerate vmax <= vmin case with tests in PyAutoArray.

The three divergence decisions

# Divergence Decision
1 array.py reads log10_min_value from config and clips; inversion.py hardcodes 1e-4 and does not clip array.py wins. The helper always reads the configured floor (fallback 1e-4) and clips. This is the behaviour fix — inversion plots change when the configured floor is not 1e-4.
2 array.py derives vmax from the clipped array; inversion.py from pixel_values, falling back to vmin * 10 when there are none Neither wins — it is not a divergence. The two call sites colour different data. The helper's array argument means "the values being coloured"; inversion.py passes pixel_values. array=None collapses into the same fallback, so inversion's separate else branch disappears with no behaviour change.
3 array.py widens a degenerate vmax <= vmin even when explicitly passed; inversion.py only inside its elif pixel_values branch array.py wins. LogNorm(vmin=10, vmax=1) is unusable whoever supplied the numbers. Inversion plots change when a caller passes vmax <= vmin explicitly.
Detailed implementation plan

Work Classification

Library — both repos are libraries. Library-first: PyAutoArray merges before
PyAutoGalaxy.

Affected Repositories

  • PyAutoArray (primary)
  • PyAutoGalaxy

Branch Survey

Repository Current Branch Dirty?
PyAutoArray main clean
PyAutoGalaxy main clean
PyAutoMind claude/autoarray-shared-log-norm-c12ges prompt file only

Branch: claude/autoarray-shared-log-norm-c12ges (remote web-github
session; no local worktree — the session clones are the working trees).

Implementation Steps

  1. PyAutoArray — autoarray/plot/utils.py: add norm_from(array=None, use_log10=False, vmin=None, vmax=None). Lazy matplotlib import, as
    everywhere else in this module. Body: read log10_min_value from
    autonerves config inside try/except (fallback 1.0e-4); clip; derive
    vmin_log/vmax_log; widen unconditionally when vmax_log <= vmin_log or
    non-finite; return LogNorm. Non-log path returns Normalize when either
    limit is given, else None. Docstring records the table above.
  2. PyAutoArray — autoarray/plot/array.py: replace the inline block (the
    # --- colour normalisation --- stanza) with a call to the helper.
  3. PyAutoArray — autoarray/plot/inversion.py: same, passing
    array=pixel_values. Drop the now-unused inline LogNorm/Normalize use.
  4. PyAutoGalaxy — autogalaxy/util/plot_utils.py: norm_from becomes a
    delegate to autoarray.plot.utils.norm_from, keeping its signature and
    docstring intent. autogalaxy may import autoarray; the reverse is not true.
  5. Tests — test_autoarray/plot/test_utils.py: new class covering the
    config floor with log10_min_value monkeypatched away from 1e-4, asserting
    both call sites honour it; explicit limits; derived vmax; degenerate
    vmax <= vmin; all-NaN input.

Key Files

  • autoarray/plot/utils.py — new shared helper (the reference implementation)
  • autoarray/plot/array.py — call site 1 (was the reference)
  • autoarray/plot/inversion.py — call site 2 (carried the bug)
  • autogalaxy/util/plot_utils.py — norm_from becomes a delegate
  • test_autoarray/plot/test_utils.py — new tests
  • test_autogalaxy/gui/test_plot_norm.py — must stay green (pins fix: apply_over_sampling keeps the sparse operator; cache operated_mapping_matrix_list #586 behaviour)

Constraints

  • Behaviour-preserving except where a divergence is deliberately fixed; each
    such fix is named in the PR body. Thin test coverage on this path means an
    unnoticed change ships silently.
  • The vitals faculty could not be consulted in this session (pyauto-heart is
    not on PATH in a web-github run) — the Heart gate still applies at ship.

Autonomy

Launched /start_dev … --auto. Prompt header Autonomy: supervised; refactor
work-type cap is safe; effective level supervised. Per the autonomy
contract that means: the plan is written here rather than presented and waited
on, work proceeds, and the run parks at ship sign-off with a question on this
issue.

Original Prompt

Click to expand starting prompt

One shared colour-norm helper — the three copies have already diverged

Type: refactor
Target: autoarray
Repos:

  • PyAutoArray
  • PyAutoGalaxy
    Difficulty: small
    Autonomy: supervised
    Priority: normal
    Status: formalised
    Filed: 2026-08-24

Split out of aplt-output-drift-remaining-repos (PyAutoGalaxy#585, 2026-08-24),
which added the third copy rather than widen that task into a third repo.

The duplication

Building the matplotlib colour norm from use_log10 / vmin / vmax is written
out three times:

Where Notes
autoarray/plot/array.py:164-188 the reference implementation
autoarray/plot/inversion.py:92-107 already divergent — see below
autogalaxy/util/plot_utils.py::norm_from added by PyAutoGalaxy#586, faithful to array.py

This is not a tidiness task — the copies disagree

array.py reads the configured floor:

log10_min = _conf.instance["visualize"]["general"]["general"]["log10_min_value"]   # fallback 1.0e-4
clipped = np.clip(array, log10_min, None)

inversion.py hardcodes 1e-4 and never consults autonerves config, and
does not clip. So a user who changes log10_min_value gets it honoured on array
plots and silently ignored on inversion plots. That is a live behaviour bug, not
a style issue, and it is the thing to fix first — the deduplication is how you
stop it recurring.

They differ in a second way worth preserving deliberately rather than by
accident: array.py derives vmax from the clipped array, inversion.py from
pixel_values and falls back to vmin_log * 10.0 when there are none. And
inversion.py's vmax_log <= vmin_log guard sits inside its elif pixel_values
branch, so an explicitly-passed degenerate vmax is not widened there but is
in array.py.

Decide which behaviour is correct for each difference before merging them —
do not assume array.py wins on all three. Write the decision into the helper's
docstring.

Shape of the fix

  1. Add one helper in PyAutoArray — the lowest repo that owns this logic;
    autoarray/plot/utils.py sits alongside auto_mask_edge and the other
    shared plot helpers. Give it the array/use_log10/vmin/vmax signature
    plus whatever parameter reconciles the pixel_values difference.
  2. Call it from array.py and inversion.py.
  3. Make autogalaxy.util.plot_utils.norm_from delegate to it rather than
    reimplement — autogalaxy may import autoarray (dependency direction is fine;
    the reverse is not). Keep norm_from as the autogalaxy-facing name so the
    Clicker/Scribbler callers added in PyAutoGalaxy#586 keep working.

Constraints

  • Behaviour-preserving except where you deliberately fix a divergence, and
    each such fix is named in the PR body. This is a plotting path with thin test
    coverage, so an unnoticed change ships silently.
  • Library-first: PyAutoArray merges before PyAutoGalaxy.
  • test_autogalaxy/gui/test_plot_norm.py (added by fix: apply_over_sampling keeps the sparse operator; cache operated_mapping_matrix_list #586) already pins the
    autogalaxy behaviour — it must stay green, or its change must be justified.

Verification

Add tests in PyAutoArray covering the config-floor path (log10_min_value set to
something other than 1e-4, asserting both call sites honour it), the
explicit-limits path, the derived-vmax path, and the degenerate vmax <= vmin
case. Then the full suites in both repos, on 3.12 and 3.13.

Activity

  1. Jammy2211 commented on Aug 24, 2026

    @Jammy2211
    Jammy2211
    on Aug 24, 2026 – with Claude
    CollaboratorAuthor

    Implemented and pushed — parked at ship sign-off

    Effective autonomy is supervised (prompt header supervised, refactor cap safe), so per the autonomy contract the run stops here: the code is written, tested and pushed, but the PRs are not opened until you sign off. Two questions below.

    What was done

    Branch claude/autoarray-shared-log-norm-c12ges in both repos.

    • PyAutoArray (accc253) — norm_from added to autoarray/plot/utils.py, called from plot/array.py and plot/inversion.py. The config read lands as _conf_log10_min_value alongside the other _conf_* readers. The three divergence decisions are in the helper's docstring.
    • PyAutoGalaxy (00a0499) — util/plot_utils.py::norm_from is now a thin delegate; the autogalaxy-facing name and signature are unchanged, so the Clicker / Scribbler callers are untouched.

    The deliberate behaviour changes

    Both land on plot_inversion_reconstruction only; plot_array and the autogalaxy GUI path are unchanged.

    1. The configured log floor is now honoured. It hardcoded 1e-4 and never read visualize.general.general.log10_min_value. Inversion plots change when the configured floor is not 1e-4 — this is the fix the task exists for.
    2. A degenerate vmax <= vmin is now widened even when passed explicitly. Its guard sat inside the elif pixel_values branch, so an explicit degenerate pair reached matplotlib unmodified.

    The third difference — array.py deriving vmax from the image, inversion.py from pixel_values — was not merged into one behaviour. The helper's array argument means "the values being coloured"; the two call sites genuinely colour different data. array=None takes the same fallback as an all-NaN array, which is exactly what inversion's separate else branch did, so no behaviour changed there.

    Verification

    Suite Result
    test_autoarray/plot 17 passed (11 new)
    test_autoarray (full) 1157 passed, 55 skipped
    test_autogalaxy/gui/test_plot_norm.py (pins #586) 9 passed, file untouched
    test_autogalaxy (full, against this PyAutoArray branch) 1119 passed, 1 skipped

    Two caveats, stated plainly:

    • The 8 inversion/inversion/test_{abstract,factory}.py sparse-operator failures in this environment are present on unmodified main (verified by stashing) and are unrelated to plotting. CI is the real check.
    • Ran on Python 3.12 only — this session has no 3.13 environment. The prompt asks for both; CI covers 3.13.

    The new tests were checked for teeth, not just green: reverting plot_inversion_reconstruction to the hardcoded floor makes test__plot_inversion_reconstruction fail with assert 0.0001 == 0.003.

    Question 1 — ship sign-off (blocking)

    Open the two PRs, PyAutoArray first (library-first: PyAutoArray#488 merges before the PyAutoGalaxy one)?

    • Yes → the PRs go up with the two behaviour changes named in the bodies, and the task moves to ready-to-ship in the Mind registry. Merge stays yours either way.
    • No / changes wanted → say which of the two behaviour changes you want reverted to the old inversion behaviour and I will re-work before opening anything.

    One thing the ship gate could not do here: pyauto-heart is not on PATH in this web-github session, so the vitals faculty was never consulted. The Heart verdict is unknown, not green. If you would rather the gate ran properly, this needs a session that can reach Heart.

    Question 2 — an orphaned prompt file (non-blocking)

    The prompt this task started from did not exist on PyAutoMind/main. It was only ever committed to claude/aplt-output-drift-repos-33n81z (dd7501f1, "file the three aplt-output-drift follow-ups + queue them"), and **no PR was ever opened for that branch** — so all three follow-ups filed that day, plus the queue.mdentries, are invisible tomain`, the dashboard and the Brain router.

    I recovered this one file onto the task branch and advanced it draft/ → active/. The other two are still stranded:

    • draft/workspaces/euclid_crlf_line_endings.md
    • the queue.md additions from that commit

    Want me to open a PR for that PyAutoMind branch, or recover the remaining files the same way? Either works; doing nothing loses them quietly, which is why it is worth a look. This is not blocking the ship decision.

    Generated by Claude Code

  2. Jammy2211
    closed this as completedin #489on Aug 24, 2026

  3. Jammy2211 commented on Aug 24, 2026

    @Jammy2211
    Jammy2211
    on Aug 24, 2026 – with Claude
    CollaboratorAuthor

    Shipped

    Repo PR Merge commit CI
    PyAutoArray #489 b690c3e 3/3 green — 3.12, 3.13, nojax
    PyAutoGalaxy #587 0c7bc62 4/4 green — 3.12, 3.13, nojax, docs-build
    PyAutoMind #313 d7132c9 no checks configured on PRs (registry state only)

    Library-first order honoured: PyAutoArray merged first, PyAutoGalaxy after. Every leg on every run was completed + success before either merge; both branches verified merged into main afterwards (rev-list origin/main..branch = 0).

    What landed

    autoarray.plot.utils.norm_from is now the single colour-norm implementation, called by plot/array.py, plot/inversion.py, and by delegation from autogalaxy.util.plot_utils.norm_from. The config read lands once as _conf_log10_min_value.

    Two deliberate behaviour changes, both on plot_inversion_reconstruction only — plot_array and the autogalaxy GUI path are untouched:

    1. The configured log10_min_value is honoured. It hardcoded 1e-4. This is the bug the task existed for.
    2. A degenerate vmax <= vmin is widened however it arose, not only in the derived branch.

    The third difference was deliberately not reconciled: array means "the values being coloured", so array.py passing the image and inversion.py passing pixel_values is correct rather than divergent. All three decisions are in the helper's docstring, as the prompt required.

    Notes worth keeping

    • The Heart gate could not give a real verdict. pyauto-heart readiness returned stale, score 35 — every reason status unknown (no report.json, install verification not run, five repos unknown). That is Heart with an empty sensor bank in a fresh web-github container, not a health signal. The merge decision rested on real CI instead, which is the sensor Heart was missing. Flagging it because a stale Heart on a green-CI change is a gate that did not actually gate.
    • The prompt file was not on main when the task started. It lived only on the unmerged filing branch claude/aplt-output-drift-repos-33n81z, so /start_dev on the given path found nothing until it was recovered from that branch. PR PyAutoMind#311 landed that branch mid-task, which is why the Mind close-out hit a merge conflict — main was adding the prompt to draft/ as this branch folded it into complete/. Resolved by dropping the draft copy and marking its queue.md line # DONE. Nothing warns about a filing branch with no PR.

    Completion record: complete/2026/08/shared-log-norm-helper.md. Both merged PRs remain in the pending-release queue; no release was performed.

    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions