Skip to content

feat: audit the colormap lever end-to-end, make malformed values loud, magma for Euclid #509

New issue
New issue
Closed
Closed
feat: audit the colormap lever end-to-end, make malformed values loud, magma for Euclid#509

Description

@Jammy2211
Jammy2211
opened on Aug 29, 2026

Overview

Users regularly complain about the default colormap. The lever to change it already exists — PyAutoArray/autoarray/plot/utils.py::_default_colormap() reads visualize/general.yaml → colormap and every 2D plot function takes a colormap argument — but it is undocumented, and the config read is wrapped in a bare except Exception so a typo'd key or an unknown colormap name silently reverts to the bundled autoarray map with no warning. This task audits the lever end-to-end, makes malformed values loud, documents the two override routes, and defaults the Euclid pipeline repo to magma.

Phase 0 of 10 in the euclid-dr1-prep epic. Independent of every other phase — it gates nothing and nothing gates it.

Plan

  • Audit that the colormap config key actually reaches every 2D plotting surface across PyAutoArray / PyAutoGalaxy / PyAutoLens, and document the deliberate exceptions (hardcoded, semantically-chosen colormaps) rather than leaving them silent.
  • Make _default_colormap() loud: an absent config still falls back quietly to autoarray, but a present but unresolvable colormap name raises a clear ValueError naming the config key and the bad value. Replace the bare except Exception with a narrow one.
  • Verify and document the per-figure override route through the public plot API (colormap= on every plot_* entry point).
  • Add a "Changing the colormap" section to the config/visualize/README.md files (PyAutoArray + the Euclid pipeline repo).
  • Set colormap: magma in euclid_strong_lens_modeling_pipeline/config/visualize/general.yaml. Leave PyAutoArray and autolens_workspace on autoarray.
Detailed implementation plan

Affected Repositories

  • PyAutoArray (primary — library)
  • euclid_strong_lens_modeling_pipeline (workspace follow-up)

PyAutoLens / PyAutoGalaxy are audited but not edited — their plot surfaces already thread colormap down to autoarray's plot_array / plot_grid / plot_mapper.

Branch Survey

Repository Current Branch Dirty?
./PyAutoArray main clean
./PyAutoLens main clean
./autolens_workspace main clean
./euclid_strong_lens_modeling_pipeline main clean

worktree_check_conflict cmap-magma-default PyAutoArray euclid_strong_lens_modeling_pipeline → no conflict.

Suggested branch: feature/cmap-magma-default

The lever as it stands (surveyed 2026-08-28)

# PyAutoArray/autoarray/plot/utils.py
def _default_colormap() -> str:
    try:
        from autonerves import conf
        name = conf.instance["visualize"]["general"]["colormap"]
    except Exception:            # <-- swallows a typo'd key AND a real failure
        name = "autoarray"
    if name == "autoarray":
        from autoarray.plot.segmentdata import register
        register()
    return name

Consumers, all of which default colormap=None and then call _default_colormap():

  • autoarray/plot/array.py:140-142 → imshow(cmap=colormap) (lines 180, 187)
  • autoarray/plot/grid.py:105-107 → scatter(cmap=colormap) (lines 118-124)
  • autoarray/plot/inversion.py:79-81 → _plot_rectangular (imshow/pcolormesh, lines 198/214) and _plot_delaunay (tripcolor, line 269)

Everything above those three modules (autoarray/dataset/plot/, autoarray/fit/plot/, autoarray/inversion/plot/, PyAutoGalaxy's _resolve_colormap("default") in autogalaxy/util/plot_utils.py:97-102, PyAutoLens's tracer_plots.py / sensitivity_plots.py / fit_imaging_plots.py) either passes colormap through or leaves it None, so the config value reaches the raster call either way.

Implementation Steps

  1. PyAutoArray — autoarray/plot/utils.py::_default_colormap(). Split the two failure modes:
    • config unavailable / key absent (ImportError, KeyError, AttributeError, autoconf's missing-key exception) → quiet fallback to "autoarray" (unchanged behaviour, no warning: a fresh install has no config yet).
    • key present but the value neither "autoarray" nor resolvable via matplotlib.colormaps → raise ValueError naming the config file, the key path (visualize/general.yaml → colormap), the offending value, and a hint to use any matplotlib colormap name.
    • Do not leave a bare except Exception.
  2. Sibling readers. _conf_imshow_origin and _conf_output_format have the same bare-except shape. Tighten only if trivially safe in the same pass; otherwise note them in the PR as follow-up (they are not part of the acceptance gate).
  3. Tests — PyAutoArray/test_autoarray/plot/test_utils.py. Add unit tests for _default_colormap() using the same conf.instance[...] mutate/restore idiom the existing tests use (see test_arcsec_labels_default_suffix_format):
    • key absent → returns "autoarray" (and the custom colormap registers)
    • valid matplotlib name ("magma") → returned verbatim
    • bogus name ("nonexistent_cmap") → ValueError mentioning colormap and the bad value
      No JAX in unit tests.
  4. Docs — PyAutoArray/autoarray/config/visualize/README.md. Add a "Changing the colormap" section: the config key (global) and the colormap= argument on any plot_* function (per-figure), plus a note that array_overlay is deliberately fixed at Greys and that the weak-lensing / cluster plots use semantically-chosen fixed maps.
  5. Acceptance demo (not committed). A scratchpad script that sets colormap: magma, renders one imaging Array2D figure and one rectangular-mapper inversion figure to PNG, proving one config edit moves both.
  6. Euclid workspace (after the library PR opens). euclid_strong_lens_modeling_pipeline/config/visualize/general.yaml → colormap: magma; add the same "Changing the colormap" section to that repo's config/visualize/README.md.

Key Files

  • PyAutoArray/autoarray/plot/utils.py — _default_colormap() and its bare-except siblings
  • PyAutoArray/autoarray/plot/array.py, grid.py, inversion.py — the three consumers
  • PyAutoArray/autoarray/plot/segmentdata.py — the bundled autoarray colormap and its register()
  • PyAutoArray/autoarray/config/visualize/general.yaml — the colormap: key (stays autoarray)
  • PyAutoArray/test_autoarray/plot/test_utils.py — where the new tests go
  • PyAutoArray/autoarray/config/visualize/README.md — library-side docs
  • euclid_strong_lens_modeling_pipeline/config/visualize/general.yaml + README.md — the magma default

Documented deliberate exceptions (audited, not changed)

  • autoarray/plot/array.py:208 — cmap="Greys" for the array_overlay (a second array drawn over the main one; it must contrast with whatever the main colormap is).
  • autolens/weak/plot/weak_dataset_plots.py — viridis (ellipticities), twilight (cyclic position angles), magma (noise map).
  • autolens/weak/plot/fit_weak_plots.py:119 — RdBu_r (diverging, centred residuals).
  • autolens/weak/plot/convergence_plots.py:202 — magma.
  • autolens/cluster/plot/cluster_plots.py:49 — CLUSTER_CMAP = "gnuplot2".
  • autolens/potential_correction/* and autogalaxy/gui/clicker.py — jet in research/GUI-only code paths.

Original Prompt

Click to expand starting prompt 
# Colormap control: audit the cmap lever end-to-end and default the Euclid configs to magma

Type: feature
Target: PyAutoLens
Repos:
- PyAutoArray
- PyAutoLens
- autolens_workspace
- euclid_strong_lens_modeling_pipeline
Themes:
- visualization
- euclid
Difficulty: small-medium
Autonomy: safe
Priority: high
Status: formalised
Epic: euclid-dr1-prep
Phase: 0
Parent: draft/feature/euclid/euclid_dr1_prep_epic.md
Filed: 2026-08-28

Phase 0 of 10 in the Euclid DR1 preparation epic. Independent of every other phase —
it can start immediately and does not gate anything.

User request (verbatim):

"""
0) change default cmap to magma: lots of people compain about the cmap so make sure there is functionality to adjust 
cmap used throughout autolens and update euclid repo configs so that magma is used.
"""

## What already exists (surveyed 2026-08-28 — do not rebuild it)

The lever is already there. `PyAutoArray/autoarray/plot/utils.py::_default_colormap()`
reads `conf.instance["visualize"]["general"]["colormap"]` and falls back to the bundled
custom `"autoarray"` colormap (registered from
`PyAutoArray/autoarray/plot/segmentdata.py`, `COLORMAP_NAME = "autoarray"`). The key
`colormap: autoarray` is present in all three `config/visualize/general.yaml` files
(PyAutoArray, autolens_workspace, euclid_strong_lens_modeling_pipeline).

So the task is **audit + default change**, not lever construction.

## Deliverables

1. **Reach audit.** Confirm the config value actually reaches *every* 2D plotting
   surface, not just `plot/array.py` (which is the only confirmed consumer —
   `array.py:140-142` calls `_default_colormap()`). Check at minimum
   `PyAutoArray/autoarray/plot/inversion.py` (`_plot_rectangular`, `_plot_delaunay` both
   take a `colormap` argument — trace who supplies it), `plot/grid.py`, and any
   hardcoded colormap literal. `array.py:208` hardcodes `cmap="Greys"` for the
   `array_overlay`; decide whether that is deliberate (it probably is) and document it
   rather than silently leaving it undocumented.
2. **Per-figure override.** Verify a user can override the colormap for a single figure
   through the public plot API without editing config, and document how. If no such
   route exists, add the minimal one — prefer the lean existing lever over a new class.
3. **The silent fallback.** `_default_colormap()` (and its siblings
   `_conf_imshow_origin`, `_conf_output_format`) wrap the config read in a bare
   `except Exception`. A typo'd or missing key therefore fails silently to `autoarray`
   and the user never learns their setting was ignored. Tighten this so a genuinely
   absent config falls back quietly but a *malformed* value is loud. (Workspace
   convention: no silent guards.)
4. **Magma in the Euclid configs.** Set `colormap: magma` in
   `euclid_strong_lens_modeling_pipeline/config/visualize/general.yaml`. Leave the
   PyAutoArray/autolens_workspace defaults alone unless the user asks otherwise — the
   request scopes the magma change to "euclid repo configs".
5. **Documentation.** One short, discoverable place telling users how to change the
   colormap (config key + per-figure override). The `config/visualize/README.md` files
   are the natural home.

## Acceptance / gate

- A single config edit changes the colormap on every 2D figure the pipeline produces
  (imaging data, fits, residuals, inversion reconstructions), demonstrated on at least
  one imaging figure and one inversion figure.
- A malformed `colormap` value produces a clear error or warning, not a silent revert.
- The Euclid pipeline repo renders in magma out of the box.
- Nothing else in the epic gates on this; it can ship on its own.

Activity

  1. Jammy2211 commented on Aug 29, 2026

    @Jammy2211
    Jammy2211
    on Aug 29, 2026
    CollaboratorAuthor

    Library PR Created

    PR: #510 (feature/cmap-magma-default, label pending-release)
    Tests: python -m pytest test_autoarray -q -n auto → 1337 passed, 0 failed (82 warnings), commit c8fe47f.

    What landed

    • _default_colormap() no longer swallows a typo. An absent config (no autonerves, no colormap key) still falls back quietly to autoarray; a value matplotlib cannot resolve now raises a ValueError naming the config key, the offending value and the fix. The bare except Exception is gone.
    • _conf_imshow_origin() / _conf_output_format() get the same narrow excepts; imshow_origin additionally rejects anything but upper/lower.
    • autoarray/config/visualize/README.md gains a "Changing the colormap" section (config key, per-figure colormap=, and the figures that deliberately fix their colormap).
    • 11 new tests in test_autoarray/plot/test_utils.py, control-tested (the cmap spies were flipped to a sentinel and confirmed to fail, so they really observe the cmap reaching imshow).

    Reach audit result

    No 2D surface silently ignores the config key. All three PyAutoArray drawing modules default colormap=None → _default_colormap():

    Module Resolves at Raster call
    autoarray/plot/array.py 140-142 imshow (180, 187)
    autoarray/plot/grid.py 105-107 scatter (118-124)
    autoarray/plot/inversion.py 79-81 imshow/pcolormesh (198, 214), tripcolor (269)

    Everything above them (dataset/plot/, fit/plot/, inversion/plot/, PyAutoGalaxy's _resolve_colormap("default"), PyAutoLens's tracer_plots / sensitivity_plots / fit_imaging_plots) threads colormap through or leaves it None. Deliberate fixed-colormap figures (array_overlay Greys; the weak-lensing twilight/RdBu_r/viridis/magma; cluster gnuplot2; GUI/research jet) are unchanged and now documented rather than silent.

    Per-figure override

    Already existed, now documented and tested — colormap= on any plot function:

    aplt.plot_array(array=image, colormap="magma")
aplt.plot_inversion_reconstruction(pixel_values=values, mapper=mapper, colormap="viridis")

    Acceptance demo

    One config edit (colormap: magma) rendered both an imaging Array2D figure and a rectangular-mapper inversion reconstruction in magma — visually confirmed — and a malformed value (magmaa) raised:

    Unknown colormap 'magmaa'.

The config key `visualize/general.yaml -> colormap` is set to 'magmaa', which is not a
colormap matplotlib recognises, so no figure can be drawn with it.

Use either `autoarray` (the colormap bundled with PyAutoArray) or any matplotlib colormap
name, for example `magma`, `viridis`, `inferno`, `plasma` or `jet`.
The full list is `list(matplotlib.colormaps)`.

    Heart

    pyauto-heart readiness was RED at ship time for a reason unrelated to this task — verbatim: release validation FAILED (stage integrate). YELLOW reasons, also unrelated, verbatim: workspace validation not passing (2 failed, cloud#33179766004: autolens_test scripts/imaging/rectangular_mge.py, autolens_test scripts/imaging/rectangular_mge_rtu.py); manifest drift: session-start hooks (generated) — 2 mismatch(es) vs PyAutoMind/repos.yaml. The PR is opened under the standing human authorisation recorded in PyAutoMind/active.md on 2026-08-28 ("open prs under red and merge i acknowledge") and is not merged.

    Next: /start_workspace → euclid_strong_lens_modeling_pipeline (colormap: magma + docs), shipped behind the library-first merge gate.

  2. Jammy2211 commented on Aug 29, 2026

    @Jammy2211
    Jammy2211
    on Aug 29, 2026
    CollaboratorAuthor

    Both PRs Open — awaiting merge

    PR Contents
    Library #510 loud _default_colormap(), narrowed excepts, 11 tests, config/visualize/README.md
    Workspace PyAutoLabs/euclid_strong_lens_modeling_pipeline#42 colormap: magma, config/visualize/README.md, a ## Visualization section in the top-level README.md

    Both carry pending-release. Library-first merge gate: PyAutoArray#510 merges before euclid#42. Neither is merged; no /prm run.

    Acceptance gates

    • A single config edit changes the colormap on every 2D figure — verified two ways: the reach audit (all three PyAutoArray drawing modules resolve colormap=None through _default_colormap(), and every module above them threads it), and a rendered demo of an imaging Array2D figure plus a rectangular-mapper inversion reconstruction, both magma from the one key.
    • A malformed value is loud, not a silent revert — ValueError naming the key, the value and the fix; covered by two unit tests.
    • The Euclid pipeline renders in magma out of the box — pushing this repo's config/ onto conf.instance returns magma from _default_colormap().
    • Ships on its own — nothing else in the epic gates on it. PyAutoMind/draft/feature/euclid/euclid_dr1_prep_epic.md phase 0 updated to the new prompt path and marked in flight.

    Scope note

    PyAutoLens and PyAutoGalaxy were audited but not edited — their plot surfaces already thread colormap down to PyAutoArray, so the config key reaches them unchanged. PyAutoArray and autolens_workspace keep colormap: autoarray; the magma default is scoped to the Euclid repo as requested.

  3. Jammy2211
    added a commit that references this issue on Aug 29, 2026

    Merge pull request #510 from PyAutoLabs/feature/cmap-magma-default

    22191ec

  4. Jammy2211 commented on Aug 29, 2026

    @Jammy2211
    Jammy2211
    on Aug 29, 2026
    CollaboratorAuthor

    Shipped

    Both PRs merged 2026-08-28.

    What changed

    PyAutoArray — the visualize/general.yaml -> colormap lever is now loud and documented.

    • _default_colormap() separates absent from malformed: no autonerves or no colormap key still falls back quietly to autoarray; a value matplotlib cannot resolve now raises ValueError naming the key, the value and the fix. The bare except Exception is gone, replaced by ImportError / (KeyError, ConfigException).
    • Siblings tightened the same way — _conf_imshow_origin() (now rejects anything but upper/lower) and _conf_output_format().
    • Reach audit: every 2D raster call in PyAutoArray resolves its colormap from the config — plot/array.py (imshow), plot/grid.py (scatter), plot/inversion.py (imshow/pcolormesh/tripcolor). No surface silently ignores the key. The deliberate exceptions (the contrasting array_overlay Greys, cyclic twilight position angles, diverging RdBu_r residuals, the cluster/potential-correction maps) are now documented in autoarray/config/visualize/README.md rather than left silent.
    • Per-figure colormap= override documented and covered by a test — the route already existed, no new code needed.
    • 11 new tests in test_autoarray/plot/test_utils.py, control-tested (the cmap-spy assertions were flipped to a sentinel and confirmed to fail).

    euclid_strong_lens_modeling_pipeline — config/visualize/general.yaml: colormap: autoarray → colormap: magma, so the Euclid pipeline renders in magma out of the box, plus a "Changing the colormap" section in config/visualize/README.md and a Visualization section in README.md.

    API changes

    None — internal only. One user-visible behaviour change: a colormap or imshow_origin config value that was previously ignored now raises at plot time instead of silently reverting. Any config that was actually working is unaffected.

    Notes

    • PyAutoArray and autolens_workspace deliberately keep colormap: autoarray — the magma default is scoped to the Euclid repo, as the original request asked.
    • CI: PyAutoArray Tests [pull_request] green on all three legs (3.12, 3.13, nojax). The euclid pipeline repo runs no workflows of its own — url_check.yml was removed in 7ff9405 when URL hygiene moved to PyAutoPulse — so that PR carried no checks to judge; it is config + documentation only, with no scripts changed.
    • Phase 0 of the euclid-dr1-prep epic.
  5. Jammy2211
    closed this as completedon Aug 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions