Repository navigation
fix(inversion): guard data_subtracted_dict on data=None, harden sparse noise check and data-term dtype #590
Description
Overview
Corrective follow-up to phase 1 of https://github.com/orgs/PyAutoLabs/discussions/13 (PyAutoArray#589, PyAutoGalaxy#637, PyAutoLens#757). An independent Codex review reproduced three defects on the sparse data=None path (record: PyAutoLabs/PyAutoLens#757 (comment)): AbstractInversion.data_subtracted_dict raises TypeError / yields {mapper: None} when the inversion was built without data, which breaks subplot_of_mapper / subplot_mappings and the two workspace interferometer/features/pixelization/fit.py scripts; check_noise_map_real_imag_equal uses an absolute tolerance of 1e-8; and apply_sparse_operator squares complex64 data in float32 so the cached data_term disagrees with the chunked builder.
Plan
- Raise a clear
InversionExceptionfromdata_subtracted_dictwhen the inversion has no data, pointing atfit.inversion_with_data. - Catch that exception in the two mapper subplots so the data-subtracted panel is skipped cleanly.
- Set
atol=0.0on the real/imag noise equality check. - Promote to complex128 before computing the cached data term so one-shot equals chunked for any input dtype.
- Add tests for all four; then migrate the two workspace pixelization
fit.pyscripts tofit.inversion_with_data(library first, workspace follows).
Detailed implementation plan
Affected Repositories
- PyAutoArray (primary)
- autogalaxy_workspace, autolens_workspace (workspace follow-up,
/ship_workspace)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./PyAutoArray | main | clean |
| ./autogalaxy_workspace | main | clean |
| ./autolens_workspace | main | clean |
Worktree guard clear on all three.
Suggested branch: feature/sparse-data-none-guard
Implementation Steps
autoarray/inversion/inversion/abstract.pydata_subtracted_dict(~L868):if self.data is None: raise exc.InversionException(...)namingfit.inversion_with_data/fit.data.autoarray/inversion/plot/inversion_plots.py: theexcept (AttributeError, KeyError)guards at ~L87 (subplot_of_mapper) and ~L397 (subplot_mappings) also catchexc.InversionException; do not widen to bareException.autoarray/inversion/inversion/interferometer/inversion_interferometer_util.pycheck_noise_map_real_imag_equal(~L58, L61):atol=0.0onnp.allcloseandnp.isclose, default rtol kept.autoarray/dataset/interferometer/dataset.pyapply_sparse_operator(~L391-394): computedata_termfrom complex128-promoted local copies of data and noise; dirty image expression unchanged.- Tests:
DatasetInterface(data=None)inversion →pytest.raises(exc.InversionException, match="inversion_with_data"); both subplots run whendata_subtracted_dictraises; sigma1e-9+2e-9j→exc.DatasetException, equal 1e-9 sigmas pass; complex64apply_sparse_operatordata_term== complex128 value ==sparse_terms_from_chunks(...).data_term. - Workspace (after the library PR):
scripts/interferometer/features/pixelization/fit.pyin autogalaxy_workspace (~L319) and autolens_workspace (~L291):inversion = fit.inversion_with_dataplus one sentence of prose; regenerate notebooks per each workspace's convention; smoke both scripts.
Key Files
autoarray/inversion/inversion/abstract.py— guardautoarray/inversion/plot/inversion_plots.py— handlersautoarray/inversion/inversion/interferometer/inversion_interferometer_util.py— noise checkautoarray/dataset/interferometer/dataset.py— data term dtypeautogalaxy_workspace/autolens_workspacescripts/interferometer/features/pixelization/fit.py— accessor migration
Original Prompt
Click to expand starting prompt
Sparse data=None path: guard data_subtracted_dict, harden noise check and data-term dtype (Discussion #13 review fixes)
Type: bug
Target: PyAutoArray
Repos:
- PyAutoArray
- autogalaxy_workspace
- autolens_workspace
Themes: - interferometer
- sparse-operator
- visualization
Difficulty: small
Autonomy: supervised
Priority: high
Status: draft
Consequence: glance
Witness: on a sparse-operator dataset with the precomputed-data-term gate on (fit.inversion.dataset.data is None),fit.inversion.data_subtracted_dictraisesInversionExceptionnaminginversion_with_data(no TypeError, no{mapper: None}), andInversionPlotter.subplot_of_mapper/subplot_mappingson that inversion skip cleanly instead of raising;check_noise_map_real_imag_equalrejects sigma 1e-9 vs 2e-9;apply_sparse_operatoron complex64 data givesdata_termequal to the complex128 value and tosparse_terms_from_chunks; the two workspaceinterferometer/features/pixelization/fit.pyscripts run under the smoke profile usingfit.inversion_with_data.
Review-minutes: 5
Unattended: ready
Parent: complete/2026/09/interferometer-streaming-visibilities.md
Source: Codex gpt-6-astra review of PyAutoArray#589 / PyAutoGalaxy#637 / PyAutoLens#757
(record: PyAutoLabs/PyAutoLens#757 (comment)), all four
claims reproduced 2026-09-30.
Why
Phase 1 of https://github.com/orgs/PyAutoLabs/discussions/13 builds the sparse inversion with
data=None when no galaxy has a non-linear light profile. Three review findings follow from
the merged code:
- B (introduced, user-facing).
AbstractInversion.data_subtracted_dict
(autoarray/inversion/inversion/abstract.py~L868-885) doescopy.copy(self.data)then
-=; withdata=Nonea multi-object inversion raises
TypeError: unsupported operand type(s) for -: 'NoneType' and 'complex'and a single-mapper
inversion yields{mapper: None}, which makessubplot_of_mapperfail with aValueError
inplot/array.py. The handlers inautoarray/plot/.../inversion_plots.py(~L87, ~L397)
catch only(AttributeError, KeyError). The library visualizers already pass
fit.inversion_with_data, butautogalaxy_workspace/scripts/interferometer/features/pixelization/fit.py
(~L321) passesfit.inversionand now fails on PyAutoGalaxy main; the autolens_workspace copy
(~L293) fails once PyAutoLens#757 lands. - D (pre-existing, hardening).
check_noise_map_real_imag_equal
(inversion_interferometer_util.py~L58, L61) usesnp.allclose/np.iscloseat default
atol=1e-8, so sigma 1e-9 vs 2e-9 passes and the sparse curvature is 1.5x wrong. Unreachable
with Jy-unit data but a scale-dependent check is wrong on principle. - A (edge, pre-existing path).
Interferometer.apply_sparse_operator
(autoarray/dataset/interferometer/dataset.py~L391-394) squaresdata.real/data.imagin
the data's own dtype; complex64 data + complex128 noise givesdata_term700140000.0 vs exact
700140007.0 (log_evidence +3.5), and disagrees withsparse_terms_from_chunks, which promotes
to complex128. Loaders always yield complex128, so only hand-built complex64 arrays hit it.
Finding C (noise map mutated after apply_sparse_operator stales the operator) predates phase 1
and is out of scope; document only if convenient.
What
AbstractInversion.data_subtracted_dict: whenself.data is None, raise
exc.InversionExceptionwith a message pointing atfit.inversion_with_data(andfit.data).inversion_plots.pyhandlers arounddata_subtracted_dictreads: also catch
exc.InversionException(andTypeError), so the mapper subplots skip that panel cleanly.check_noise_map_real_imag_equal:atol=0.0on both calls (keep the default rtol).apply_sparse_operator: cast data (and noise) to float64 components before squaring so
one-shot == chunked for any input dtype.- Tests in
test_autoarray: (1) raises with the named message; (2) plotter skips; (3) the
1e-9/2e-9 rejection; (4) complex64 data_term == complex128 == chunked. - Workspace: both
interferometer/features/pixelization/fit.pyscripts →fit.inversion_with_data
in the inversion plotter call, with a one-line prose note on why; regenerate notebooks per
each workspace's convention. Library first (/ship_library), workspace follows
(/ship_workspace).
🤖 Generated with Claude Code
Activity
Jammy2211 commentedon Sep 30, 2026
Library PR created
#591
🤖 Generated with Claude Code
Jammy2211 commentedon Sep 30, 2026
Shipped
data_subtracted_dictguard + plotter handlers,atol=0.0noise check, complex128 data term. Labelledpending-release.fit.py→fit.inversion_with_data.Merged library-first after PyAutoLens#757. Review finding C (noise map mutated after
apply_sparse_operator) is pre-existing and left as is.🤖 Generated with Claude Code