Repository navigation
profiling(PyAutoArray#575): interferometer MGE breakdown on the library W~ route — CPU + A100 - #313
Merged
Merged
Conversation
…se submits (PyAutoArray#575) The MGE breakdown cell gains a --sparse arm that fits through the library on a dataset with apply_sparse_operator() applied, which since PyAutoArray#575 takes InversionInterferometerSparse for an MGE-only model. Steps/jit_phases in the dense schema, jitted FitInterferometer log_likelihood vs the dense reference (witness |dlogL| <= 1e-6), an MGE + lens Sersic variant with an uncorrected control, and a small sparse vmap probe. Dense and W~ arms kept (before/after in one JSON); outputs carry the _sparse suffix. The A100 submits prepend a private copy of the branch libraries to PYTHONPATH and assert the import location. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…VRAM sparse rows (PyAutoArray#575) CPU sma/alma and A100 alma/alma_high/jvla (RAL 356359-356361) on the #575 branch libraries: MGE-only fits take InversionInterferometerSparse; |dlogL| vs the dense reference 0.0 nats (CPU sma/alma; A100 vs CPU pins); A100 full pipeline 8.9 / 19.6 / 92.2 ms where the dense library path OOMs. MGE + lens Sersic sparse == dense (uncorrected control 779 nats off at alma). VMAP_BATCH_SPARSE gains interferometer mge alma 64, alma_high 64, jvla 16. Note addendum 'Library W~ route (2026-09-26)'; README dashboards regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…age NUFFT fix (PyAutoArray#575) PyAutoGalaxy 36d1b436 / PyAutoLens ac333b17 skip the Fourier transform of the all-zero standard-light image when there is no ordinary light. Re-ran CPU sma/alma and A100 alma/jvla (RAL 356362, 356363): full pipeline alma CPU 1.29 s -> 34.7 ms, alma A100 8.92 -> 2.69 ms, jvla A100 92.2 -> 16.7 ms; witness unchanged (|dlogL| 0.0 nats). jvla vmap 64 still asks 41.1 GiB, so the VMAP_BATCH_SPARSE jvla row stays 16 and its comment is corrected. Note section updated with round 1 / round 2 before/after; README dashboards regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the workspace half of PyAutoArray#575: the MGE breakdown cell re-run on the library W~ route (lever 1 of #308). With the library PRs applied, an MGE-only
FitInterferometeron a dataset withapply_sparse_operator()usesInversionInterferometerSparse, and the full likelihood matches dense to 0.0 nats wherever a dense reference exists. Where the old library path ran out of A100 memory, it now runs:The library route runs within 1.2–1.4x of the measurement-only W~ chain from #308. The MGE + lens Sersic variant matches dense exactly; with the dirty-image correction switched off it is 779 nats off at alma. Two measurement rounds are recorded: round 2 includes the zero-image NUFFT fix that was folded into PyAutoGalaxy#629 and PyAutoLens#750.
Not run:
apply_sparse_operator()build. This is recorded as a finding.VMAP_BATCH_SPARSEstays at 16 for jvla.Scripts Changed
scripts/interferometer/likelihood_breakdown/mge.py: new--sparselibrary W~ route arm (and--sparse-vmap-batch). It records the inversion class, library steps and jit phases alongside the dense rows (dense_steps), plus the witness and an MGE + lens Sersic variant; outputs get a_sparsesuffix. Existing arms are unchanged.hpc/batch_gpu/submit_breakdown_interferometer_mge_a100_{alma,alma_high,jvla}_fp64_sparse: new A100 submits.scripts/misc/vram/config.py: newVMAP_BATCH_SPARSErows (alma 64, alma_high 64, jvla 16). The dense rows stayNone, with a comment saying why.results/breakdown/interferometer/**/*_sparse.{json,png}, the "Library W~ route (2026-09-26)" section ofresults/notes/interferometer_mge_breakdown_2026_09.md, and the regenerated READMEs.Upstream PR
(Library-first: merge these three before this PR.)
Test Plan
build_readme.py --check,ruff check,ruff format --check,check_submits.py --checkpytest scripts/misc/test/: 912 passed, 5 skipped__file__andsource_revisionsrecorded); RAL A100 jobs 356359–356363 completed with clean logs, branch libs asserted, zero fp32 truncationsmanifest drift: hub organism blurb (organs present) — 7 mismatch(es) vs PyAutoMind/repos.yaml;manifest drift: organism-map blocks (generated) — 1 mismatch(es) vs PyAutoMind/repos.yaml), unrelated to this repo. The human authorized shipping this PR on 2026-09-26.🤖 Generated with Claude Code