Repository navigation
Description
Overview
autolens_profiling#332's memo lever (levers.memo in results/breakdown/interferometer/{delaunay,pixelization}_numba_hpc_ral_cpu_fp64_levers.json, alma r3.5) measured the fnnls warm-start memo 2.17× slower (mean, Delaunay solve) on an iid evaluation stream, 1.08× on rect. PyAutoArray#613 was addressed by #615 (scattered-stream back-off, released 2026.10.7.1). This issue owes the after-measurement of that same lever on the released 2026.10.17.1 stack. It reports a ratio with its interval; it does not conclude "fixed".
Human-authorized compute 2026-10-17 ("Do all that, I authorise all"). Measurement only: no library edits. Mind prompt active/interferometer_nnls_memo_after_measurement.md; Pulse task tasks/interferometer_nnls_memo_scattered_stream_guard.md.
Plan
- Same cells (
scripts/interferometer/{delaunay,rectangular}/likelihood_breakdown_numba.py), same dataset (alma, mask r3.5), same arm (numba, one thread) and same memo lever as research(interferometer): numba vs FFT CPU crossover + CPU lever list — campaign 3/3 phase 2 #332, at the five libraries'2026.10.17.1tags from a private RAL checkout. - CPU
--partition=ral(the research(interferometer): numba vs FFT CPU crossover + CPU lever list — campaign 3/3 phase 2 #332 job ran oneuclid-ral-gpu-2CPUs with no--gres; the host class therefore differs, and this is recorded beside the result). - Six independent rounds (one process each) per cell, so the ratio carries the repo's paired whole-round bootstrap 90 % interval (
round_bootstrap.round_median_ratio, ≥ 5 rounds perab_verdict.MIN_AB_ROUNDS), median headline (fix: intervals for the memo-policy, scaling and logdet gates — timing-noise phase 3b (#362) #410–fix: median headline everywhere, wall basis and repeat-summary support — timing-noise phase 10 (#362) #412). The original mean ratio is reported beside it for comparison with the 2.17×. - Two stream sizes: the exact research(interferometer): numba vs FFT CPU crossover + CPU lever list — campaign 3/3 phase 2 #332 flags (
--n-instances 6, so 4 timed iid solves) and--n-instances 17(16 iid instances, matching the walk). - Tier: judge (or undeclared) — merge mode: human /prm
Detailed plan
- Harness:
_lever_memo(scripts/misc/likelihood_breakdown/interferometer_pixelized_numpy.py) clears the memo withnnls_memo.memo_clear()when the library has it (2026.10.17.1 does). It clears the #615 back-off state too. Clearing only_nnls_passive_set_memo, as it did for research(interferometer): numba vs FFT CPU crossover + CPU lever list — campaign 3/3 phase 2 #332, would carry the iid pass's back-off into the walk pass. The method used is recorded in the lever block. - Submit
hpc/batch_cpu/submit_memo_after_interferometer_numba_ral_alma_fp64: one job, one node, rounds interleaved across cells and stream sizes; import guard on the private checkout; provenance header. - Summary script
scripts/interferometer/delaunay/memo_after_summary.pyreads the round JSONs and writesresults/breakdown/interferometer/memo_after_hpc_ral_cpu_fp64_v2026.10.17.1.json: per cell × stream the median on/off ratio with its paired round interval, the mean ratio, the figure-of-merit Δ, the per-solve slow-seed counts and the host/load. - A journal entry in
wiki/campaigns/interferometer_likelihood.md, then dashboard regen andship_workspace(one PR with research: interferometer streaming vs in-memory scaling (phase 1, CPU) #368's phase 2). Judge CI once; no merge.
Branch: feature/interferometer-streaming-memo-after.
Claim note: autolens_profiling is also claimed by hst-mge-pilot-qualification (#418, shipped, awaiting merge, touches only scripts/misc/jax_compile/*.md). The files do not overlap.
Original Prompt
B. Pulse task
tasks/interferometer_nnls_memo_scattered_stream_guard.md(PyAutoArray#613 fixed by #615, released 2026.10.7.1). Owed: re-run the autolens_profiling#332 ALMA Delaunay memo on/off measurement that showed 2.17× slowdown on scattered streams, now on 2026.10.17.1, same cell/dataset/host class as the original, CPUralpartition, quiet core, memo on vs off, iid vs walk streams. Report the ratio with its interval using the repo's timing-audit conventions (median headline, repeats as #410–#412 define). Never "fixed" as a conclusion — report the after-measurement.
Activity
fix: interferometer numpy breakdown harness reads a None (cost-model)…
results: streaming scaling phase 2 (RAL CPU 1e8 + A100) and fnnls mem…
Jammy2211 commentedon Oct 17, 2026
Measured on RAL at 2026.10.17.1 and shipped as PR #435 (awaiting human /prm). The tables, hardware, versions and caveats are in the PR body and the campaign wiki. Heart at ship: STALE, "release validation incomplete: no rehearsal for current source" (not RED, no override). RAL jobs: 412522 (ral CPU streaming), 412523 (A100 streaming, euclid-ral-gpu-1 exclusive), 412526 (memo after-measurement; 412521 crashed on the harness's float(None) gate read after PyAutoArray#632, fixed in 1e98f87).
Jammy2211 commentedon Oct 17, 2026
Shipped
PR #435 merged 2026-10-17: fnnls memo after-measurement on 2026.10.17.1 (RAL job 412526, ral compute-10-2, 1 thread, 6 rounds, median on/off with paired bootstrap 90 % CI). Delaunay iid at #332 flags 1.918 [1.887, 1.925] (was 2.17); 15 timed solves 1.018 [1.013, 1.023]; Delaunay walk 0.46; rect iid 0.887/0.987; rect walk 0.15; FoM identical. Reported as an after-measurement, not as fixed. Owed Nautilus-replay witness and varied-seed rounds remain research per the human decision 2026-10-17 (interferometer CPU lane closed).