Skip to content

research: fnnls memo after-measurement on the #332 ALMA cells (2026.10.17.1) #426

Closed

Description

@Jammy2211

Overview

autolens_profiling#332's memo lever (levers.memo in results/breakdown/interferometer/{delaunay,pixelization}_numba_hpc_ral_cpu_fp64_levers.json, alma r3.5) measured the fnnls warm-start memo 2.17× slower (mean, Delaunay solve) on an iid evaluation stream, 1.08× on rect. PyAutoArray#613 was addressed by #615 (scattered-stream back-off, released 2026.10.7.1). This issue owes the after-measurement of that same lever on the released 2026.10.17.1 stack. It reports a ratio with its interval; it does not conclude "fixed".

Human-authorized compute 2026-10-17 ("Do all that, I authorise all"). Measurement only: no library edits. Mind prompt active/interferometer_nnls_memo_after_measurement.md; Pulse task tasks/interferometer_nnls_memo_scattered_stream_guard.md.

Plan

Detailed plan

  1. Harness: _lever_memo (scripts/misc/likelihood_breakdown/interferometer_pixelized_numpy.py) clears the memo with nnls_memo.memo_clear() when the library has it (2026.10.17.1 does). It clears the #615 back-off state too. Clearing only _nnls_passive_set_memo, as it did for research(interferometer): numba vs FFT CPU crossover + CPU lever list — campaign 3/3 phase 2 #332, would carry the iid pass's back-off into the walk pass. The method used is recorded in the lever block.
  2. Submit hpc/batch_cpu/submit_memo_after_interferometer_numba_ral_alma_fp64: one job, one node, rounds interleaved across cells and stream sizes; import guard on the private checkout; provenance header.
  3. Summary script scripts/interferometer/delaunay/memo_after_summary.py reads the round JSONs and writes results/breakdown/interferometer/memo_after_hpc_ral_cpu_fp64_v2026.10.17.1.json: per cell × stream the median on/off ratio with its paired round interval, the mean ratio, the figure-of-merit Δ, the per-solve slow-seed counts and the host/load.
  4. A journal entry in wiki/campaigns/interferometer_likelihood.md, then dashboard regen and ship_workspace (one PR with research: interferometer streaming vs in-memory scaling (phase 1, CPU) #368's phase 2). Judge CI once; no merge.

Branch: feature/interferometer-streaming-memo-after.
Claim note: autolens_profiling is also claimed by hst-mge-pilot-qualification (#418, shipped, awaiting merge, touches only scripts/misc/jax_compile/*.md). The files do not overlap.

Original Prompt

B. Pulse task tasks/interferometer_nnls_memo_scattered_stream_guard.md (PyAutoArray#613 fixed by #615, released 2026.10.7.1). Owed: re-run the autolens_profiling#332 ALMA Delaunay memo on/off measurement that showed 2.17× slowdown on scattered streams, now on 2026.10.17.1, same cell/dataset/host class as the original, CPU ral partition, quiet core, memo on vs off, iid vs walk streams. Report the ratio with its interval using the repo's timing-audit conventions (median headline, repeats as #410–#412 define). Never "fixed" as a conclusion — report the after-measurement.

Activity

  1. Jammy2211 commented on Oct 17, 2026

    @Jammy2211
    ContributorAuthor

    Measured on RAL at 2026.10.17.1 and shipped as PR #435 (awaiting human /prm). The tables, hardware, versions and caveats are in the PR body and the campaign wiki. Heart at ship: STALE, "release validation incomplete: no rehearsal for current source" (not RED, no override). RAL jobs: 412522 (ral CPU streaming), 412523 (A100 streaming, euclid-ral-gpu-1 exclusive), 412526 (memo after-measurement; 412521 crashed on the harness's float(None) gate read after PyAutoArray#632, fixed in 1e98f87).

  2. Jammy2211 commented on Oct 17, 2026

    @Jammy2211
    ContributorAuthor

    Shipped

    PR #435 merged 2026-10-17: fnnls memo after-measurement on 2026.10.17.1 (RAL job 412526, ral compute-10-2, 1 thread, 6 rounds, median on/off with paired bootstrap 90 % CI). Delaunay iid at #332 flags 1.918 [1.887, 1.925] (was 2.17); 15 timed solves 1.018 [1.013, 1.023]; Delaunay walk 0.46; rect iid 0.887/0.987; rect walk 0.15; FoM identical. Reported as an after-measurement, not as fixed. Owed Nautilus-replay witness and varied-seed rounds remain research per the human decision 2026-10-17 (interferometer CPU lane closed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions