Skip to content

research: phase 2a imaging profiling campaign at standard size (likelihood-lifecycle) #456

Closed

Description

@Jammy2211

Overview

Phase 2a of the likelihood-lifecycle epic: re-measure all 106 planned imaging cells at the standard size on the pinned tag 2026.10.23.1, as current rows with runtime, breakdown, fit.png, provenance and parity.json. Plan and rulings approved by the human on 2026-10-24.

Plan

  • PR A, code (laptop): standard size wired into the runtime and numba scripts, a parity.json writer (eager NumPy reference), and new RAL CPU and A100 submits with wall-basis blocks.
  • Batch 0 (human go): short timing arms to set --time.
  • Batch 1 (human go): all 106 cells from a private RAL checkout.
  • PR B: commit the results, regenerate the catalogue and coverage, write the campaign wiki page, refetch Pulse, adopt hst_mge_cpu_pilot.

Tier: judge — merge mode: human /prm

Original Prompt

Click to expand starting prompt

Likelihood lifecycle phase 2a: profiling campaign — imaging/{mge, rectangular, delaunay, delaunay_nn} (106 planned cells)

Type: research
Target: autolens_profiling
Repos:

  • autolens_profiling
    Themes:
  • profiling
    Difficulty: large
    Autonomy: supervised
    Priority: high
    Status: formalised
    Consequence: judge
    Witness: the coverage matrix shows zero missing and zero stale cells for class ∈ {optimal, secondary} × measurement ∈ {runtime, breakdown} for this campaign's likelihoods; every new current cell carries fit.png, provenance and parity.json; build_catalogue --check --validate-with ../PyAutoPulse valid at @3; the campaign wiki page records the pre-registered rule and verdict
    Review-minutes: 45
    Unattended: never
    Epic: likelihood-lifecycle
    Phase: 2
    Parent: draft/research/autolens_profiling/likelihood_lifecycle_epic.md
    Filed: 2026-10-24

Phase 2 sub-issue 2a of the likelihood-lifecycle epic (report §4 "Phase 2"; ledger "Phase 2" card). Depends on phase 0 (complete) and phase 1B (merged, autolens_profiling#453); start after autolens_profiling#455 (follow-ups) merges. Scope: imaging/{mge, rectangular, delaunay, delaunay_nn} (106 planned cells).

Size: the 2026-10-17 imaging results are at Delaunay 1500 / rectangular 39², not the standard size, so every cell is re-measured at 1250 / 35². That reuse is no longer possible, and the ledger card's "reuse" line is superseded by the size ruling. First, add the missing parity.json writer (#451 defined only the format): each cell computes its log-likelihood against a named reference and writes parity.json beside the row. Adopts Pulse hst_mge_cpu_pilot.

Rules (every phase-2 sub-issue; epic ledger "Phase 2" card)

  • The coverage matrix (python scripts/misc/tooling/coverage_matrix.py → dashboard/coverage.md) names this campaign's missing and stale cells; those cells are the work list.
  • Every cell is measured at the standard size (Delaunay 1250, rectangular 35² = 1225; human ruling 2026-10-23) on a pinned released tag, from a private RAL checkout (never the shared mirror). CPU arrays go on --partition=ral only; A100 runs use --gres. The human gives a go for each RAL batch. Never touch another campaign's jobs.
  • Every cell lands as current, carrying runtime and breakdown, provenance, fit.png (phase 1B renders it automatically) and a parity.json against a reference log-likelihood. Prior rows on the key become superseded through the catalogue builder.
  • A campaign page at the mirrored wiki path (wiki/campaigns/<dataset_type>/<variant>/) records the rule and the expected verdict before the runs, then the result.
  • Pulse receipt refetched after merge. Phase 2 ends only on the human's "campaigns complete" confirmation (the gate for phase 3).

Plan approved 2026-10-24 (human), with rulings

  • Pinned tag: 2026.10.23.1 (the latest PyPI release), shared by every phase-2 sub-issue. Rows on other likelihoods go stale and are re-measured in 2b–2g.
  • Standard Delaunay = 1250 Hilbert + 30 edge (1280 recorded). Rows record source_pixels_requested: 1250 beside the actual count, and the size rule matches on the requested count. Rectangular = 35².
  • PR A (code, laptop):
    • --source-pixels in the 3 JAX runtime scripts (hard-coded 1500 / 39² today) and the numba scripts (presets; jwst/ao have none); a --standard-size flag.
    • A parity.json writer: an in-process eager NumPy reference on the same inputs and size, absolute tolerance 1e-4·|ref| (1e-3 mixed).
    • New submits: a RAL CPU array on --partition=ral for the 44 CPU cells, plus one --gres=gpu:1 A100 job per instrument. The 18 jax_cpu_sparse@hpc_a100 cells run inside those GPU jobs. Each submit gets a WALL-BASIS block.
  • Batch 0 (human go): one short timing arm per new submit kind, to set --time from measurement.
  • Batch 1 (human go): all 106 cells from a private RAL checkout at the pinned tag.
  • PR B: commit the rows, regenerate the catalogue and coverage, write the campaign wiki page (pre-registered rule: every cell current with parity match; any divergence blocks that setup), refetch Pulse, and adopt and close Pulse hst_mge_cpu_pilot.
  • Tier: judge, merge by human /prm. Starts after autolens_profiling#455 merges.

Activity

  1. Jammy2211 commented on Oct 24, 2026

    @Jammy2211
    ContributorAuthor

    PR A review rulings (human, 2026-10-24):

    1. N1: under --standard-size, JAX pixelized cells use the same production model as the numba cells (Delaunay 1250 + 30 edge, rectangular 35², production MGE, weights and regularisation), so optimal and secondary measure the same likelihood.
    2. (c): JAX sparse cells check parity against the independent eager dense-mapping reference.
    3. Fixes: the delaunay_nn witness is not_applicable; batch 1 writes to a scratch tree that PR B copies from; RAL memory is sized per instrument.

    Independent review, first pass: FINDINGS. These fixes are being applied before re-review.

  2. Jammy2211 commented on Oct 24, 2026

    @Jammy2211
    ContributorAuthor

    PR A shipped as #457. The independent review took three rounds to reach CLEAN, with human rulings along the way: N1, a single production model for JAX and numba; (c), a dense parity reference; and matching real production (RectangularBilinearAdaptImage, over-sampling cut on the uncapped S/N map). Next: human /prm, then batch 0 (timing) on RAL with the human's go.

  3. added 2 commits that reference this issue on Oct 24, 2026
  4. Jammy2211 commented on Oct 25, 2026

    @Jammy2211
    ContributorAuthor

    Shipped: phase 2a imaging campaign

    All work for this issue is merged. Everything ran at tag 2026.10.23.1 from a private RAL checkout of the five libraries.

    PR What
    #457 PR A: standard-size production cells (Delaunay 1250 + 30 edge, rectangular 35²), parity.json, submits
    #458 Batch-0 breakdown fixes: sparse triplets masked like the library; pins only for instruments that have one; standard-size sparse build time; rectangular Adapt
    #459 Guard test: interferometer/datacube W~ triplets mirror the library call
    #460 Batch 1 --time/--mem from batch 0's measured walls and MaxRSS
    #461 PR B: 104 batch-1 rows (53 cells), 13 superseded rows moved to history/, catalogue/dashboard/coverage regenerated, wiki page wiki/campaigns/imaging/likelihood_lifecycle_p2a.md; release_pin from library SHAs
    PyAutoPulse#75 Lens receipt refetched (679 records at eef5edf); hst_mge_cpu_pilot superseded (#415 closed)

    Results:

    • Runs: batch 0 (RAL jobs 412877 and 412884, plus reruns 412911 and 412915) completed 32 of 34. Batch 1 (412923–412926 and 412927, 412943–412945) completed 104 of 106. The 2 missing in each are the known limitation: the ao jax_gpu_dense breakdowns run out of A100 memory.
    • Parity: 53/53 cells match.
    • Coverage, all 246 planned cells: current / stale / missing went from 2 / 6 / 238 to 104 / 7 / 135. Imaging is 104 current; only the 2 limitations are missing.
    • Speed: jax_gpu_dense@a100 is the fastest per-call setup for every likelihood and instrument. Choosing the optimal setup is phase 3's job, and it must also account for numba's multi-process throughput on CPU nodes. That follow-up is being planned.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions