Repository navigation
Description
Overview
Design an independent scientific correctness witness for the shipped HST/MGE CPU pilot. The human approved this bounded audit and documentation phase on 2026-10-13: "I approve, go".
Plan
- Trace the exact objective, resolved settings and reconstruction semantics.
- Assess independent reference candidates and shared-code limitations.
- Specify comparisons, tolerance decisions and a bounded implementation follow-up.
Tier: judge — merge mode: human /prm.
Branch survey
Only autolens_profiling is affected. Main at e4266a1; pre-existing untracked dataset/abell_1201/ must be preserved. No competing active claim. Use feature/hst-mge-pilot-qualification in .worktrees/hst-mge-pilot-qualification. Pulse's concurrent check-in owns campaign updates.
Detailed plan and original prompt
HST/MGE pilot: independent correctness qualification design
Filed: 2026-10-13
Type: research
Consequence: judge
Target: @autolens_profiling
Status: draft; awaiting approval of the bounded plan
Original request
ok continue
Context: continue after merging autolens_profiling#416 and PyAutoPulse#55 and
closing implementation issue #415. The raw measurement tooling is shipped;
independent scientific correctness and collection approval remain pending.
Canonical intent: https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/hst_mge_cpu_pilot.md
Previous phase: complete/2026/10/hst-mge-pilot-observations.md.
Proposed bounded phase
Audit the exact imaging-mge-runtime-v1 objective and design its independent
correctness witness before implementing or executing that witness. This phase
produces a source-grounded qualification design, not numerical acceptance.
- Trace current.from_dataset/build_workload and cell_builders.imaging_mge through
the actual AnalysisImaging/FitImaging objective and inversion settings. Pin
inspected source revisions; distinguish configured defaults from values only
knowable on the chosen execution host. Enumerate input, mask, PSF,
oversampling, parameter, coefficient-ordering, solver and normalization identity. - Audit candidate references and existing regression fixtures. Preserve
non-negativity, actual objective terms and reconstruction semantics. Explicitly
assess shared-code weaknesses; same-code eager/JIT agreement is insufficient.
Recommend an independent construction/solve route or record why unavailable. - Specify the witness artifact and comparisons: reference hashes/provenance,
likelihood terms, coefficients and reconstructed image, absolute/relative
errors, near-zero normalization, and conditioning/non-uniqueness handling.
Propose justified tolerance choices for human review, never silently accept
inherited constants or fabricate numerical evidence. - Document remaining collection choices (host, immutable stack, production
dataset, resolved settings, load method and resource ceiling) as unresolved
where no verified evidence exists. Define a separately bounded implementation
and synthetic validation follow-up, routed to the correctness-owning regression
workspace if required; do not expand this research phase into library fixes.
Planned files and validation
Add scripts/misc/jax_compile/QUALIFICATION.md with source links and the decision
table; add a short link from PILOT.md. No producer, library, test, default or
timing-record changes. Review every source citation and objective term, check
relative links, and run the applicable project documentation/generated checks.
Do not regenerate measurement timestamps or claim accepted baseline evidence.
No inference, production profiling, scientific witness execution, HPC submission,
baseline freeze/promotion or release. Production inputs and scientific tolerance
approval remain a later human decision. Pulse retains campaign state; this phase
does not duplicate its campaign or interfere with a concurrent Pulse check-in.
Tier: judge — merge mode: human /prm.
Activity
Jammy2211 commentedon Oct 13, 2026
Shipped design — 2026-10-13
PR: #418, head
3d6f83f73b84d81ed708129b09a97b4c873e5d3c. CI was running at publication; tier judge, human/prm.Added the source-pinned qualification design and a link from PILOT.md. It distinguishes the stabilized non-negative reconstruction from the reported data likelihood, assesses reference independence, proposes an independent operator/augmented NNLS approach, and specifies artifacts, comparisons and candidate error budgets. The small-dataset count/provenance limitation is documented for a later guard; runner code is unchanged.
Passed: ruff check/format, README/catalogue (including Pulse reader)/wiki/dashboard checks, 23 pinned-source links plus local links, and the existing MGE import-only smoke. Heart readiness is STALE only for
release validation incomplete: no rehearsal for current source, permitting development publication.No inference, scientific reference solve or production profiling ran. Numerical criteria remain proposals; coefficient tolerances, reference validation and production collection approval remain pending. The next bounded implementation belongs in the correctness regression workspace plus a project adapter, with separately planned synthetic execution caps. Pulse retains campaign choices and scientific acceptance.
Mind handoff:
active/hst_mge_pilot_qualification.md; task worktree and branch remain for review.Jammy2211 commentedon Oct 17, 2026
Shipped
PR #418 merged 2026-10-17 (human /prm after an independent review, verdict FINDINGS, none blocking):
scripts/misc/jax_compile/QUALIFICATION.mddesigns the independent HST/MGE pilot correctness qualification (what the pilot computes, an independent operator + scipy NNLS witness, identities to capture, candidate tolerances marked unaccepted). Still open, carried to the next bounded pilot step: background-load method (none proposed), the 1e-8 relative image L2 budget (likely unattainable; calibration on the synthetic harness must be allowed to revise it), and the correctness-process feasibility Pulse asked for before the 32-process budget is finalized. No compute, freeze or acceptance authorized.