Skip to content

docs: design independent HST MGE pilot qualification - #418

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/hst-mge-pilot-qualification
Oct 17, 2026
Merged

Jammy2211 merged 1 commit into
mainfrom
feature/hst-mge-pilot-qualification

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

The raw HST/MGE pilot can measure execution but has no independent scientific correctness witness. Document the exact calculation to qualify, the proposed independent reference and the decisions required before collection.

The audit distinguishes the stabilized non-negative reconstruction from the reported Gaussian data likelihood: the diagonal stabilization affects the coefficients but contributes no extra likelihood term for these unregularized bases. It proposes independently constructed imaging columns and an augmented NNLS reference, with explicit shared-code limitations, artifact identity, comparison metrics and unaccepted tolerance candidates. It also records the small-dataset environment/count mismatch as a follow-up guard requirement.

Closes #417. This completes the approved qualification-design phase only. Coefficient thresholds, executable witness validation, production context and collection approval remain pending. No inference, numerical reference solve or production profiling ran.

Scripts Changed

  • scripts/misc/jax_compile/QUALIFICATION.md: source-pinned audit, reference design and decision table.
  • scripts/misc/jax_compile/PILOT.md: link to the design.

No executable scripts, tests, defaults, measurements or generated timing metadata changed.

Test Plan

  • Ruff check and format check passed (288 Python files unchanged).
  • README, catalogue with independent Pulse contract reader, wiki and dashboard checks passed.
  • All 23 pinned source links verified against Git objects and origin/main; local document links and explicit reference definitions checked.
  • Existing MGE import-only smoke passed; it exited before likelihood execution.
  • Heart readiness: STALE solely for release validation incomplete: no rehearsal for current source; development publication permitted, no release.

Tier: judge — human /prm. The numerical thresholds in this document are proposals for review, not accepted scientific criteria.

@Jammy2211

Copy link
Copy Markdown
Contributor Author

Review 2026-10-17 — FINDINGS (none blocking a doc-only merge)

Independent read of QUALIFICATION.md against the #416 runner, the Pulse pilot task and choices docs, and the pinned library sources. No contradiction with #416 (environment filter, 20/20 Gaussians, small-dataset switch, raw tolerance formula, PDIP routing, 1e-3 diagonal and the 31-process / 15.5 h figures all match). Nothing asserts acceptance or authorizes compute; nothing it calls unmerged has since merged.

Open questions vs the design:

Question Status
Data origin / suitability partial — records what to capture, picks nothing, does not cite the Pulse choices' FITS hashes (:99, :234)
Host / stack / full settings settings answered; host and stack deferred to selection (:97–106, :235)
Witness and tolerances answered except the coefficient threshold, explicitly left as a blocker (:116–164, :179–185, :233)
Background-load method missing — the row restates the need, proposes no method (:236)
No compute / freeze / acceptance answered (:3, :233, :237)

Findings:

  1. Medium. The candidate relative image L2 budget of 1e-8 (:183) is likely unattainable: packaged config allows up to 3.3e-4 fnnls flux on inactive columns after polish, and the doc forbids loosening tolerances after seeing production disagreements (:16–18). Calibration on the synthetic harness must be allowed to revise these numbers, or the pilot fails by construction.
  2. Medium. Pulse hst_mge_cpu_pilot_choices.md:121 asks docs: design HST MGE pilot correctness qualification #417 to establish correctness-process feasibility before a final budget; the design defers it (:236, :244), so the 32-process / 16 CPU-hour ceiling stays unfinalized.
  3. Low. Witness lives in autolens_workspace_test with a project adapter (:14–16, :232) while Pulse says the project owns correctness artefacts; the adapter probably reconciles this — human to confirm.
  4. Low. Source pins are four days old; MGE-relevant keys unchanged; the doc already requires a re-audit before collection (:29–30).
  5. Info. wiki/campaigns/measurement_tools.md has no HST/MGE pilot section; ledger gap, not a doc defect.

Branch is 30 commits behind main but touches only scripts/misc/jax_compile/, so no conflict; worth an update + link/catalogue check before /prm.

@Jammy2211
Jammy2211 merged commit 3f12474 into main Oct 17, 2026
1 check passed
@Jammy2211
Jammy2211 deleted the feature/hst-mge-pilot-qualification branch October 17, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: design HST MGE pilot correctness qualification

1 participant