docs: prepare HST/MGE CPU pilot contract

campaigns.yaml

@@ -5,10 +5,10 @@ campaigns:
   title: Setup baseline collection and scientific acceptance
   status: needs-decision
   updated: '2026-10-13'
-  next: Draft unchanged; no frozen revisions/environment lock, selected hardware/affinity, complete per-cell
-    settings/input hashes/witness tolerances or collection authorization. Compile builder remains explicitly
-    unavailable. Resolve the manifest choices, then request separate collection authorization.
-  evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/setup_baseline.md
+  next: HST/MGE fp64 CPU pilot preparation approved; review its draft protocol, resource ceiling and open
+    freeze choices. Manual HST jit builder merged (#414); global compile dispatch remains unavailable.
+    Producer gaps, exact host/stack/data and correctness witness remain unresolved. No compute authorized.
+  evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/baseline/campaign.json
 - id: point-source-image-cpu
   title: Point-source image plane · CPU
   status: active
@@ -110,12 +110,22 @@ tasks:
   priority: high
   path: tasks/setup_baseline_campaign.md
   updated: '2026-10-13'
-  next: Specification merged as draft v1 (autolens_profiling#385, 2026-10-06). Human resolves the six recorded
-    unknowns and freezes the specification; collection requires separate compute authorization. The report
-    prepares evidence for human scientific review and never promotes baselines. CPU production arrays use ral
-    only.
+  next: Prepare the HST/MGE CPU pilot first; review its protocol/resource proposal and resolve exact
+    host/stack/data/witness choices. Manual HST builder shipped, but collection producer remains pending.
+    Wider specification stays draft; collection, scientific acceptance and baseline promotion are separate.
   issue: https://github.com/PyAutoLabs/autolens_profiling/issues/384
   evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/719459fefb40c7e5be157f652818dbf923830557/baseline/campaign.json
+- id: hst_mge_cpu_pilot
+  campaign: setup-baseline
+  title: Prepare and review the HST/MGE fp64 CPU pilot
+  status: needs-decision
+  priority: high
+  path: tasks/hst_mge_cpu_pilot.md
+  updated: '2026-10-13'
+  next: Review the draft pilot, resolve host/stack/data/witness choices and plan the bounded project producer.
+    Proposed maximum 32 sequential processes, 16 CPU-hours; compute and freeze remain unauthorized.
+  issue: https://github.com/PyAutoLabs/PyAutoPulse/issues/53
+  evidence: https://github.com/PyAutoLabs/autolens_profiling/pull/414
 - id: runtime_cell_single_jit_gpu_warmup
   campaign: point-source-source
   title: Runtime cells' A100 `single_jit` includes the post-compile warm-up — source-plane 0.642 ms vs a steady

dashboard.md

@@ -1,6 +1,6 @@
 # PyAutoPulse — profiling dashboard
 
-<!-- pulse-campaigns:949e65e28d6d53e41dd3f86d7170e8b3419a385e6debf61756a518b0e1877ccd -->
+<!-- pulse-campaigns:b22a787746f3dc695099e748c7d1bb4ca52c932cb99185bf625878506ca73f04 -->
 ## Profiling Check In
 
 <details><summary>Full check-in prompt</summary>
@@ -31,13 +31,14 @@ Last check-in: 2026-10-13T14:47:44Z (review date, not measurement freshness).
 
 <details><summary>Setup baseline collection and scientific acceptance — needs-decision</summary>
 
-Reviewed 2026-10-13. Draft unchanged; no frozen revisions/environment lock, selected hardware/affinity, complete per-cell settings/input hashes/witness tolerances or collection authorization. Compile builder remains explicitly unavailable. Resolve the manifest choices, then request separate collection authorization.
+Reviewed 2026-10-13. HST/MGE fp64 CPU pilot preparation approved; review its draft protocol, resource ceiling and open freeze choices. Manual HST jit builder merged (#414); global compile dispatch remains unavailable. Producer gaps, exact host/stack/data and correctness witness remain unresolved. No compute authorized.
 
-[Campaign evidence](https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/setup_baseline.md)
+[Campaign evidence](https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/baseline/campaign.json)
 
 ### Active tasks
 
-- [Freeze the setup baseline specification, then collect and review evidence](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/setup_baseline_campaign.md) — needs-decision: Specification merged as draft v1 (autolens_profiling#385, 2026-10-06). Human resolves the six recorded unknowns and freezes the specification; collection requires separate compute authorization. The report prepares evidence for human scientific review and never promotes baselines. CPU production arrays use ral only.
+- [Freeze the setup baseline specification, then collect and review evidence](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/setup_baseline_campaign.md) — needs-decision: Prepare the HST/MGE CPU pilot first; review its protocol/resource proposal and resolve exact host/stack/data/witness choices. Manual HST builder shipped, but collection producer remains pending. Wider specification stays draft; collection, scientific acceptance and baseline promotion are separate.
+- [Prepare and review the HST/MGE fp64 CPU pilot](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/hst_mge_cpu_pilot.md) — needs-decision: Review the draft pilot, resolve host/stack/data/witness choices and plan the bounded project producer. Proposed maximum 32 sequential processes, 16 CPU-hours; compute and freeze remain unauthorized.
 
 </details>
 

tasks/hst_mge_cpu_pilot.md

@@ -0,0 +1,159 @@
+# HST/MGE fp64 CPU pilot — draft contract
+
+Campaign: setup-baseline
+Task: hst_mge_cpu_pilot
+Reviewed: 2026-10-13
+Status: needs-decision
+
+The human approved preparation in this chat ("ok go", 2026-10-13). This document
+proposes the experiment; it does not freeze a specification or authorize compute,
+scientific acceptance, defaults, pins or expansion of the wider baseline matrix.
+[Parent task](setup_baseline_campaign.md). Preparation: [Pulse #53](https://github.com/PyAutoLabs/PyAutoPulse/issues/53).
+
+## Question and useful outcome
+
+Can one current HST/MGE likelihood be measured repeatably on a specified RAL CPU,
+with auditable cold compilation, persistent-cache reuse and steady execution?
+The outcome is a qualified measurement procedure and a first within-configuration
+repeatability assessment. There is no compatible before/after baseline yet.
+Historical physical-vector posterior timings and this model-instance likelihood
+have different objectives; do not calculate a speedup or regression between them.
+A faster warm compile alone does not prove a cache hit or scientific correctness.
+
+## Verified capability, reviewed 2026-10-13
+
+[Project #414](https://github.com/PyAutoLabs/autolens_profiling/pull/414) merged at
+`b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76`. It supplies an explicit manual path
+for imaging/mge/hst, fp64, `jit` only. The global registry still declares the
+compile grid unavailable; the profiling conductor emits no dispatch. That global
+restriction does not remove the narrow manual builder. No project profile was
+run during this preparation, and no HPC state was queried.
+
+The baseline draft validator remains blocked, with 620 unverified, 366 unsupported
+and 58 not-applicable slots; specification SHA256:
+`468721f7e0e0c0d52957596ddbea964c61273a31d4371b65edeea45858c0a091`.
+These are readiness counts, not completed measurements. The 1,044-slot manifest
+is unchanged; a pilot never implicitly accepts or removes its remaining slots.
+
+## Workload proposed for the pilot
+
+- Objective: `imaging-mge-runtime-v1`, current `AnalysisImaging` likelihood/figure
+  of merit, without external prior or scaling; registered model-instance pytree
+  initialized from physical prior medians. Transform `jit`, batch size one.
+- Existing HST FITS: `data.fits`, `noise_map.fits`, `psf.fits`. No simulation on
+  missing inputs. Pixel scale 0.05 arcsec; circular mask radius 3.5 arcsec;
+  radial LP oversampling [4,2,2], radii [0.3,0.6], center (0,0).
+- Twenty Gaussian lens-light components and twenty Gaussian source components;
+  lens redshift 0.5, source redshift 1.0. Fixed Isothermal mass centered (0,0),
+  Einstein radius 1.6, axis ratio 0.9, angle 45 degrees; shear (0.05,0.05).
+  Dense MGE inversion; sparse-operator flag false. Actual solver/config defaults
+  and model priors must be captured, not reconstructed from this prose.
+- CPU backend required and verified at execution, x64 enabled and floating input
+  leaves fp64. No gradients, mixed precision, GPU, batching, solver change or
+  component breakdown in this first pilot. Host RSS is a resource diagnostic,
+  not acceptance of the broad campaign's memory axis.
+
+Dataset bytes, origin (simulated or observational), dimensions, PSF dimensions,
+masked-pixel count, resolved full model/solver configuration and every input leaf
+remain **unverified for the selected run**. Hash actual inputs and source/config
+files. If observational data are chosen, inspect image/contaminants and agree the
+mask before use. An instrument name or synthetic development fixture is not the
+production dataset identity.
+
+## Proposed measurement protocol
+
+1. Use one fixed named host, CPU model, affinity/core, thread configuration and
+   frozen environment across every leg. Capture topology, RAM, OS/kernel, Python,
+   JAX/jaxlib, numerical dependencies/build flags, CPU backend, actual process
+   affinity and effective thread/XLA/cache settings. Pin clean 40-character SHAs
+   for profiling, Nerves, Fit, Array, Galaxy and Lens, and hash an environment
+   lock reproducible on that host. The merged builder SHA is an implementation
+   reference, not a frozen executable stack.
+2. Ten cold compile repetitions: fresh process and fresh empty isolated persistent
+   cache per repetition. No pre-call/warmup before trace or compile. Measure
+   tracing plus lowering together (label honestly), then AOT compilation, then
+   first synchronized execution as three separate intervals. Keep setup and total
+   process wall time separately; never add compilation to steady runtime.
+3. Ten warm compile repetitions: each has its own separate seed process and a
+   matching cache, followed by a fresh measured process. Bind seed/run/cache to
+   identical software, flags, workload and input identities. Keep seed cost
+   separately. Require positive JAX cache-hit evidence associated with the
+   workload executable. File counts and faster timings are insufficient; absent
+   or ambiguous hit evidence means unknown cache state, not a warm result.
+   Interleave cold and seed/warm groups using a recorded order to expose drift.
+4. Separate steady-runtime process: record its setup, compilation and first
+   execution separately, then three untimed warmups and ten individual calls.
+   Synchronize all outputs before and after each timed call. Retain unrounded raw
+   seconds; report median, range and IQR. Ten calls in one process describe
+   within-process variability only, not ten independent process replications.
+   Retain each cold/warm process sample too; show dispersion and time order,
+   without inventing a speedup significance threshold.
+5. Collect host load alongside repetitions and the actual affinity/thread state.
+   The reference proposal is background load per allocated CPU <=0.1, but its
+   method must distinguish other work from this process and account for shared
+   host activity. Freeze that method before execution; do not subtract an assumed
+   one CPU from load averages. Missing host/load evidence remains unqualified.
+6. Correctness: choose an independently computed reference for the identical
+   objective, input and reconstruction, preserving reference bytes/hash and
+   implementation provenance. An independent NumPy/dense calculation is a
+   candidate only after objective/normalization parity is demonstrated. Comparing
+   the same JAX function eager versus compiled is a useful consistency test but
+   not the independent scientific witness. Human review must fix absolute AND
+   relative bounds and near-zero normalization before collection. No numerical
+   tolerances or reference are accepted by this draft. Gradients are out of scope.
+
+The current probe rounds samples, reports a three-call steady mean and deliberately
+labels configured caches unknown. It is not a baseline-observation producer.
+A project-owned runner must implement these missing observations before use;
+never convert old probe rows into fresh baseline evidence by relabeling them.
+
+## Proposed resources and stop rules — not a submission authorization
+
+RAL `ral` partition only; one task/process at a time, one allocated CPU with fixed
+affinity and one numerical thread; no GPU allocation. Propose 16 GiB RAM per
+process, subject to host selection and a reviewed resource preflight. Neither
+that memory request nor the duration is an observed requirement.
+
+Hard ceiling: 1,800 seconds per process, including setup. Ten cold + ten seeds +
+ten warm + one steady-runtime + one correctness process = at most 32 processes,
+16 allocated CPU-hours and 256 GiB-hours at the proposed memory request. Sequential
+execution is at most 16 compute wall-hours plus queue/driver overhead. All pilot
+work must fit the approved total; extra validation, retries or additional witness
+work require a revised budget. Development unit tests are separate from collection.
+
+Stop on timeout, OOM, correctness failure, backend/precision/provenance mismatch,
+missing inputs or invalid affinity/load evidence. Preserve logs and incomplete
+results; never raise caps, move to GPU or retry silently. Missing cache-hit proof
+blocks warm qualification. Caches live under allocated project output on RAL,
+never node /tmp or home. Select the exact node class/name, available RAM, storage
+quota, load method and collection window before approval; these remain unknown.
+
+## Ownership, deliverables and gates
+
+Pulse retains this intent and the outstanding decisions. The bounded Mind draft
+[HST/MGE pilot observations](https://github.com/PyAutoLabs/PyAutoMind/blob/9251122a/draft/feature/autolens_profiling/hst_mge_pilot_observations.md) covers a project-owned
+fresh-process runner, raw samples, cache evidence and honest observation output.
+The project will own the executable pilot specification, drivers, correctness
+artifacts, raw measurements and report. This document is not an executable spec.
+
+Before compute, require: reviewed producer; explicit pilot scope distinct from the
+full campaign; immutable project spec and stack/environment lock; named hardware,
+affinity and load method; complete hashed dataset/model/solver settings; independent
+witness and reviewed tolerances; accepted resource ceiling; collection window and
+separate human compute authorization. A partial broader baseline report must
+continue showing all missing cells; do not narrow the main manifest silently.
+
+The project report should preserve exclusions and failed/missing observations,
+separate integrity from qualification, and remain `accepted: false` pending human
+review. Scientific acceptance and any baseline/pin promotion are later decisions.
+
+## Dated source links
+
+Reviewed 2026-10-13 at project merge `b8b3b23`; source review does not refresh measurements:
+
+- [Current workload/provenance](https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/scripts/misc/jax_compile/current.py)
+- [MGE builder](https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/scripts/misc/cell_builders/imaging_mge.py)
+- [Probe measurement boundaries](https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/scripts/misc/jax_compile/probe.py)
+- [Baseline procedure](https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/baseline/README.md)
+- [Draft baseline manifest](https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/baseline/campaign.json)
+- [Global route registry](https://github.com/PyAutoLabs/autolens_profiling/blob/b8b3b23cd1f0aabc2d7d4ce1cd297f7420b14f76/catalogue/script_routes.json)

tasks/setup_baseline_campaign.md

@@ -87,3 +87,18 @@ the later collection and review deliverables.
 - Task next step: Specification merged as draft v1 (autolens_profiling#385, 2026-10-06). Human resolves the six recorded unknowns and freezes the specification; collection requires separate compute authorization. The report prepares evidence for human scientific review and never promotes baselines. CPU production arrays use ral only.
 - Evidence: [project main reviewed at 4db4d988](https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/measurement_tools.md), [Mind claims at 35281b1d](https://github.com/PyAutoLabs/PyAutoMind/blob/35281b1d/active.md), [full dated review](../checkins/2026-10-13.md). Review date does not refresh measurement dates.
 - Availability: project `hpc/sync jobs` returned an empty current queue; `hpc/sync status` timed out after 25 seconds during the output preview. Historical job outcomes, unpulled results and RAL cleanup contents are unverified; no collection or compute was performed.
+
+
+## Focused preparation 2026-10-13 — HST/MGE CPU pilot
+
+The human approved preparation of the [bounded pilot](hst_mge_cpu_pilot.md).
+[Project #414](https://github.com/PyAutoLabs/autolens_profiling/pull/414) merged:
+manual HST/MGE fp64 jit construction exists; the global compile grid remains
+unavailable. Earlier blanket builder-unavailable wording above is historical.
+The new probe is not yet a baseline observation producer and has no verified
+cache-hit witness. The wider manifest remains draft/blocked and unchanged.
+
+Next: review the pilot's protocol/resource proposal and unresolved host, stack,
+data and independent correctness choices; develop the bounded producer through
+Mind before separately authorized collection. No freeze, compute or acceptance
+was granted. This focused review does not change last_checkin or measurement dates.