tasks: correlated_nd pilot (B5c) — task row, campaign next, board

campaigns.yaml

@@ -92,9 +92,11 @@ campaigns:
 - id: gaussian-x3-search-benchmark
   title: PyAutoFit search benchmark on gaussian_x3 (autofit_inference)
   status: running
-  updated: '2026-10-15'
-  next: After the B5b-2 PRs merge, capture the fit snapshot from main under inference-summary@3, run
-    pyauto-insight check online and drop the registry's migrating_from; then B5c (a second model family).
+  updated: '2026-10-16'
+  next: 'B5c-2: RAL 410866 (the correlated_nd pilot, 930 runs, from the private copy inference/autofit_inference_b5c)
+    finished 2026-10-16 with 926/930 completed and 4 OUT_OF_MEMORY at 8 G; pull results/ only, settle the 4 OOM
+    cells (more memory or excluded), judge under correlated_nd@1, freeze correlated_nd@2 from the pilot and submit its scored wave; the family is
+    catalogued apart from gaussian_x3 and ranks nothing across families.'
   recent_progress: Wave 2 closed (B5b-2, 2026-10-15, autofit_inference#12) — RAL 409930 pulled, 12,750 rows
     admitted at the pin df6d218f10cd with 0 missing; NSS 50-step leads both evidence rankings; the catalogue
     ranks only legs with success rate >= 50 % over >= 20 attempts (post-hoc presentation convention); fit
@@ -175,6 +177,17 @@ tasks:
   next: Pilot stopped at 223 of 520 runs and is not restarted; freeze gaussian_x3@2 from its 17-row calibration
     before any scored run; the deferred cells move to scored wave 2 (after A2 and A4).
   evidence: https://github.com/PyAutoLabs/autofit_inference/issues/4
+- id: correlated_nd_pilot
+  campaign: gaussian-x3-search-benchmark
+  title: correlated_nd search benchmark — the pilot (wave 1, second model family)
+  status: running
+  priority: normal
+  path: tasks/correlated_nd_pilot.md
+  updated: '2026-10-16'
+  next: RAL 410866, the 930-run correlated_nd@1 pilot (protocol first committed 9ec37795f404, A1 before any
+    row), finished 2026-10-16 (926/930 completed, 4 OUT_OF_MEMORY at 8 G, results unpulled); B5c-2 pulls it,
+    settles the 4 OOM cells, judges it and freezes correlated_nd@2.
+  evidence: https://github.com/PyAutoLabs/autofit_inference/issues/14
 - id: gaussian_x3_search_wave2
   campaign: gaussian-x3-search-benchmark
   title: gaussian_x3 search benchmark — wave 2 (scored)

dashboard.md

@@ -1,6 +1,6 @@
 # PyAutoInsight — inference control room
 
-<!-- insight-inputs:ca0a1a5ea41ed220089ce9b6eb86ed7ffa7f918318e1790038d07fb2f246b2e6 -->
+<!-- insight-inputs:2a804822c123f7976eed584cc44bd8b96da5f21a9e9b7c8199758fc3172675de -->
 
 | Where | Count |
 |---|---:|
@@ -10,7 +10,7 @@
 | [Cached](#evidence) | 0 |
 | [Failed](#evidence) | 0 |
 
-<!-- insight-campaigns:bc4e241a4b624a5cbaceb3a77abc2c84cf5bb57a506760acc1f2fb98acc21210 -->
+<!-- insight-campaigns:c09fff24924c3c8bc5e2bfb0e9e37109d0202153cc34c91f3b9ee6281893322a -->
 ## Check in on all inference work
 
 Copy this into one chat. Add a campaign focus or idea before or after it, or leave it unchanged.
@@ -46,7 +46,7 @@ After taking action, report what changed, what the evidence supports and what re
 | [N=50 NUTS versus three EP repeats](https://github.com/PyAutoLabs/PyAutoCortex/blob/d45f198804615c090e6b6694cf725ddb0be2354e/projects/ep_toy_gaussian.md) | Cortex records failed wave 1 (EMFILE/OOM), wave-2 preparation and no submission. | Present the existing wave-2 proposal with current dependency evidence for decision. |
 | [Hierarchical slope recovery at scale](https://github.com/PyAutoLabs/PyAutoCortex/blob/d45f198804615c090e6b6694cf725ddb0be2354e/projects/slope_hierarchy_scale.md) | Cortex retains graphical baseline, post-processing OOM and stale EP hierarchical factor; Mind records later moments fix. | Verify the moments fix release and project mirror; recover missing summaries from existing samples before a proposed rerun. |
 | [IC50 EP and graphical scale ladder](https://github.com/PyAutoLabs/PyAutoCortex/blob/d45f198804615c090e6b6694cf725ddb0be2354e/projects/ic50_workspace.md) | Cortex records N=5/10/25 results, failed N=50 EP rung and width-interpretation caveat. | Check implementation close-outs, then propose a seeded ladder with appropriate output and mean-field diagnostics. |
-| [PyAutoFit search benchmark on gaussian_x3 (autofit_inference)](https://github.com/PyAutoLabs/autofit_inference/issues/12) | Wave 2 closed (B5b-2, 2026-10-15, autofit_inference#12) — RAL 409930 pulled, 12,750 rows admitted at the pin df6d218f10cd with 0 missing; NSS 50-step leads both evidence rankings; the catalogue ranks only legs with success rate &gt;= 50 % over &gt;= 20 attempts (post-hoc presentation convention); fit records published as inference-summary@3 (101 record shards). | After the B5b-2 PRs merge, capture the fit snapshot from main under inference-summary@3, run pyauto-insight check online and drop the registry&#x27;s migrating_from; then B5c (a second model family). |
+| [PyAutoFit search benchmark on gaussian_x3 (autofit_inference)](https://github.com/PyAutoLabs/autofit_inference/issues/12) | Wave 2 closed (B5b-2, 2026-10-15, autofit_inference#12) — RAL 409930 pulled, 12,750 rows admitted at the pin df6d218f10cd with 0 missing; NSS 50-step leads both evidence rankings; the catalogue ranks only legs with success rate &gt;= 50 % over &gt;= 20 attempts (post-hoc presentation convention); fit records published as inference-summary@3 (101 record shards). | B5c-2: RAL 410866 (the correlated_nd pilot, 930 runs, from the private copy inference/autofit_inference_b5c) finished 2026-10-16 with 926/930 completed and 4 OUT_OF_MEMORY at 8 G; pull results/ only, settle the 4 OOM cells (more memory or excluded), judge under correlated_nd@1, freeze correlated_nd@2 from the pilot and submit its scored wave; the family is catalogued apart from gaussian_x3 and ranks nothing across families. |
 
 ## Active tasks
 
@@ -58,6 +58,7 @@ After taking action, report what changed, what the evidence supports and what re
 | [Graphical Model Scale-Up — Scoping](https://github.com/PyAutoLabs/PyAutoInsight/blob/main/tasks/graphical_scoping.md) | graphical-ep | needs-slicing | high | Reconcile current Cortex scale results and library dependencies; select one bounded inference phase with a witness. |
 | [NUTS/HMC trial on validated likelihoods](https://github.com/PyAutoLabs/PyAutoInsight/blob/main/tasks/gradient_sampler_validated_likelihoods.md) | point-source-admission | needs-decision | normal | An unscoped idea, not permission to submit: reconcile point-source admission with imaging and weak-lensing candidate targets. |
 | [gaussian_x3 search benchmark — wave 1 (the pilot)](https://github.com/PyAutoLabs/PyAutoInsight/blob/main/tasks/gaussian_x3_search_wave1.md) | gaussian-x3-search-benchmark | blocked | normal | Pilot stopped at 223 of 520 runs and is not restarted; freeze gaussian_x3@2 from its 17-row calibration before any scored run; the deferred cells move to scored wave 2 (after A2 and A4). |
+| [correlated_nd search benchmark — the pilot (wave 1, second model family)](https://github.com/PyAutoLabs/PyAutoInsight/blob/main/tasks/correlated_nd_pilot.md) | gaussian-x3-search-benchmark | running | normal | RAL 410866, the 930-run correlated_nd@1 pilot (protocol first committed 9ec37795f404, A1 before any row), finished 2026-10-16 (926/930 completed, 4 OUT_OF_MEMORY at 8 G, results unpulled); B5c-2 pulls it, settles the 4 OOM cells, judges it and freezes correlated_nd@2. |
 | [gaussian_x3 search benchmark — wave 2 (scored)](https://github.com/PyAutoLabs/PyAutoInsight/blob/main/tasks/gaussian_x3_search_wave2.md) | gaussian-x3-search-benchmark | running | high | Wave 2 is closed and judged (B5b-2); the fit snapshot is captured from main under inference-summary@3 after the B5b-2 PRs merge. |
 
 [Ledger](https://github.com/PyAutoLabs/PyAutoInsight/blob/main/campaigns.yaml)

tasks/correlated_nd_pilot.md

@@ -0,0 +1,78 @@
+# correlated_nd search benchmark — the pilot (wave 1)
+
+Type: research
+Target: autofit_inference
+Campaign: gaussian-x3-search-benchmark
+Epic: search-extensibility (phase B5c)
+Issue: https://github.com/PyAutoLabs/autofit_inference/issues/14
+Filed: 2026-10-15
+
+## Intent
+
+The second toy-likelihood family of the PyAutoFit search benchmark, so rankings stop
+resting on `gaussian_x3` alone. `correlated_nd` stresses scaling with dimension and
+adaptation to an ill-conditioned, correlated or curved posterior. It has correlated
+Gaussians at D = 20, 50 and 100, each with condition number 10³, and a D = 20 hybrid
+Rosenbrock banana. Their posteriors and ln Z are **exact by construction**.
+
+The rows are judged by the family's own pre-registered protocol `correlated_nd@1`
+([`wiki/project/protocol_correlated_nd.md`](https://github.com/PyAutoLabs/autofit_inference/blob/main/wiki/project/protocol_correlated_nd.md)).
+It was first committed as autofit_inference `9ec37795f404`, before any row, and amended
+by A1 before any row or submission. The catalogue groups this family by task and
+dataset, in its own `correlated_nd@1` section, separately from `gaussian_x3`. **Nothing
+is ranked across families**, and the pilot ranks nothing at all.
+
+## Menu and identity
+
+The menu is the `gaussian_x3` wave-2 search menu, unchanged. It runs 10 search seeds
+(0–9) × realisation 1 × every non-deferred cell × the four datasets.
+
+- **Planned: 930 runs.** The family expects 1040 runs in all.
+- **Deferred: 110 runs, each with its reason** (protocol §7):
+  - Emcee and Zeus at D ≥ 50: their 50 walkers are fewer than 2D, so both samplers
+    refuse to run.
+  - DynestyStatic `nlive_200` and NSS at D = 100: `n_live` is below the exact
+    information H = 267.6 nat.
+
+The runs go on RAL `--partition=ral` against the private PyAutoFit pin `df6d218f10cd`.
+Each run has a 4 h operational process cap. A capped run is a censored failure, not a
+protocol timeout.
+
+A row belongs to the pilot only when its key
+`(dataset, sampler, settings_name, config_id, data_seed, protocol_id, pyautofit_sha)` is a
+cell of the pilot, and its `model` is `correlated_nd`
+(`scripts/misc/searches/_correlated_nd.py`).
+
+## References
+
+Every dataset × backend has an exact reference, complete in `results/reference/` (8 of
+8). Each one passed the §6 agreement test:
+
+- the backend's `Fitness` equals the analytic ln L to ≤ 6e-14 nat;
+- 200 000 exact draws reproduce every median within 0.006 σ;
+- the control verdicts are accepted.
+
+## Submission (2026-10-15, B5c)
+
+- RAL job **410866_[0-929]** (`hpc/batch_cpu/submit_correlated_nd_pilot`): 930 one-run array
+  tasks, `%100`, from the private copy `/mnt/ral/jnightin/inference/autofit_inference_b5c`.
+  It was pending at submission.
+- **Finished 2026-10-16** (wall 4:16): 926 of 930 tasks COMPLETED, 4 OUT_OF_MEMORY at the
+  8 G request, all on `corr_gaussian_d50`: task 263 (`nautilus` `n_live_100`, numpy, seed
+  3), 273 (`nautilus` `n_live_100`, jax_cpu, seed 3), 304 (`nautilus` `n_live_400`, numpy,
+  seed 4) and 383 (`blackjax_nuts_warm` `warmup_200_samples_500`, jax_cpu, seed 3). By the
+  human's decision they are not rerun in stage 1; B5c-2 raises their memory or excludes
+  them, with the reason recorded. The results stay unpulled in the private copy.
+- The first submission, 409957, failed at import in every task: the shared RAL clone had
+  been switched by another session. It was cancelled and wrote no row.
+
+This task authorises nothing further. Pulling, judging and the `correlated_nd@2` freeze
+are B5c-2.
+
+## Next (B5c-2)
+
+1. Pull the pilot's rows (`results/` only) and judge them under `correlated_nd@1`; settle
+   the 4 OOM cells (more memory or excluded).
+2. Calibrate the PLACEHOLDERs and timeouts, then freeze `correlated_nd@2` before any
+   scored row.
+3. Submit the scored wave: 50 seeds × realisations 1–3 × every non-deferred cell.