B5c-2a: autofit_inference — correlated_nd pilot judged under @1, correlated_nd@2 frozen (44c5e77f1fb0); Now -> B5c-2b

dashboard.html

@@ -333,7 +333,7 @@
 - Open autolens_inference: https://github.com/PyAutoLabs/autolens_inference
 - Open autofit_inference: https://github.com/PyAutoLabs/autofit_inference
 - Open euclid_detailed: https://github.com/PyAutoLabs/euclid_detailed
-- Open cowls: https://github.com/PyAutoLabs/cowls</textarea></details><div class="orchestration-freshness" data-freshness="grey"><details data-freshness-stamp data-refreshed-at="2026-10-16T10:04:26Z"><summary><span data-freshness-label>Last updated 2026-10-16 10:04:26 UTC</span></summary><time datetime="2026-10-16T10:04:26Z">2026-10-16 10:04:26 UTC</time></details><a data-refresh-link href="https://github.com/PyAutoLabs/PyAutoCortex/actions/workflows/dashboard_refresh.yml" title="Open the dashboard refresh controls">↻ Update</a></div></div><p class="orchestration-status" role="status" aria-live="polite"></p></section>
+- Open cowls: https://github.com/PyAutoLabs/cowls</textarea></details><div class="orchestration-freshness" data-freshness="grey"><details data-freshness-stamp data-refreshed-at="2026-10-16T11:38:08Z"><summary><span data-freshness-label>Last updated 2026-10-16 11:38:08 UTC</span></summary><time datetime="2026-10-16T11:38:08Z">2026-10-16 11:38:08 UTC</time></details><a data-refresh-link href="https://github.com/PyAutoLabs/PyAutoCortex/actions/workflows/dashboard_refresh.yml" title="Open the dashboard refresh controls">↻ Update</a></div></div><p class="orchestration-status" role="status" aria-live="polite"></p></section>
 <nav class="board-nav" aria-label="Board sections"><a class="board-nav-card" href="#summary"><span class="board-nav-count">22</span><span class="board-nav-label">Running</span></a><a class="board-nav-card" href="#summary"><span class="board-nav-count">47</span><span class="board-nav-label">Open</span></a><a class="board-nav-card" href="#projects"><span class="board-nav-count">11</span><span class="board-nav-label">Projects</span></a><a class="board-nav-card" href="#orchestration-cortex"><span class="board-nav-label">Check in</span></a></nav>
 
 <p class="muted mdsrc"><a href="https://github.com/PyAutoLabs/PyAutoCortex/blob/main/dashboard.md" class="board-source-link" title="Markdown version" aria-label="Markdown version"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" focusable="false"><path d="M14 3H6a1 1 0 0 0-1 1v16a1 1 0 0 0 1 1h12a1 1 0 0 0 1-1V8Z"/><path d="M14 3v5h5M8 12h8M8 16h6"/></svg></a></p>
@@ -655,18 +655,18 @@ <h4>Runs (6)</h4>
 <dt>RAL:</dt><dd>/mnt/ral/jnightin/inference/autofit_inference</dd>
 </dl><div class="position"><div class="position-text">
 <h4>Current position</h4>
-<p>B5c (2026-10-15, autofit_inference#14): a second model family, correlated_nd (correlated Gaussians D = 20/50/100, condition number 10^3, and a D = 20 hybrid Rosenbrock banana, exact posteriors and ln Z), pre-registered as correlated_nd@1 (first committed 9ec37795f404; amendment A1 before any row) with seeded datasets and eight exact references that pass the section 6 agreement test. Its 930-run pilot RAL 410866 finished 2026-10-16 (926/930 completed, 4 OUT_OF_MEMORY at 8 G on corr_gaussian_d50, carried to B5c-2; 409957 failed at import on the disturbed shared clone and was cancelled) on --partition=ral from the private copy inference/autofit_inference_b5c at the pin df6d218f10cd; it ranks nothing and is catalogued apart from gaussian_x3 (nothing ranked across families). gaussian_x3 wave 2 stays closed and judged under gaussian_x3@2. Next: B5c-2 pulls the pilot (results/ only) from the private copy, settles the 4 OOM cells (more memory or excluded), judges it under @1, freezes correlated_nd@2 and submits the scored wave.</p></div>
+<p>B5c-2a (2026-10-16, autofit_inference#17): the correlated_nd pilot (RAL 410866) is pulled from the private copy and judged under correlated_nd@1 (1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred; 307 accepted, 623 rejected; ranks nothing, nothing ranked across families), and correlated_nd@2 is frozen in autofit_inference 44c5e77f1fb0 (protocol section 11 A2) before any scored row: sigma-ratio bands per task (evidence [0.85, 1.1] on 79 rows, posterior kept at [0.5, 2.0] on 1 row), ESS/R-hat bars kept, 75 timed + 8 probe-first cells, 21 cells deferred with reasons (incl. d50 Nautilus n_live_100 by the cap rule and all d100 Nautilus by A2), memory 2x the pilot peak (d50 Nautilus n_live_400 16 G, d100 cold NUTS 17 G). Calibration: autofit_inference results/calibration/correlated_nd_wave1.json. gaussian_x3 wave 2 stays closed under gaussian_x3@2. Next: B5c-2b (draft/research/autofit_inference/search_ext_b5c2b_scored_wave_submit.md) adds realisations 2-3 and their exact references, the 12 450-run scored-wave manifest and its probe, and submits on --partition=ral from a private pinned copy; nothing is submitted until then.</p></div>
 <button class="copy resume" aria-label="Copy resume prompt for autofit_inference" title="Copy resume prompt for autofit_inference" data-cmd="Use the cortex skill. — resume autofit_inference: read PyAutoCortex projects/autofit_inference.md (Now, Runs, Log) and then /home/jammy/Code/PyAutoLabs/fit/autofit_inference/wiki/project/state.md and the assistant autofit_assistant's AGENTS.md; tell me where I left off and what I said I would do next. Submit nothing and log nothing until I say.">📋</button>
 </div>
 <h4>Runs (0)</h4>
 <p class="muted">nothing on the cluster</p>
-<details class="sub"><summary>Recent log — 5 of 27 entries</summary>
+<details class="sub"><summary>Recent log — 5 of 28 entries</summary>
 <ul class="log">
+<li>2026-10-16 — <i>result</i> — <details class="full"><summary><span class="clip">correlated_nd pilot RAL 410866 judged under correlated_nd@1 (B5c-2a, autofit_inference#17): results/ pulled from the private copy inference/autofit_inference_b5c (930 rows; exact references byte-identical); 1040 expected = 749 completed ro…</span><span class="less">show less</span></summary><p>correlated_nd pilot RAL 410866 judged under correlated_nd@1 (B5c-2a, autofit_inference#17): results/ pulled from the private copy inference/autofit_inference_b5c (930 rows; exact references byte-identical); 1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred, reconciled with sacct; 307 accepted, 623 rejected; ranks nothing. Failures: 80 Emcee/Zeus (d20, banana) crashed in the harness's chain diagnostics after the search (relabelling by g0.centre; fixed, no verdict moves); 60 d100 Nautilus (n_live_100: needs n_live &gt; D; n_live_200/400: nautilus HDF5 checkpoint cannot store a D = 100 bound); 10 d100 warm NUTS (provider); 27 at the 4 h cap (d50 Nautilus n_live_100, d50 warm NUTS). The 4 OUT_OF_MEMORY tasks 263/273/304/383 (MaxRSS 7.6/7.7/3.1/8.4 G at 8 G) had been given cap stubs by the exit-137 path and are now an oom attempt class. A2 ruling: the OOM cells d50 Nautilus n_live_100 and d50 warm NUTS are deferred by the cap rule (no attempt ended on its own); d50 Nautilus n_live_400 gets 16 G and d100 cold NUTS 17 G (2x the measured peak, 8 G for an OOM class); d100 Nautilus n_live_200 (and 100, 400) is deferred (fails at the pin before sampling, affordability unmeasured); an oom run is rerun once at 2x. correlated_nd@2 frozen in autofit_inference 44c5e77f1fb0 (44c5e77f1fb0779f675a8f51247d812183e93b48) before any scored row; nothing submitted.</p></details></li>
 <li>2026-10-16 — <i>run</i> — <details class="full"><summary><span class="clip">410866_[0-929] finished — wall 4:16: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs,…</span><span class="less">show less</span></summary><p>410866_[0-929] finished — wall 4:16: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2 — 926/930 COMPLETED, 4 OUT_OF_MEMORY at 8 G, all corr_gaussian_d50: task 263 nautilus n_live_100 numpy seed 3, 273 nautilus n_live_100 jax_cpu seed 3, 304 nautilus n_live_400 numpy seed 4, 383 blackjax_nuts_warm warmup_200_samples_500 jax_cpu seed 3; not rerun in stage 1 (human decision), B5c-2 raises their memory or excludes them; results unpulled in the private copy, pulled and judged in B5c-2</p></details></li>
 <li>2026-10-15 — <i>run</i> — <details class="full"><summary><span class="clip">410866_[0-929] submitted: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x…</span><span class="less">show less</span></summary><p>410866_[0-929] submitted: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2</p></details></li>
 <li>2026-10-15 — <i>run</i> — <details class="full"><summary><span class="clip">409957_[0-929] failed — wall 0:06: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbro…</span><span class="less">show less</span></summary><p>409957_[0-929] failed — wall 0:06: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, one run per task, 4 h process cap, %100; submit_correlated_nd_pilot; pin df6d218f10cd; pulled in B5c-2 — every task failed at import (ModuleNotFoundError _autofit_inference_cli): the shared RAL clone had been switched to an old main by another session (stash pre-main-switch 19:24 UTC) and lost the CLI and config/ as the array started; 800 tasks exited, the rest scancelled; no row written; resubmitted from the private copy inference/autofit_inference_b5c as 410866</p></details></li>
 <li>2026-10-15 — <i>run</i> — <details class="full"><summary><span class="clip">409957_[0-929] submitted: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, o…</span><span class="less">show less</span></summary><p>409957_[0-929] submitted: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, one run per task, 4 h process cap, %100; submit_correlated_nd_pilot; pin df6d218f10cd; pulled in B5c-2</p></details></li>
-<li>2026-10-15 — <i>run</i> — <details class="full"><summary><span class="clip">409930_[0-19] finished — wall 1:30: wave-2 probe-first remainder (B5b): Zeus JAX + NSS n_live_200_mcmc_steps_50, both datasets, 980 runs after the probe under probe-derived @2 timeouts (NSS 600 s, Zeus 1168 s); %20; submit_wave2_remainder;…</span><span class="less">show less</span></summary><p>409930_[0-19] finished — wall 1:30: wave-2 probe-first remainder (B5b): Zeus JAX + NSS n_live_200_mcmc_steps_50, both datasets, 980 runs after the probe under probe-derived @2 timeouts (NSS 600 s, Zeus 1168 s); %20; submit_wave2_remainder; pin df6d218f10cd; pulled in B5b-2 — 20/20 COMPLETED (16:08-17:38 UTC); 980 remainder rows pulled by per-directory tar (20 probe rows byte-identical); all admitted under gaussian_x3@2, 0 missing; judged B5b-2</p></details></li>
 </ul>
 <p class="muted"><a href="https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/autofit_inference.md">full log</a></p>
 <p class="muted">active · ral · <a href="https://github.com/PyAutoLabs/autofit_inference/issues/8">autofit_inference#8</a></p>

dashboard.md

@@ -524,7 +524,7 @@ active · both · [autolens_inference#4](https://github.com/PyAutoLabs/autolens_
 
 **Current position**
 
-B5c (2026-10-15, autofit_inference#14): a second model family, correlated_nd (correlated Gaussians D = 20/50/100, condition number 10^3, and a D = 20 hybrid Rosenbrock banana, exact posteriors and ln Z), pre-registered as correlated_nd@1 (first committed 9ec37795f404; amendment A1 before any row) with seeded datasets and eight exact references that pass the section 6 agreement test. Its 930-run pilot RAL 410866 finished 2026-10-16 (926/930 completed, 4 OUT_OF_MEMORY at 8 G on corr_gaussian_d50, carried to B5c-2; 409957 failed at import on the disturbed shared clone and was cancelled) on --partition=ral from the private copy inference/autofit_inference_b5c at the pin df6d218f10cd; it ranks nothing and is catalogued apart from gaussian_x3 (nothing ranked across families). gaussian_x3 wave 2 stays closed and judged under gaussian_x3@2. Next: B5c-2 pulls the pilot (results/ only) from the private copy, settles the 4 OOM cells (more memory or excluded), judges it under @1, freezes correlated_nd@2 and submits the scored wave.
+B5c-2a (2026-10-16, autofit_inference#17): the correlated_nd pilot (RAL 410866) is pulled from the private copy and judged under correlated_nd@1 (1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred; 307 accepted, 623 rejected; ranks nothing, nothing ranked across families), and correlated_nd@2 is frozen in autofit_inference 44c5e77f1fb0 (protocol section 11 A2) before any scored row: sigma-ratio bands per task (evidence [0.85, 1.1] on 79 rows, posterior kept at [0.5, 2.0] on 1 row), ESS/R-hat bars kept, 75 timed + 8 probe-first cells, 21 cells deferred with reasons (incl. d50 Nautilus n_live_100 by the cap rule and all d100 Nautilus by A2), memory 2x the pilot peak (d50 Nautilus n_live_400 16 G, d100 cold NUTS 17 G). Calibration: autofit_inference results/calibration/correlated_nd_wave1.json. gaussian_x3 wave 2 stays closed under gaussian_x3@2. Next: B5c-2b (draft/research/autofit_inference/search_ext_b5c2b_scored_wave_submit.md) adds realisations 2-3 and their exact references, the 12 450-run scored-wave manifest and its probe, and submits on --partition=ral from a private pinned copy; nothing is submitted until then.
 
 <details><summary title="Copy resume prompt for autofit_inference">📋</summary>
 
@@ -538,13 +538,13 @@ Use the cortex skill. — resume autofit_inference: read PyAutoCortex projects/a
 
 _nothing on the cluster_
 
-<details><summary>Recent log — 5 of 27 entries</summary>
+<details><summary>Recent log — 5 of 28 entries</summary>
 
+- 2026-10-16 — *result* — correlated_nd pilot RAL 410866 judged under correlated_nd@1 (B5c-2a, autofit_inference#17): results/ pulled from the private copy inference/autofit_inference_b5c (930 rows; exact references byte-identical); 1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred, reconciled with sacct; 307 accepted, 623 rejected; ranks nothing. Failures: 80 Emcee/Zeus (d20, banana) crashed in the harness's chain diagnostics after the search (relabelling by g0.centre; fixed, no verdict moves); 60 d100 Nautilus (n_live_100: needs n_live > D; n_live_200/400: nautilus HDF5 checkpoint cannot store a D = 100 bound); 10 d100 warm NUTS (provider); 27 at the 4 h cap (d50 Nautilus n_live_100, d50 warm NUTS). The 4 OUT_OF_MEMORY tasks 263/273/304/383 (MaxRSS 7.6/7.7/3.1/8.4 G at 8 G) had been given cap stubs by the exit-137 path and are now an oom attempt class. A2 ruling: the OOM cells d50 Nautilus n_live_100 and d50 warm NUTS are deferred by the cap rule (no attempt ended on its own); d50 Nautilus n_live_400 gets 16 G and d100 cold NUTS 17 G (2x the measured peak, 8 G for an OOM class); d100 Nautilus n_live_200 (and 100, 400) is deferred (fails at the pin before sampling, affordability unmeasured); an oom run is rerun once at 2x. correlated_nd@2 frozen in autofit_inference 44c5e77f1fb0 (44c5e77f1fb0779f675a8f51247d812183e93b48) before any scored row; nothing submitted.
 - 2026-10-16 — *run* — 410866_[0-929] finished — wall 4:16: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2 — 926/930 COMPLETED, 4 OUT_OF_MEMORY at 8 G, all corr_gaussian_d50: task 263 nautilus n_live_100 numpy seed 3, 273 nautilus n_live_100 jax_cpu seed 3, 304 nautilus n_live_400 numpy seed 4, 383 blackjax_nuts_warm warmup_200_samples_500 jax_cpu seed 3; not rerun in stage 1 (human decision), B5c-2 raises their memory or excludes them; results unpulled in the private copy, pulled and judged in B5c-2
 - 2026-10-15 — *run* — 410866_[0-929] submitted: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2
 - 2026-10-15 — *run* — 409957_[0-929] failed — wall 0:06: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, one run per task, 4 h process cap, %100; submit_correlated_nd_pilot; pin df6d218f10cd; pulled in B5c-2 — every task failed at import (ModuleNotFoundError _autofit_inference_cli): the shared RAL clone had been switched to an old main by another session (stash pre-main-switch 19:24 UTC) and lost the CLI and config/ as the array started; 800 tasks exited, the rest scancelled; no row written; resubmitted from the private copy inference/autofit_inference_b5c as 410866
 - 2026-10-15 — *run* — 409957_[0-929] submitted: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, one run per task, 4 h process cap, %100; submit_correlated_nd_pilot; pin df6d218f10cd; pulled in B5c-2
-- 2026-10-15 — *run* — 409930_[0-19] finished — wall 1:30: wave-2 probe-first remainder (B5b): Zeus JAX + NSS n_live_200_mcmc_steps_50, both datasets, 980 runs after the probe under probe-derived @2 timeouts (NSS 600 s, Zeus 1168 s); %20; submit_wave2_remainder; pin df6d218f10cd; pulled in B5b-2 — 20/20 COMPLETED (16:08-17:38 UTC); 980 remainder rows pulled by per-directory tar (20 probe rows byte-identical); all admitted under gaussian_x3@2, 0 missing; judged B5b-2
 
 [full log](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/autofit_inference.md)