diff --git a/dashboard.html b/dashboard.html index 9905217..2598f93 100644 --- a/dashboard.html +++ b/dashboard.html @@ -237,6 +237,7 @@ + @@ -273,6 +274,14 @@

analytic_gaussian

tasks/analytic_gaussian/

+

ep_toy_gaussian

+

Local /mnt/c/Users/Jammy/Science/ep_toy_gaussian

+

RAL /mnt/ral/jnightin/ep_toy_gaussian

+

active ยท ral partition ยท tasks: submitted 1

+

Do three EP fits agree with NUTS at N=50submitted โ€” runs 342639, 342640 ยท wall 0:00 of 24:00 (0%)

+

tasks/ep_toy_gaussian/

+
+

euclid_dr1_prelim

Local /mnt/c/Users/Jammy/Science/euclid_dr1_prelim

RAL /mnt/ral/jnightin/euclid_dr1_prelim

diff --git a/dashboard.md b/dashboard.md index 9c48071..547178a 100644 --- a/dashboard.md +++ b/dashboard.md @@ -30,6 +30,7 @@ One row per active project โ€” the next thing, what it is holding, and when it l | Project | Next task | Awaiting | Running | Last ruling | |---|---|---:|---:|---| | analytic_gaussian | Is criterion 2's mu threshold or the minimal-EP control wrong | 0 | 0 | 2026-09-10 | +| ep_toy_gaussian | Do three EP fits agree with NUTS at N=50 | 0 | 1 | - | | euclid_dr1_prelim | Do ordered Nautilus reruns reproduce the euclid catalogue values? | 1 | 0 | 2026-09-10 | | ic50_workspace | Does EP match the graphical joint fit end to end | 0 | 1 | - | | slope_hierarchy_scale | Does hierarchical slope recovery hold at N=25 to 50 | 0 | 1 | - | @@ -130,6 +131,24 @@ Finish: write today's journal entry, rewrite wiki/project/state.md, and return ( [tasks/analytic_gaussian/](https://github.com/PyAutoLabs/PyAutoCortex/tree/main/tasks/analytic_gaussian/) +### ep_toy_gaussian + +**Local** `/mnt/c/Users/Jammy/Science/ep_toy_gaussian` + +**RAL** `/mnt/ral/jnightin/ep_toy_gaussian` + +active ยท ral partition ยท tasks: submitted 1 + +
๐Ÿ“‹ Do three EP fits agree with NUTS at N=50 โ€” submitted โ€” runs 342639, 342640 ยท wall 0:00 of 24:00 (0%) + +``` +cd /mnt/c/Users/Jammy/Science/ep_toy_gaussian && hpc/sync jobs +``` + +
+ +[tasks/ep_toy_gaussian/](https://github.com/PyAutoLabs/PyAutoCortex/tree/main/tasks/ep_toy_gaussian/) + ### euclid_dr1_prelim **Local** `/mnt/c/Users/Jammy/Science/euclid_dr1_prelim` diff --git a/projects.yaml b/projects.yaml index 59a7c59..623c086 100644 --- a/projects.yaml +++ b/projects.yaml @@ -115,6 +115,20 @@ analytic_gaussian: status: active note: "born 2026-09-09 as the graphical-ep campaign's phase-1 keystone at ensemble scale; benchmark model with a closed-form posterior, borrowed from autofit_workspace_test scripts/graphical (PR #92 lineage); CPU-only, no dataset (every draw is simulated in memory from its seed); remote none until the human runs gh repo create PyAutoLabs/analytic_gaussian --private" +ep_toy_gaussian: + remote: none + local_path: /mnt/c/Users/Jammy/Science/ep_toy_gaussian + ral_root: /mnt/ral/jnightin/ep_toy_gaussian + mirror: none + sync_cli: hpc/sync + sync_verbs: [push, pull, logs, sync, status, submit, push-submit, jobs, sacct, cancel, tail, du, check] + ledger: wiki/project/state.md + assistant: autofit_assistant + witness_file: results/**/*.json + partition: ral + status: active + note: "born 2026-09-10 as the graphical-ep campaign's confidence test on the PyAutoFit#1405 collapse toy at N=50: joint BlackJAX NUTS vs EP x3 on identical data; model and EP settings verbatim from complete/2026/07 ep_toy_diagnostic; CPU-only, data simulated in memory from its seed; remote none until the human runs gh repo create PyAutoLabs/ep_toy_gaussian --private" + # --------------------------------------------------------------------------- # # dormant โ€” git repos only. The non-git trees (subhalo, euclid_group, z_*) are # storage, not projects, and aris_PJ011646 is an assistant clone; none get a diff --git a/tasks/ep_toy_gaussian/nuts_vs_ep_x3_n50.md b/tasks/ep_toy_gaussian/nuts_vs_ep_x3_n50.md new file mode 100644 index 0000000..136c5b2 --- /dev/null +++ b/tasks/ep_toy_gaussian/nuts_vs_ep_x3_n50.md @@ -0,0 +1,109 @@ +# Ep_toy_gaussian โ€” do three EP fits agree with NUTS on the #1405 collapse toy at N=50 + +Project: ep_toy_gaussian +Summary: Do three EP fits agree with NUTS at N=50 +State: submitted +Gates: +Witness: over one 50-dataset draw (seed 42), NUTS is healthy and every one of 3 EP repeats on identical data recovers the NUTS parent mean and sigma within the pre-registered thresholds in `## Witness`, committed as `results/n50_seed42/{summary.json, comparison.md}` +Budget: 24:00 +Runs: 342639, 342640 +Ruling: +Review-minutes: 20 +Epic: graphical-ep +Filed: 2026-09-10 + +## Question + +The `graphical-ep` campaign has three independent lines of evidence that graphical and EP are +sound after the D1-D6 wave: the cosmology run (`slope_hierarchy_scale`, N=25), IC50 (33/33 at +N=5) and the analytic Gaussian ensemble (200 seeds, ruling R-20260910-04). What none of them is, +is the toy on which the hierarchical-parameter collapse was *first* found. This task is the last +confidence test before the campaign scales up: run **that** model, at 50 datasets, with a joint +**JAX + NUTS** reference, and fit it with EP **three times on identical data**. Do all three EP +fits agree with each other and with NUTS? + +### What the earlier "collapse" runs actually were + +The "the toy collapsed" memory is a composite of four different runs, none of which is the run +this task performs. Recorded here so nobody re-derives it: + +| run | what it was | collapse seen | +|---|---|---| +| `PyAutoMind/complete/2026/07/ep_scale_collapse_assets/ep_toy_diagnostic.py` + `EP_TOY_FINDINGS.md` (2026-07-21, PyAutoFit `f83f2f493`) | **The original.** HowToFit `gaussian_x1__hierarchical` toy: N=5 Gaussians, `centre_i ~ N(50, 10)`, `normalization=0.5`, `sigma=5` fixed, `centre` prior `TruncatedGaussian(50, 20, 0, 100)`, `HierarchicalFactor(GaussianPrior, mean=TG(50,10,0,100), sigma=TG(10,5,0,100))`, per-factor `DynestyStatic(nlive=100, sample="rwalk")`, `LaplaceOptimiser`, `EPHistory(kl_tol=0.05)`, `max_steps=20`. Joint reference = **Dynesty** on `global_prior_model`, not NUTS. | 30 identical runs: 70% RECOVER / 7% COLLAPSE (`scatter=0.0030 +/- 0.0000`, mean -> 54.4) / 23% CRASH. The joint fit was stable every time. | +| `PyAutoMind/complete/2026/09/ep-scale-collapse-leg2-assets/{toy.py,run_once.py,classify.py,results_baseline.txt}` | Self-contained rebuild of the same model (data regenerated in memory; `TOY_N`, `TOY_SEED`, `TOY_MAX_STEPS`, `TOY_OPT`, `TOY_UPDATER`), noise `TOY_NOISE=0.05`. | 20 seeds at N=5: 13 COLLAPSE / 7 RECOVER (`scatter=0.0000 err=1e-11`) โ€” harsher than the original. Three-state classifier PATHOLOGICAL / BIASED-TIGHT / RECOVER. | +| `autofit_workspace_test/scripts/graphical/analytic_gaussian_collapse.py` | The closed-form referee's collapse configuration (N=5, 5 seeds, `max_steps=20`). | pre-fix FAIL 3/5 (STALE `10.00 +/- 3.69`, BIASED-TIGHT `7.78 +/- 0.51`); post #1558/#1560/#1562 RECOVER 5/5. | +| `Science/z_projects_complete/slope_hierarchy` (lensing, N=5) | The only **NUTS-vs-EP** hierarchical comparison to date. | NUTS sigma `0.143 [0.117, 0.185]` vs EP `0.026 +/- 0.00001` โ€” 4x low, errors ~1000x too tight. | +| "50 datasets" | `autofit_workspace/scripts/simulators/simulators_sample.py:29` and `Science/concr/simulators/toy.py` โ€” the **fixed-centre** low-SNR sample, shared-centre EP, no hierarchy. | not a collapse run | + +**Nothing has ever run this toy with a NUTS joint fit, at N=50, or with repeated EP fits on +identical data.** The project is new in exactly those three respects, and keeps the original +model and EP settings verbatim. + +### The fixes that might have cured it + +Everything below landed in PyAutoFit after the 2026-07-21 original run (local source checkout +`PyAutoFit` main `81927cce9`, version string still 2026.8.17.1 โ€” cite the SHA, not the version): +**#1465** (collapse guard), **#1558** (D1 id-0 gradient), **#1560** (D4 message limits), +**#1562** (D2/D3 fd-Hessian + skip-not-write), **#1572 / #1573 / #1574 / #1576** (the review wave, +per-variable stale tracking), **#1578 / #1580** (stale-mask fixed point), plus the 2026-08-05 +`InitializerException` no-abort commit, which is the 23% CRASH mode of the original run. The RAL +PyAuto mirror was verified at PyAutoFit `66f9f8d5d` (contains `08207bad0`, #1580) on 2026-09-09; +re-verify before submitting. + +### What is new here + +1. A **joint NUTS reference** (`af.BlackJAXNUTS`, `use_jax=True`) instead of Dynesty โ€” the first + time this toy has had a gradient-based joint posterior to be judged against. +2. **N=50**, ten times the original's N=5, which is where the campaign wants to scale to. +3. **Three EP repeats on identical data**, differing only in RNG โ€” so EP's run-to-run spread is + measured directly rather than inferred across seeds. A pass on criteria 2-5 is the evidence + that the phase-2 / analytic-Gaussian fixes cured #1405 on its own toy; a failure isolates it + to the Dynesty-per-factor EP path, since the analytic ensemble used Laplace factors. + +## Witness + +Pre-registered before submission and mirrored machine-readably as `WITNESS` in +`scripts/compare.py`. Reference = the NUTS posterior on `n50_seed42`; truth = the parent +`(50, 10)` and the realised sample statistics of the 50 drawn centres. **A missed criterion is a +finding to rule on, never a number to relax so a run can pass**; changing one is its own task, +with the reason in the ruling. + +| # | criterion | threshold | +|---|---|---| +| 1 | NUTS health | 0 divergences; r-hat <= 1.01 and ESS >= 400 on parent mean and sigma; realised sample mean/std inside the NUTS 3 sigma interval | +| 2 | No collapse | every EP repeat: parent sigma >= 1.0 (10% of truth) and the #1465 guard silent โ€” PATHOLOGICAL count 0/3 | +| 3 | EP-NUTS parity | every repeat: \|mean_EP - mean_NUTS\| <= 2 sigma_NUTS(mean) and \|sigma_EP - sigma_NUTS\| <= 2 sigma_NUTS(sigma) | +| 4 | EP self-agreement | max pairwise spread across the 3 repeats <= 1 sigma_NUTS, for parent mean and for sigma | +| 5 | Honest widths | every repeat: sigma_EP / sigma_NUTS within [1/3, 3] on parent mean and sigma โ€” BIASED-TIGHT count 0/3 | +| 6 | Per-dataset centres | >= 95% of the 50 centres within 3 sigma_NUTS of the NUTS median, in every repeat | +| 7 | EP flag health | no factor ends any repeat with zero SUCCESS updates; no CRASH; no NaN `RuntimeWarning` | + +Deliverable: `results/n50_seed42/summary.json` (the witness file) + `comparison.md`, committed in +`ep_toy_gaussian`, carrying every criterion's measured number. + +## Where to look + +- `ep_toy_gaussian` (this project's row): `/mnt/c/Users/Jammy/Science/ep_toy_gaussian` โ€” + `scripts/{toy.py, one_by_one.py, graphical_nuts.py, ep_repeat.py, compare.py}`, + `hpc/batch_cpu/{submit_nuts, submit_ep}`, `wiki/project/state.md` +- `results/n50_seed42/` โ€” `nuts.json`, `one_by_one.json`, `ep_r{0,1,2}.json`, and the two + deliverables `summary.json` + `comparison.md`. The local smoke sample is `n5_smoke` +- `PyAutoMind/complete/2026/07/ep_scale_collapse_assets/` โ€” the original run: `ep_toy_diagnostic.py` + and `EP_TOY_FINDINGS.md`, the source of the model and the EP settings used verbatim here +- `PyAutoMind/complete/2026/09/ep-scale-collapse-leg2-assets/` โ€” `toy.py` (borrowed for + `scripts/toy.py`), `run_once.py`, `classify.py` (the three-state labels), `results_baseline.txt` +- `PyAutoMind/draft/research/graphical_ep/ep_campaign.md` โ€” the campaign ledger; this task is + phase 1's confidence test +- Cortex `tasks/analytic_gaussian/ensemble_parity.md` (ruling R-20260910-04) โ€” the 200-seed + closed-form ensemble whose fixes this task tests on the original toy +- `Science/z_projects_complete/slope_hierarchy` โ€” the only prior NUTS-vs-EP hierarchical comparison +- PyAutoFit#1405 โ€” the collapse basin, still open pending the moment-matching cure + +## Runs + +- 342639: submitted โ€” ral โ€” submitted 2026-09-10 โ€” wall 0:00 โ€” one_by_one + BlackJAX NUTS, partition ral +- 342640: submitted โ€” ral โ€” submitted 2026-09-10 โ€” wall 0:00 โ€” EP x3 on identical data, array 0-2 + +## Ruling + +(none)
ProjectNext taskAwaitingRunningLast ruling
analytic_gaussianIs criterion 2's mu threshold or the minimal-EP control wrong002026-09-10
ep_toy_gaussianDo three EP fits agree with NUTS at N=5001-
euclid_dr1_prelimDo ordered Nautilus reruns reproduce the euclid catalogue values?102026-09-10
ic50_workspaceDoes EP match the graphical joint fit end to end01-
slope_hierarchy_scaleDoes hierarchical slope recovery hold at N=25 to 5001-