From 9134910c17e4952e2857c61a66d1f6a4462d7b30 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Wed, 9 Sep 2026 18:11:32 -0400 Subject: [PATCH] =?UTF-8?q?cortex:=20birth=20analytic=5Fgaussian=20?= =?UTF-8?q?=E2=80=94=20the=20EP=20campaign's=20keystone=20at=20ensemble=20?= =?UTF-8?q?scale?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `graphical-ep` campaign's end goal 1 asks for a model that *statistically* demonstrates the graphical and EP source code is correct — means AND errors against a closed form. Phase 1 built that model (autofit_workspace_test#91, PR #92) and ran it at ONE seed as a CI gate; it found six PyAutoFit defects (D1–D6, all shipped) but cannot answer a question about a distribution: its 41-cell count moved 37/41 -> 38/41 between two runs of identical code. Phases 3 and 4 became science projects that run on RAL and are scored against pre-registered witnesses (`slope_hierarchy_scale`, `ic50_workspace`). The keystone that referees both was the arm still living as a parked CI script. This row gives it the same standing. - `projects.yaml`: `analytic_gaussian`, active, partition `ral`, assistant `autofit_assistant`, `remote: none` until a human runs `gh repo create PyAutoLabs/analytic_gaussian --private`. CPU-only and with no `dataset/`: every draw is simulated in memory from its seed, so a seed IS the dataset and nothing can drift between laptop and cluster. - `tasks/analytic_gaussian/ensemble_parity.md`: State `submitted`, runs 342413_[0-199] — a 200-seed ensemble at N=5 on the `ral` partition. The `Witness:` was pre-registered BEFORE submission and is mirrored machine-readably as `WITNESS` in the project's `aggregate_ensemble.py`; criterion 4 (leg B EP) is expected to miss, which is the point — the documented Laplace-on-scatter caveat becomes a measured rate with a Wilson interval, and the moment-matching cure gets a number to beat. The RAL PyAuto mirror was verified at PyAutoFit `66f9f8d5d` before submitting, which contains `08207bad0` (#1580) — phase 3's EP arm ran on a mirror that predated the fix it was meant to exercise, and that is not repeated here. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01GkvYQqRKrZU4K9siYCSd8M --- dashboard.html | 9 ++ dashboard.md | 19 ++++ projects.yaml | 14 +++ tasks/analytic_gaussian/ensemble_parity.md | 104 +++++++++++++++++++++ 4 files changed, 146 insertions(+) create mode 100644 tasks/analytic_gaussian/ensemble_parity.md diff --git a/dashboard.html b/dashboard.html index 5de1717..d8372b2 100644 --- a/dashboard.html +++ b/dashboard.html @@ -236,6 +236,7 @@

One row per active project — the next thing, what it is holding, and when it last ruled.

+ @@ -257,6 +258,14 @@

Nothing open

Projects markdown version

+

analytic_gaussian

+

Local /mnt/c/Users/Jammy/Science/analytic_gaussian

+

RAL /mnt/ral/jnightin/analytic_gaussian

+

active · ral partition · tasks: submitted 1

+

Do graphical and EP recover closed-form means and errorssubmitted — runs 342413_[0-199] · wall 0:00 of 2:00 (0%)

+

tasks/analytic_gaussian/

+
+

euclid_dr1_prelim

Local /mnt/c/Users/Jammy/Science/euclid_dr1_prelim

RAL /mnt/ral/jnightin/euclid_dr1_prelim

diff --git a/dashboard.md b/dashboard.md index 29fe44e..1371680 100644 --- a/dashboard.md +++ b/dashboard.md @@ -29,6 +29,7 @@ One row per active project — the next thing, what it is holding, and when it l | Project | Next task | Awaiting | Running | Last ruling | |---|---|---:|---:|---| +| analytic_gaussian | Do graphical and EP recover closed-form means and errors | 0 | 1 | - | | euclid_dr1_prelim | Is the ordered run's 8-nat gain ordering or library drift | 0 | 4 | - | | ic50_workspace | Does EP match the graphical joint fit end to end | 0 | 1 | - | | slope_hierarchy_scale | Does hierarchical slope recovery hold at N=25 to 50 | 0 | 1 | - | @@ -104,6 +105,24 @@ One row per active project — the next thing, what it is holding, and when it l [markdown version](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects.yaml) +### analytic_gaussian + +**Local** `/mnt/c/Users/Jammy/Science/analytic_gaussian` + +**RAL** `/mnt/ral/jnightin/analytic_gaussian` + +active · ral partition · tasks: submitted 1 + +
📋 Do graphical and EP recover closed-form means and errors — submitted — runs 342413_[0-199] · wall 0:00 of 2:00 (0%) + +``` +cd /mnt/c/Users/Jammy/Science/analytic_gaussian && hpc/sync jobs +``` + +
+ +[tasks/analytic_gaussian/](https://github.com/PyAutoLabs/PyAutoCortex/tree/main/tasks/analytic_gaussian/) + ### euclid_dr1_prelim **Local** `/mnt/c/Users/Jammy/Science/euclid_dr1_prelim` diff --git a/projects.yaml b/projects.yaml index 345a195..59a7c59 100644 --- a/projects.yaml +++ b/projects.yaml @@ -101,6 +101,20 @@ euclid_dr1_prelim: status: active note: "created by euclid-dr1-prep phase 4 (a Cortex task after migration); every path is the planned one; active 2026-09-07: phase 4 launched (jobs 342301_[0-9])" +analytic_gaussian: + remote: none + local_path: /mnt/c/Users/Jammy/Science/analytic_gaussian + ral_root: /mnt/ral/jnightin/analytic_gaussian + mirror: none + sync_cli: hpc/sync + sync_verbs: [push, pull, logs, sync, status, submit, push-submit, jobs, sacct, cancel, tail, du, check] + ledger: wiki/project/state.md + assistant: autofit_assistant + witness_file: results/**/*.json + partition: ral + status: active + note: "born 2026-09-09 as the graphical-ep campaign's phase-1 keystone at ensemble scale; benchmark model with a closed-form posterior, borrowed from autofit_workspace_test scripts/graphical (PR #92 lineage); CPU-only, no dataset (every draw is simulated in memory from its seed); remote none until the human runs gh repo create PyAutoLabs/analytic_gaussian --private" + # --------------------------------------------------------------------------- # # dormant — git repos only. The non-git trees (subhalo, euclid_group, z_*) are # storage, not projects, and aris_PJ011646 is an assistant clone; none get a diff --git a/tasks/analytic_gaussian/ensemble_parity.md b/tasks/analytic_gaussian/ensemble_parity.md new file mode 100644 index 0000000..ae6426b --- /dev/null +++ b/tasks/analytic_gaussian/ensemble_parity.md @@ -0,0 +1,104 @@ +# analytic_gaussian — task 1: does autofit recover the closed form over an ensemble of draws + +Project: analytic_gaussian +Summary: Do graphical and EP recover closed-form means and errors +State: submitted +Gates: +Witness: over >= 200 independent dataset draws at N=5, the a/b error distributions and the empirical [q05, q95] coverage per column meet the pre-registered thresholds in the `## Witness` section below, committed as `results/ens_n5/{summary.json, parity_ensemble.md}` +Budget: 2:00 +Runs: 342413 +Ruling: +Review-minutes: 20 +Epic: graphical-ep +Filed: 2026-09-09 +Migrated-from: PyAutoMind/complete/2026/09/analytic-gaussian-benchmark.md + +## Question + +The `graphical-ep` campaign's end goal 1 is *"an analytic Gaussian model that statistically +demonstrates the graphical and EP source code is correct (means **and** errors against closed +form)"*. Phase 1 built the model and answered it at **one seed**, as a CI gate in +`autofit_workspace_test/scripts/graphical/`. That form did its job — it found six PyAutoFit +defects, D1–D6, all now shipped (#1558, #1560, #1562, #1572, #1573, #1574, #1576, #1578, #1580) +— but a single seed cannot demonstrate anything statistically: its 41-cell count moved 37/41 → +38/41 between two runs of identical code, because a verdict on a tolerance edge is a coin flip. + +So: over many independent dataset draws of the same conjugate hierarchical Gaussian, does each +column recover the closed-form posterior — mean *and* error — and how often? + + mu ~ N(50, 10^2), x_i | mu ~ N(mu, sigma^2), y_ij | x_i ~ N(x_i, s_i^2) + + leg A sigma = 10 fixed the graph is exactly Gaussian: + Laplace EP should be EXACT here + leg B sigma ~ TruncatedGaussian(10, 5, 0, 100) the phase-2 scatter-collapse configuration + + columns closed form | minimal EP | autofit graphical (joint DynestyStatic) | autofit EP + +Three specific things this answers that the campaign has been carrying as assumptions: + +- **Is leg A fixed?** The parked CI script failed leg A: `mu` never left its starting prior. The + first local run on the post-fix library returns it exact (a = b = 0.000). Over 200 seeds, is + that exactness the rule or the lucky draw? +- **How bad is leg B, really?** The Laplace-on-scatter caveat (`autofit/graphical/README.md` + §3.5, PyAutoFit#1405) is documented but never quantified: the first local seed returns + sigma 9.37 ± 3.57 against a reference 6.57 ± 2.88 — outside the cell tolerance, inside the + closed-form [q05, q95], not collapsed. A rate with an interval turns that caveat into a + measurement, and gives the moment-matching cure + (`PyAutoMind draft/feature/autofit/ep_hierarchical_scatter_moment_matching.md`) a number to + beat. +- **Is the collapse basin still reachable?** The scale-collapse rate (sigma < 0.1) over 200 + independent draws is the campaign's first unbiased estimate of how often PyAutoFit#1405's + basin is entered. + +The **minimal EP** column is the analytic ceiling, not a competitor: EP with a Gaussian site on +sigma is biased on this model by construction, so it — not zero — is what an autofit fix should +be judged against. + +## Witness + +Pre-registered before wave 1 was submitted, and mirrored machine-readably as `WITNESS` in +`scripts/aggregate_ensemble.py`. A missed criterion is a finding to rule on, never a number to +relax so a wave can pass; changing one is its own task, with the reason in the ruling. + +| # | criterion | threshold | +|---|---|---| +| 1 | minimal EP, leg A | max a, b ≤ 1e-6 on every row of every seed — a deterministic identity | +| 2 | minimal EP, leg B (the analytic ceiling) | ≥ 95% of seeds within (scatter a 0.20 / b 0.30, other rows a 0.05 / b 0.16); E[sigma] coverage ≥ 0.90 | +| 3 | autofit EP, leg A | ≥ 95% of seeds within (a 0.01, b 0.02) on **every** row, and no seed worse than a = 0.10 | +| 4 | autofit EP, leg B | ≥ 90% of seeds within (a 0.15, b 0.25); E[sigma] inside the closed-form [q05, q95] for ≥ 90%; scale-collapse rate (sigma < 0.1) = 0 | +| 5 | autofit graphical | ≥ 90% of seeds within (a 0.10, b 0.15) on both legs; scatter coverage ≥ 0.85 | +| 6 | EP flag health | no seed in which any factor ends the run with zero SUCCESS updates — the STALE state #1562/#1574/#1576/#1580 taught the library to report | + +Deliverable: `results/ens_n5/summary.json` (the witness file) and `parity_ensemble.md` committed +in `analytic_gaussian`, carrying every criterion's measured number — including a Wilson interval +on each coverage rate, so "3/3 covered" cannot be read as certainty. + +Criterion 4 is **expected to miss**, on the campaign's own evidence. That is the point: the miss +becomes a measured rate rather than a documented caveat, and the ruling records it as the cure's +baseline. + +## Where to look + +- `analytic_gaussian` (this project's row): `scripts/ensemble_seed.py` (one seed per array task), + `scripts/aggregate_ensemble.py` (the referee), `wiki/project/state.md` +- `autofit_workspace_test` @ `165a19e` — `scripts/graphical/analytic_{reference,ep_minimal,autofit}.py`, + the modules this project borrows verbatim, and `analytic_gaussian.py`, the single-seed CI form + parked NEEDS_FIX in `config/build/no_run.yaml` +- Mind `draft/research/graphical_ep/ep_campaign.md` — phase 1 and end goal 1 +- PyAutoFit#1405 — the collapse basin, still open pending the moment-matching cure +- Mind `draft/bug/graphical_ep/analytic_gaussian_unseeded_graphical_column.md` — the CI script's + unseeded graphical column; this project seeds dynesty's `rstate` from the seed instead, and + records `graphical_seeded` in every sidecar +- `hpc/batch_cpu/submit_ensemble_n25` — the N=25 rung, written but **not submitted**: its cost is + unmeasured (27 free parameters in the joint fit against 7 at N=5), and the local timing run was + still going when wave 1 went out. Submit it once one seed has given a per-seed wall time; it + runs the expensive graphical column on seeds 0-19 only + +## Runs + +- 342413_[0-199]: submitted — ral — submitted 2026-09-09 — wall 0:00 — the N=5 seed ensemble, 200 seeds, 50 concurrent, `hpc/batch_cpu/submit_ensemble_n5`, sample `ens_n5`. RAL PyAuto mirror verified at PyAutoFit `66f9f8d5d` before submission, which **contains** `08207bad0` (#1580) — unlike phase 3's EP arm, this wave carries the full D1–D6 wave plus the review bundle + + +## Ruling + +(none)
ProjectNext taskAwaitingRunningLast ruling
analytic_gaussianDo graphical and EP recover closed-form means and errors01-
euclid_dr1_prelimIs the ordered run's 8-nat gain ordering or library drift04-
ic50_workspaceDoes EP match the graphical joint fit end to end01-
slope_hierarchy_scaleDoes hierarchical slope recovery hold at N=25 to 5001-