diff --git a/dashboard.html b/dashboard.html index c1cc297..4d0dc40 100644 --- a/dashboard.html +++ b/dashboard.html @@ -236,6 +236,7 @@

One row per active project โ€” the next thing, what it is holding, and when it last ruled.

+ @@ -257,6 +258,14 @@

Nothing open

Projects markdown version

+

analytic_gaussian

+

Local /mnt/c/Users/Jammy/Science/analytic_gaussian

+

RAL /mnt/ral/jnightin/analytic_gaussian

+

active ยท ral partition ยท tasks: submitted 1

+

Do graphical and EP recover closed-form means and errorssubmitted โ€” runs 342413_[0-199] ยท wall 0:00 of 2:00 (0%)

+

tasks/analytic_gaussian/

+
+

euclid_dr1_prelim

Local /mnt/c/Users/Jammy/Science/euclid_dr1_prelim

RAL /mnt/ral/jnightin/euclid_dr1_prelim

diff --git a/dashboard.md b/dashboard.md index d69c533..a1dcd59 100644 --- a/dashboard.md +++ b/dashboard.md @@ -29,6 +29,7 @@ One row per active project โ€” the next thing, what it is holding, and when it l | Project | Next task | Awaiting | Running | Last ruling | |---|---|---:|---:|---| +| analytic_gaussian | Do graphical and EP recover closed-form means and errors | 0 | 1 | - | | euclid_dr1_prelim | Do ordered Nautilus reruns reproduce the euclid catalogue values? | 0 | 1 | 2026-09-10 | | ic50_workspace | Does EP match the graphical joint fit end to end | 0 | 1 | - | | slope_hierarchy_scale | Does hierarchical slope recovery hold at N=25 to 50 | 0 | 1 | - | @@ -104,6 +105,24 @@ One row per active project โ€” the next thing, what it is holding, and when it l [markdown version](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects.yaml) +### analytic_gaussian + +**Local** `/mnt/c/Users/Jammy/Science/analytic_gaussian` + +**RAL** `/mnt/ral/jnightin/analytic_gaussian` + +active ยท ral partition ยท tasks: submitted 1 + +
๐Ÿ“‹ Do graphical and EP recover closed-form means and errors โ€” submitted โ€” runs 342413_[0-199] ยท wall 0:00 of 2:00 (0%) + +``` +cd /mnt/c/Users/Jammy/Science/analytic_gaussian && hpc/sync jobs +``` + +
+ +[tasks/analytic_gaussian/](https://github.com/PyAutoLabs/PyAutoCortex/tree/main/tasks/analytic_gaussian/) + ### euclid_dr1_prelim **Local** `/mnt/c/Users/Jammy/Science/euclid_dr1_prelim` diff --git a/projects.yaml b/projects.yaml index 345a195..59a7c59 100644 --- a/projects.yaml +++ b/projects.yaml @@ -101,6 +101,20 @@ euclid_dr1_prelim: status: active note: "created by euclid-dr1-prep phase 4 (a Cortex task after migration); every path is the planned one; active 2026-09-07: phase 4 launched (jobs 342301_[0-9])" +analytic_gaussian: + remote: none + local_path: /mnt/c/Users/Jammy/Science/analytic_gaussian + ral_root: /mnt/ral/jnightin/analytic_gaussian + mirror: none + sync_cli: hpc/sync + sync_verbs: [push, pull, logs, sync, status, submit, push-submit, jobs, sacct, cancel, tail, du, check] + ledger: wiki/project/state.md + assistant: autofit_assistant + witness_file: results/**/*.json + partition: ral + status: active + note: "born 2026-09-09 as the graphical-ep campaign's phase-1 keystone at ensemble scale; benchmark model with a closed-form posterior, borrowed from autofit_workspace_test scripts/graphical (PR #92 lineage); CPU-only, no dataset (every draw is simulated in memory from its seed); remote none until the human runs gh repo create PyAutoLabs/analytic_gaussian --private" + # --------------------------------------------------------------------------- # # dormant โ€” git repos only. The non-git trees (subhalo, euclid_group, z_*) are # storage, not projects, and aris_PJ011646 is an assistant clone; none get a diff --git a/tasks/analytic_gaussian/ensemble_parity.md b/tasks/analytic_gaussian/ensemble_parity.md new file mode 100644 index 0000000..ae6426b --- /dev/null +++ b/tasks/analytic_gaussian/ensemble_parity.md @@ -0,0 +1,104 @@ +# analytic_gaussian โ€” task 1: does autofit recover the closed form over an ensemble of draws + +Project: analytic_gaussian +Summary: Do graphical and EP recover closed-form means and errors +State: submitted +Gates: +Witness: over >= 200 independent dataset draws at N=5, the a/b error distributions and the empirical [q05, q95] coverage per column meet the pre-registered thresholds in the `## Witness` section below, committed as `results/ens_n5/{summary.json, parity_ensemble.md}` +Budget: 2:00 +Runs: 342413 +Ruling: +Review-minutes: 20 +Epic: graphical-ep +Filed: 2026-09-09 +Migrated-from: PyAutoMind/complete/2026/09/analytic-gaussian-benchmark.md + +## Question + +The `graphical-ep` campaign's end goal 1 is *"an analytic Gaussian model that statistically +demonstrates the graphical and EP source code is correct (means **and** errors against closed +form)"*. Phase 1 built the model and answered it at **one seed**, as a CI gate in +`autofit_workspace_test/scripts/graphical/`. That form did its job โ€” it found six PyAutoFit +defects, D1โ€“D6, all now shipped (#1558, #1560, #1562, #1572, #1573, #1574, #1576, #1578, #1580) +โ€” but a single seed cannot demonstrate anything statistically: its 41-cell count moved 37/41 โ†’ +38/41 between two runs of identical code, because a verdict on a tolerance edge is a coin flip. + +So: over many independent dataset draws of the same conjugate hierarchical Gaussian, does each +column recover the closed-form posterior โ€” mean *and* error โ€” and how often? + + mu ~ N(50, 10^2), x_i | mu ~ N(mu, sigma^2), y_ij | x_i ~ N(x_i, s_i^2) + + leg A sigma = 10 fixed the graph is exactly Gaussian: + Laplace EP should be EXACT here + leg B sigma ~ TruncatedGaussian(10, 5, 0, 100) the phase-2 scatter-collapse configuration + + columns closed form | minimal EP | autofit graphical (joint DynestyStatic) | autofit EP + +Three specific things this answers that the campaign has been carrying as assumptions: + +- **Is leg A fixed?** The parked CI script failed leg A: `mu` never left its starting prior. The + first local run on the post-fix library returns it exact (a = b = 0.000). Over 200 seeds, is + that exactness the rule or the lucky draw? +- **How bad is leg B, really?** The Laplace-on-scatter caveat (`autofit/graphical/README.md` + ยง3.5, PyAutoFit#1405) is documented but never quantified: the first local seed returns + sigma 9.37 ยฑ 3.57 against a reference 6.57 ยฑ 2.88 โ€” outside the cell tolerance, inside the + closed-form [q05, q95], not collapsed. A rate with an interval turns that caveat into a + measurement, and gives the moment-matching cure + (`PyAutoMind draft/feature/autofit/ep_hierarchical_scatter_moment_matching.md`) a number to + beat. +- **Is the collapse basin still reachable?** The scale-collapse rate (sigma < 0.1) over 200 + independent draws is the campaign's first unbiased estimate of how often PyAutoFit#1405's + basin is entered. + +The **minimal EP** column is the analytic ceiling, not a competitor: EP with a Gaussian site on +sigma is biased on this model by construction, so it โ€” not zero โ€” is what an autofit fix should +be judged against. + +## Witness + +Pre-registered before wave 1 was submitted, and mirrored machine-readably as `WITNESS` in +`scripts/aggregate_ensemble.py`. A missed criterion is a finding to rule on, never a number to +relax so a wave can pass; changing one is its own task, with the reason in the ruling. + +| # | criterion | threshold | +|---|---|---| +| 1 | minimal EP, leg A | max a, b โ‰ค 1e-6 on every row of every seed โ€” a deterministic identity | +| 2 | minimal EP, leg B (the analytic ceiling) | โ‰ฅ 95% of seeds within (scatter a 0.20 / b 0.30, other rows a 0.05 / b 0.16); E[sigma] coverage โ‰ฅ 0.90 | +| 3 | autofit EP, leg A | โ‰ฅ 95% of seeds within (a 0.01, b 0.02) on **every** row, and no seed worse than a = 0.10 | +| 4 | autofit EP, leg B | โ‰ฅ 90% of seeds within (a 0.15, b 0.25); E[sigma] inside the closed-form [q05, q95] for โ‰ฅ 90%; scale-collapse rate (sigma < 0.1) = 0 | +| 5 | autofit graphical | โ‰ฅ 90% of seeds within (a 0.10, b 0.15) on both legs; scatter coverage โ‰ฅ 0.85 | +| 6 | EP flag health | no seed in which any factor ends the run with zero SUCCESS updates โ€” the STALE state #1562/#1574/#1576/#1580 taught the library to report | + +Deliverable: `results/ens_n5/summary.json` (the witness file) and `parity_ensemble.md` committed +in `analytic_gaussian`, carrying every criterion's measured number โ€” including a Wilson interval +on each coverage rate, so "3/3 covered" cannot be read as certainty. + +Criterion 4 is **expected to miss**, on the campaign's own evidence. That is the point: the miss +becomes a measured rate rather than a documented caveat, and the ruling records it as the cure's +baseline. + +## Where to look + +- `analytic_gaussian` (this project's row): `scripts/ensemble_seed.py` (one seed per array task), + `scripts/aggregate_ensemble.py` (the referee), `wiki/project/state.md` +- `autofit_workspace_test` @ `165a19e` โ€” `scripts/graphical/analytic_{reference,ep_minimal,autofit}.py`, + the modules this project borrows verbatim, and `analytic_gaussian.py`, the single-seed CI form + parked NEEDS_FIX in `config/build/no_run.yaml` +- Mind `draft/research/graphical_ep/ep_campaign.md` โ€” phase 1 and end goal 1 +- PyAutoFit#1405 โ€” the collapse basin, still open pending the moment-matching cure +- Mind `draft/bug/graphical_ep/analytic_gaussian_unseeded_graphical_column.md` โ€” the CI script's + unseeded graphical column; this project seeds dynesty's `rstate` from the seed instead, and + records `graphical_seeded` in every sidecar +- `hpc/batch_cpu/submit_ensemble_n25` โ€” the N=25 rung, written but **not submitted**: its cost is + unmeasured (27 free parameters in the joint fit against 7 at N=5), and the local timing run was + still going when wave 1 went out. Submit it once one seed has given a per-seed wall time; it + runs the expensive graphical column on seeds 0-19 only + +## Runs + +- 342413_[0-199]: submitted โ€” ral โ€” submitted 2026-09-09 โ€” wall 0:00 โ€” the N=5 seed ensemble, 200 seeds, 50 concurrent, `hpc/batch_cpu/submit_ensemble_n5`, sample `ens_n5`. RAL PyAuto mirror verified at PyAutoFit `66f9f8d5d` before submission, which **contains** `08207bad0` (#1580) โ€” unlike phase 3's EP arm, this wave carries the full D1โ€“D6 wave plus the review bundle + + +## Ruling + +(none)
ProjectNext taskAwaitingRunningLast ruling
analytic_gaussianDo graphical and EP recover closed-form means and errors01-
euclid_dr1_prelimDo ordered Nautilus reruns reproduce the euclid catalogue values?012026-09-10
ic50_workspaceDoes EP match the graphical joint fit end to end01-
slope_hierarchy_scaleDoes hierarchical slope recovery hold at N=25 to 5001-