Skip to content

cortex: birth analytic_gaussian — the EP campaign's keystone at ensemble scale - #27

Merged
Jammy2211 merged 2 commits into
mainfrom
claude/birth-analytic-gaussian
Sep 10, 2026
Merged

Jammy2211 merged 2 commits into
mainfrom
claude/birth-analytic-gaussian

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Why

The graphical-ep campaign's end goal 1 asks for a model that statistically demonstrates the graphical and EP source code is correct — means and errors against a closed form. Phase 1 built that model (autofit_workspace_test#91 / PR #92) and ran it at one seed as a CI gate. That form earned its keep: it found six PyAutoFit defects (D1–D6, all shipped: #1558, #1560, #1562, #1572, #1573, #1574, #1576, #1578, #1580). But one seed cannot answer a question about a distribution — its 41-cell count moved 37/41 → 38/41 between two runs of identical code, because a verdict on a tolerance edge is a coin flip.

Phases 3 and 4 became science projects that run on RAL and are scored against pre-registered witnesses (slope_hierarchy_scale, ic50_workspace). The keystone that referees both was the one arm still living as a parked CI script. This gives it the same standing.

What

  • projects.yaml — new active row analytic_gaussian: partition ral, assistant autofit_assistant, remote: none until a human runs gh repo create PyAutoLabs/analytic_gaussian --private. CPU-only, and with no dataset/: every draw is simulate(seed, n_datasets) in memory, so a seed is the dataset and nothing can drift between laptop and cluster.
  • tasks/analytic_gaussian/ensemble_parity.md — State submitted, Runs: 342413. A 200-seed ensemble at N=5, live on the ral partition (50 concurrent).
  • Dashboard regenerated in the same commit.

The witness was pre-registered before the wave went out

Six criteria, mirrored machine-readably as WITNESS in the project's aggregate_ensemble.py, covering the a/b error distributions per column, the empirical coverage of the closed form's [q05, q95] on the scatter (with a Wilson interval, so "3/3 covered" cannot be read as certainty), the scale-collapse rate, and EP flag health.

Criterion 4 (autofit EP, leg B) is expected to miss — that is the point. The documented Laplace-on-scatter caveat (PyAutoFit#1405, autofit/graphical/README.md §3.5) becomes a measured rate, and the moment-matching cure gets a number to beat.

Evidence already in hand

First local run on the post-D1–D6 library (seed 0, N=5, autofit 2026.8.17.1):

  • leg A autofit EP is now exact — mu 50.8595 ± 4.1104 against a reference of 50.8595 ± 4.1104 (a = b = 0.000), and the same on every x_i. This is the leg that parked analytic_gaussian.py NEEDS_FIX, where mu used to sit at its starting prior 50.00 ± 20.6.
  • leg B autofit EP returns σ 9.3722 ± 3.5664 against 6.5667 ± 2.8832: outside the cell tolerance, inside [q05, q95] = [2.980, 12.154], not collapsed.
  • Over three local seeds the referee already reads: leg A EP 3/3, leg B EP 2/3, minimal EP exact to 1.16e-14.

The RAL PyAuto mirror was verified at PyAutoFit 66f9f8d5d (contains 08207bad0, #1580) before submitting — phase 3's EP arm ran on a mirror that predated the fix it was meant to exercise, and that is not repeated here.

Human call

projects.yaml is code, so this needs a human merge. The science tree is born locally at /mnt/c/Users/Jammy/Science/analytic_gaussian (commit ea3853f) and is unpushed until gh repo create PyAutoLabs/analytic_gaussian --private.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GkvYQqRKrZU4K9siYCSd8M

Jammy2211 and others added 2 commits September 9, 2026 18:11
…ble scale

The `graphical-ep` campaign's end goal 1 asks for a model that *statistically*
demonstrates the graphical and EP source code is correct — means AND errors
against a closed form. Phase 1 built that model (autofit_workspace_test#91,
PR #92) and ran it at ONE seed as a CI gate; it found six PyAutoFit defects
(D1–D6, all shipped) but cannot answer a question about a distribution: its
41-cell count moved 37/41 -> 38/41 between two runs of identical code.

Phases 3 and 4 became science projects that run on RAL and are scored against
pre-registered witnesses (`slope_hierarchy_scale`, `ic50_workspace`). The
keystone that referees both was the arm still living as a parked CI script.
This row gives it the same standing.

- `projects.yaml`: `analytic_gaussian`, active, partition `ral`, assistant
  `autofit_assistant`, `remote: none` until a human runs
  `gh repo create PyAutoLabs/analytic_gaussian --private`. CPU-only and with no
  `dataset/`: every draw is simulated in memory from its seed, so a seed IS the
  dataset and nothing can drift between laptop and cluster.
- `tasks/analytic_gaussian/ensemble_parity.md`: State `submitted`, runs
  342413_[0-199] — a 200-seed ensemble at N=5 on the `ral` partition. The
  `Witness:` was pre-registered BEFORE submission and is mirrored machine-readably
  as `WITNESS` in the project's `aggregate_ensemble.py`; criterion 4 (leg B EP) is
  expected to miss, which is the point — the documented Laplace-on-scatter caveat
  becomes a measured rate with a Wilson interval, and the moment-matching cure
  gets a number to beat.

The RAL PyAuto mirror was verified at PyAutoFit `66f9f8d5d` before submitting,
which contains `08207bad0` (#1580) — phase 3's EP arm ran on a mirror that
predated the fix it was meant to exercise, and that is not repeated here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GkvYQqRKrZU4K9siYCSd8M
…-gaussian

# Conflicts:
#	dashboard.html
#	dashboard.md
@Jammy2211
Jammy2211 merged commit 474a59c into main Sep 10, 2026
3 checks passed
@Jammy2211
Jammy2211 deleted the claude/birth-analytic-gaussian branch September 10, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant