Repository navigation
bug: EP counts zero-iteration Laplace convergence (NO_CHANGE) as a stale/skipped update #1711
Description
Overview
Criterion 6 of the analytic_gaussian wave-1 witness (PyAutoCortex ruling R-20260910-04) missed on 19/400 seed-legs: a factor ended the EP run with zero SUCCESS updates. On the exactly-Gaussian leg A (seed 140) the factor is PriorFactor5 {NO_CHANGE: 2}; on 18 leg-B seeds it is a PriorFactor with only NO_CHANGE, in six of them together with HierarchicalFactor1 with only BAD_PROJECTION. NO_CHANGE is produced only by autofit/graphical/laplace/newton.py (StatusFlag.get_flag(success, i) with i == 0, updated=False), so a factor whose tilted density is already at its mode on first visit is reported exactly like a factor whose line search failed every sweep, and both the library's STALE warning and the ensemble's criterion 6 inherit that conflation. This task diagnoses the two shapes, fixes the reporting, re-checks the BAD_PROJECTION shape under the shipped projection="moments" (#1656), pins leg A seed 140 in autofit_workspace_test and amends criterion 6.
Campaign context: PyAutoInsight tasks/ep_campaign.md (graphical-ep epic); the Insight campaigns.yaml row asked to reconcile shipped fixes and select one bounded phase — this is that phase.
Plan
- Reconcile the Insight ledger (
tasks/ep_campaign.md,campaigns.yaml) with the 2026-10 shipped fixes (fix(dynesty): run serially at number_of_cores=1, build no Pool(1) #1632/fix(ep): release each factor search's internals so JAX executables are collectable #1634, fix(dynesty)+perf(ep): converged search runs once; ndtr in truncated-normal maths (#1642, PR 1 of 3) #1643–perf(mapper): frozen-cached prior tuple and instance plan on the instance_from_vector hot path (#1642, PR 3 of 3) #1645, fix: EP projection raises ProjectionException on non-finite sufficient statistics; factor_step recovers (#1653) #1655, feat: LaplaceOptimiser(projection="moments") — moment-matching projection for the hierarchical scatter (#1654) #1656) and the four Cortex project states; pointgraphical-ep.nextat this issue. - Reproduce seed 140 leg A and leg-B seeds 29/38/71 on current PyAutoFit main with the science scripts, under
projection="mode"(as run) and"moments", dumping per-sweep flags and messages. Repro path is/mnt/c/Users/Jammy/Science/active/analytic_gaussian(the prompt predates the 2026-10-18 category layout). - Diagnose the NO_CHANGE shape: which sweep, is the start point already the mode, does the projected message equal the existing one. Decide legitimate fixed point vs defect.
- Fix the reporting: a zero-iteration successful convergence whose projection is accepted is a converged update, not a skip; it must not feed the stale warning, while
ep_history.csvkeeps the NO_CHANGE flag. Unit test. - Confirm the BAD_PROJECTION-only HierarchicalFactor shape is gone under moments on seeds 29/71; if so record it as cured by feat: LaplaceOptimiser(projection="moments") — moment-matching projection for the hierarchical scatter (#1654) #1656 (no mode-path change). If not, stop and report as new scope.
- Workspace regression pinning leg A seed 140 in
autofit_workspace_test/scripts/graphical/(parity rows incl. widths + flag health, STALE warning absent), in the smoke gate. - Amend criterion 6 in the Cortex task text and hand the human a diff for
ensemble_seed.py(science checkout is outside the workspace). - Ship library first, then workspace; Insight ledger PR closes the loop.
Tier: glance — merge mode: auto-merge on green if the Witness passes, post-merge summary.
Detailed implementation plan
Affected Repositories
- PyAutoFit (primary)
- autofit_workspace_test (regression script)
- PyAutoInsight (ledger), PyAutoCortex (criterion text; temp worktree off origin/main)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./fit/PyAutoFit | main (d3a7721) | clean |
| ./fit/autofit_workspace_test | main (763d79d) | clean |
Suggested branch: feature/ep-zero-success-factors — worktree ~/Code/PyAutoLabs-wt/ep-zero-success-factors/
Implementation Steps
- Repro (worktree PyAutoFit on PYTHONPATH):
cd /mnt/c/Users/Jammy/Science/active/analytic_gaussian && NUMBA_CACHE_DIR=/tmp/numba_cache MPLCONFIGDIR=/tmp/matplotlib python3 scripts/ensemble_seed.py --seed 140 --legs a --columns epand seeds 29/38/71--legs b; write to a scratch output dir soresults/ens_n5/(committed b44390d) is never overwritten. Readep_history.csv/ep_diagnostics.results; capture the"..., iter=0"status messages. Add aprojection="moments"leg (local patch ofanalytic_autofit.pyif the script has no knob; nothing committed to the science repo). - Library:
autofit/graphical/laplace/newton.py~290–380 — a stop condition met ati == 0withsuccess=Trueis convergence: reportupdated=True, keepflag=NO_CHANGE.autofit/graphical/expectation_propagation/optimiser.py_track_failure/_stale_factor_warnings(~380–520) — a NO_CHANGE with an accepted projection counts as updated and its variables as seen; the per-(factor, variable) pass must not list a variable already at the fixed point. Checkmean_field.py:511–620(update_factor_mean_field,check_valid/update_invalid) so an accepted identical projection is not misclassified. Docstrings: optimiser.py NO_CHANGE paragraph,StatusFlaginutils.py. - Unit test in
test_autofit/graphical/(next to the feat(ep): report staleness per (factor, variable) and record reverted variables in ep_history.csv #1576 stale-tracking tests): a small Gaussian graph with one PriorFactor at its mode from the start → reported updated, no stale warning, history shows NO_CHANGE. Red before, green after. - Re-check seeds 29/71 leg B under
projection="moments":HierarchicalFactor1SUCCESS ≥ 1 expected (cured by feat: LaplaceOptimiser(projection="moments") — moment-matching projection for the hierarchical scatter (#1654) #1656). Record the outcome on this issue. - Workspace:
autofit_workspace_test/scripts/graphical/analytic_gaussian_seed140.py(or a--seedleg ofanalytic_gaussian.pyif it stays within the CI cap noted at line 62): leg A, seed 140, moments projection; asserts closed-form parity rows including widths, every factor SUCCESS ≥ 1 or accepted NO_CHANGE, STALE warning absent. Not inconfig/build/no_run.yaml. - Criterion 6: PyAutoCortex
archive/tasks/analytic_gaussian/ensemble_parity.mdrow 6 + newest-first Log note inprojects/analytic_gaussian.md(cortex.py check). Hand the human a diff forensemble_seed.py:160any_factor_never_succeeded. - Insight:
tasks/ep_campaign.mdphase-table 2026-10-23 entries + findings follow-up rows;campaigns.yamlgraphical-ep.next/updated.
Key Files
autofit/graphical/laplace/newton.py— zero-iteration convergence →updatedsemanticsautofit/graphical/expectation_propagation/optimiser.py— stale tracking (_factors_skipped,_stale_factor_warnings)autofit/graphical/mean_field.py— projection acceptanceautofit/graphical/utils.py—StatusFlagautofit_workspace_test/scripts/graphical/analytic_gaussian.py— pattern for the regressionPyAutoInsight/tasks/ep_campaign.md,campaigns.yaml— ledger
Witness
python3 scripts/ensemble_seed.py --seed 140 --legs a --columns ep in analytic_gaussian writes a sidecar whose legs.A.info["ep stale"] is False and every factor has SUCCESS ≥ 1 — or, if the zero-SUCCESS factor is ruled legitimate, criterion 6 is amended with the reason; plus a regression in autofit_workspace_test/scripts/graphical/ pinning the leg-A case.
Lineage
PyAutoFit#1405, #1562, #1574, #1576, #1580, #1656; PyAutoCortex ruling R-20260910-04; Mind prompt draft/bug/autofit/ep_factors_end_the_run_with_zero.md; Insight tasks/ep_campaign.md.
Original Prompt
Click to expand starting prompt
EP factors end the run with zero SUCCESS updates on the exactly-Gaussian leg A of analytic_gaussian (seed 140) and on 18/200 leg-B seeds
Type: bug
Target: PyAutoFit
Repos:
- PyAutoFit
- autofit_workspace_test
Themes: - graphical-ep
Difficulty: medium
Autonomy: supervised
Priority: high
Status: formalised
Consequence: glance
Witness:python3 scripts/ensemble_seed.py --seed 140 --legs a --columns epinanalytic_gaussianwrites a sidecar whoselegs.A.info["ep stale"]is False and in whoselegs.A.info["ep flags"]every factor has SUCCESS >= 1 — or, if a zero-SUCCESS factor is ruled legitimate, criterion 6 is amended with the reason; plus a regression inautofit_workspace_test/scripts/graphical/that pins the leg-A case.
Review-minutes: 3
Unattended: ready
Epic: graphical-ep
Filed: 2026-09-10
Why
Criterion 6 of the pre-registered witness of the science task
PyAutoCortex/tasks/analytic_gaussian/ensemble_parity.md ("no seed in which any factor ends the
run with zero SUCCESS updates") missed: 19 of 400 seed-legs recorded at least one factor that
finished the EP run having never once been updated successfully — the STALE state PyAutoFit
#1562 / #1574 / #1576 / #1580 taught the library to report.
Ruled 2026-09-10 as PyAutoCortex/rulings/2026/09/R-20260910-04.md (accept with findings).
Affected seeds, from the 200-seed N=5 ensemble (RAL job 342413_[0-199], autofit 2026.8.17.1):
- leg B (
sigma ~ TruncatedGaussian(10, 5, 0, 100)): seeds 29, 38, 44, 52, 62, 71, 75, 83,
90, 106, 133, 138, 147, 149, 152, 160, 178, 193 - leg A (
sigma = 10fixed): seed 140
Leg A is the damning one. It is a linear-Gaussian graph — mu ~ N(50, 10^2),
x_i | mu ~ N(mu, 10^2), y_ij | x_i ~ N(x_i, s_i^2) — on which Laplace EP is exact by
construction, and criterion 3 confirms it: autofit EP recovers leg A 200/200 with worst
a = 0.001. A factor that never takes an update on a graph where every projection is available
in closed form is a library defect, not a model limitation.
The sidecars show two distinct shapes:
- a
PriorFactor*that ends with onlyNO_CHANGE(leg A seed 140:PriorFactor5 {NO_CHANGE: 2};
most leg-B seeds, e.g. 38/44/52/62PriorFactor17 {NO_CHANGE: 3}) - a
HierarchicalFactor1that ends with onlyBAD_PROJECTIONand never a SUCCESS
(leg B seeds 29, 71, 106, 152, 178, 193 — e.g.{BAD_PROJECTION: 15},{BAD_PROJECTION: 20})
Whether those are one defect or two is part of the question.
Repro
Science project analytic_gaussian at /mnt/c/Users/Jammy/Science/analytic_gaussian
(git main, no remote), results committed at b44390d:
cd /mnt/c/Users/Jammy/Science/analytic_gaussian
python3 scripts/ensemble_seed.py --seed 140 --legs a --columns ep
and read the EP flag fields of the sidecar it writes — in
results/ens_n5/seed_0140.json they are legs.A.info["ep flags"]
(a {factor_name: {FLAG: count}} map) and the derived boolean legs.A.info["ep stale"].
The witness scripts are borrowed verbatim from
autofit_workspace_test/scripts/graphical/analytic_{reference,ep_minimal,autofit}.py @ 165a19e.
Ask
- Which factor is it, on which update, and why does it never succeed — is the projection
rejected, is the update a no-op because the message is already the cavity, or is the factor
never visited? - Are the
NO_CHANGE-onlyPriorFactorcase and theBAD_PROJECTION-onlyHierarchicalFactor
case the same defect or two? - Fix it, or explain why a zero-SUCCESS factor is legitimate on an exactly-Gaussian graph — and
if it is legitimate, criterion 6 is what has to change, with the reason recorded. - Add a regression to
autofit_workspace_test/scripts/graphical/that pins the leg-A seed-140
case: on the exactly-Gaussian leg every factor ends with at least one SUCCESS update.
Lineage
- PyAutoFit#1405 — still open, the Laplace-on-scatter collapse basin
- PyAutoFit#1580 — merged, the most recent STALE-reporting fix; this evidence is from a mirror
that contains it (RAL PyAutoFit66f9f8d5dcontains08207bad0) PyAutoCortex/rulings/2026/09/R-20260910-04.md— the ruling- 2026-09-30:
LaplaceOptimiser(projection="moments")(PyAutoFit#1656, recordcomplete/2026/09/ep-moment-projection.md) gives 3/3 HierarchicalFactor SUCCESS on the tight toy (50, 50.5, 49.5) where the default mode path gives 0 (all BAD_PROJECTION); theBAD_PROJECTION-onlyHierarchicalFactorcase may be cured underprojection="moments"— re-check seed 140 / the 18 leg-B seeds with it before fixing the mode path
🤖 Generated with Claude Code
Activity
Jammy2211 commentedon Oct 23, 2026
Library PR Created
PyAutoFit PR #1712 (
feature/ep-zero-success-factors, commit bcaf6f7, label pending-release).Diagnosis verdict: the NO_CHANGE-only PriorFactor is a legitimate converged update, not a stale factor — Newton met its gradient tolerance at the start point (
iter=0), the projection was accepted and the message updated (N(50, 22.4) → N(49.9994, 10.0)), and the final posterior matches the closed form to a = 5e-5, b = 2e-5. The library's STALE tracking never flagged it; the conflation is the ensemble witness's criterion 6 (SUCCESS-only count). The BAD_PROJECTION-only HierarchicalFactor shape (leg B seeds 29/71/106/152/178/193) is the #1405 Laplace-on-scatter state, cured byprojection="moments"(#1656): seeds 29 and 71 re-run under moments giveHierarchicalFactor0 {SUCCESS: 28}/{SUCCESS: 23}, no STALE. Two findings, two causes; only the first needed a (bookkeeping-only) library change. No EP update maths touched; repro numerics bit-identical before/after.Tests: 2 new (one red→green),
test_autofit/graphical322 passed, full suite 3561 passed / 1 skipped / 1 xfailed. Heart YELLOW at ship (manifest drift; PyAutoLens release validation stale) acknowledged by the human.Next: autofit_workspace_test regression leg (seed 140 leg A inside
scripts/graphical/analytic_gaussian.py, +2 s) via ship_workspace; PyAutoCortex criterion-6 amendment;ensemble_seed.py:160diff handed to the human.Possible new scope (not touched): seed 29 leg B under moments still misses the sigma cell (a 0.225 / b 0.416 vs 0.15 / 0.25) while sigma stays inside [q05, q95].
Merge pull request #1712 from PyAutoLabs/feature/ep-zero-success-factors
Jammy2211 commentedon Oct 23, 2026
Shipped — 2026-10-23
PRs
Upstream PR
#1712 (merged abb8bfa, pending-release)
Summary
The NO_CHANGE-only PriorFactor is a converged update (Newton stop condition met at the start point, projection accepted, closed-form match a 5e-5 / b 2e-5), so
optimise_quasi_newtonnow reports itupdated=Truewith the flag kept as NO_CHANGE; the EP stale tracking already handled it. The BAD_PROJECTION-only HierarchicalFactor shape is the #1405 scatter state cured byprojection="moments"(#1656), verified on seeds 29 and 71. Witness criterion 6 now counts SUCCESS or NO_CHANGE as a completed update; the seed-140 leg pins it in autofit_workspace_test.Session Notes
ensemble_seed.py:160 any_factor_never_succeededdiff (count SUCCESS + NO_CHANGE) in the science checkout/mnt/c/Users/Jammy/Science/active/analytic_gaussian; merge Cortex#75 and Insight#38; decide whether Insight's byte-pinnedtasks/ep_campaign.mdgets re-pinned with the in-place edit.git revert abb8bfacbon PyAutoFit main andgit revert 97cb781on autofit_workspace_test main.