Skip to content

fix: EP projection raises ProjectionException on non-finite sufficient statistics; factor_step recovers (#1653) - #1655

Merged
Jammy2211 merged 2 commits into
mainfrom
feature/ep-projection-exception
Sep 30, 2026
Merged

Jammy2211 merged 2 commits into
mainfrom
feature/ep-projection-exception

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

AbstractMessage.project guarded its importance-weighted sufficient statistics with a bare assert np.isfinite(suff_stats).all(). One factor whose samples or log weights went non-finite (nan/inf samples, nan/+inf log weights, all weights zero, or overflow of T(x)·w) therefore raised a context-free AssertionError. That error escaped EP's factor_step recovery tuple and killed the whole graph fit. Under python -O the check vanished and the non-finite message went through silently.

This PR replaces the assert with a typed autofit.exc.ProjectionException(ValueError). The message says which input caused the failure, and the prior id and model parameter path are added on the way up. EP's factor_step now recovers from it the way it already recovers from InitializerException: the factor keeps its previous message for that sweep with StatusFlag.EXCEPTION, and max_consecutive_failures still aborts a factor that keeps failing.

API Changes

  • New exception autofit.exc.ProjectionException, a subclass of ValueError.
  • AbstractMessage.project, Prior.project and Result.projected_model raise it where the bare assert used to fire. The message names the failing input (non-finite samples / nan or +inf log weights / all -inf log weights / weighted-moment overflow), the prior id (Prior.project) and the parameter path (Result.projected_model). The check also survives python -O.
  • EP factor_step lists it in its recovery tuple, so one bad projection becomes a failed sweep update, not a failed graph fit.
  • Behaviour change: non-EP callers of Result.projected_model now get a ValueError subclass instead of AssertionError. Nothing in autofit/ catches AssertionError.
  • No removals, no signature changes.
    See full details below.

Test Plan

  • Red witness on the unmodified source: 10 failed, all AssertionError at autofit/messages/abstract.py:316. That covers 6 parametrised direct cases, 2 prior-id cases, the factor_step / EPOptimiser.run recovery tests and the projected_model path case.
  • New/extended tests green: test_autofit/messages/test_project_nonfinite.py + test_autofit/graphical/functionality/test_factor_failure_recovery.py give 29 passed; test_autofit/graphical/test_unification.py gives 34 passed.
  • Full suite after rebase on origin/main: python -m pytest test_autofit -x, 2946 passed, 2 skipped.
  • Workspace smoke (smoke profile: PYAUTO_TEST_MODE=2, PYAUTO_DISABLE_JAX=1) against this branch, autofit_workspace_test: graphical/analytic_ep_minimal.py PASS (MINIMAL EP SELF-TESTS: PASS, 2 s). graphical/analytic_gaussian_collapse.py PASS (COLLAPSE CONFIG: PASS 5/5 seeds, 18 s).
Full API Changes (for automation & release notes)

Added

  • autofit.exc.ProjectionException(ValueError): non-finite sufficient statistics in an importance-weighted message projection. Deliberately not a MessageException (callers turn those into silent reverts or -inf likelihoods) and not a SearchException (misconfigured search).
  • AbstractMessage._nonfinite_projection_reason(samples, log_weight_list, suff_stats) (private classmethod). Called only on the failure path, it reports the first cause in propagation order.

Changed Behaviour

  • autofit.messages.abstract.AbstractMessage.project: raises ProjectionException instead of assert on non-finite sufficient statistics.
  • autofit.mapper.prior.abstract.Prior.project: re-raises ProjectionException with prior id=<id> (<PriorClass>) context (from e).
  • autofit.non_linear.result.Result.projected_model: re-raises ProjectionException with path=<a.b.c> context (from e). Callers get ValueError rather than AssertionError.
  • autofit.graphical.expectation_propagation.optimiser.factor_step: exc.ProjectionException added to the recovery tuple. The result is the previous message, a StatusFlag.EXCEPTION status, and a logged exception.

Migration

  • None. Code that caught AssertionError from projected_model / project should catch ValueError (or autofit.exc.ProjectionException). No such caller exists in autofit/.

Generated by the PyAutoLabs agent workflow.

Heart RED override (development only)

  • Authorised by the live human in the Claude Code session at 2026-09-30 ~11:50 BST. Question: "Do you authorise the Heart RED development override for push + pending-release PR-open on PyAutoFit#1653, and, when its implementation lands green, on PyAutoFit#1654 (moment projection)? No merge, no release; recorded in the four sinks." Answer: "Yes, both fix: EP projection raises ProjectionException on non-finite stats #1653 and feat: LaplaceOptimiser(projection="moments") for hierarchical scatter #1654 (Recommended)".
  • Exact RED reasons (readiness --json, ts 2026-09-30T09:51:49Z, re-read ~11:45 BST unchanged): "PyAutoArray: 2 commit(s) behind origin"; "PyAutoLens: 2 commit(s) behind origin"; "release validation FAILED (stage integrate)". Yellow: "workspace validation not passing (0 failed, 1 timeout, cloud#36404726969: autolens_test scripts/multi_dataset/rectangular.py)"; "manifest drift: public front-door organ tables (generated) — 1 mismatch(es) vs PyAutoMind/repos.yaml".
  • Branch gates passed: rebase on origin/main clean; pytest test_autofit -x 2946 passed, 2 skipped; red witness confirmed on the old source; graphical smoke 2/2 PASS.
  • This PR does not claim to fix Heart. Heart stays RED for release purposes. Merge needs its own human command (/prm) and green required checks.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Up1fQmjtGAh4RhLrWHfcfg

Jammy2211 and others added 2 commits September 30, 2026 11:27
…— direct project cases + factor_step recovery

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Up1fQmjtGAh4RhLrWHfcfg
…ufficient statistics; factor_step recovers (PyAutoFit#1653)

AbstractMessage.project ended on a bare `assert np.isfinite(suff_stats).all()`.
AssertionError is not in factor_step's recovery tuple, so one factor's bad
projection killed a whole EP run (RAL 342411: ic50 ladder, N=50, sweep 4,
global factor), with no hint of which input was non-finite.

- exc.ProjectionException(ValueError) replaces the assert. The new
  _nonfinite_projection_reason classmethod (failure path only) names the
  cause: non-finite samples (counts, first index), nan / +inf log weights,
  all log weights -inf, or overflow of T(x)·w (max |x|). Moment maths is
  unchanged.
- Prior.project re-raises with the prior id and type (the id does not reach
  AbstractMessage.project through TransformedMessage.project), and
  Result.projected_model re-raises with the parameter path (loop instead of
  a dict comprehension; the zero-weight guard stays).
- factor_step lists exc.ProjectionException in its except tuple (already
  covered via ValueError; listed for intent): a non-finite projection of one
  factor's samples is a failed sweep update, not a failed graph fit. It gets
  StatusFlag.EXCEPTION, keeps the previous message, and counts toward
  max_consecutive_failures. AssertionError is deliberately not caught.

Why not MessageException: mean_field.py, hierarchical.py and laplace/newton.py
turn that into a silent revert or a -inf likelihood. Why not SearchException:
it is uncaught and means a misconfigured search.

Behaviour change: outside EP, Result.projected_model callers now get a clear
ProjectionException (a ValueError) where they got AssertionError; no autofit/
code catches AssertionError. Under `python -O` the old assert vanished and the
non-finite statistics flowed on silently.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Up1fQmjtGAh4RhLrWHfcfg
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Sep 30, 2026
@Jammy2211
Jammy2211 merged commit 07f6e0b into main Sep 30, 2026
4 checks passed
@Jammy2211
Jammy2211 deleted the feature/ep-projection-exception branch September 30, 2026 13:52
@Jammy2211 Jammy2211 removed the pending-release PR queued for the next release build label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant