Skip to content

feat: put the clipper in the search identifier (MLE searches only) #1493

Description

@Jammy2211

Overview

The prior-support Clipper (#1477) does not enter the search identifier, so two runs differing only in prior-support enforcement — which demonstrably changes the answer — share one output directory, and with the .completed short-circuit the later run silently returns the earlier one's numbers (this bit the phase-2 validation campaign, which worked around it with unique per-arm names). Decision recorded in PyAutoMind (2026-08-18, human): the clipper enters the identifier, scoped to the searches that consume one; nested-sampler and MCMC identifiers must stay byte-for-byte identical.

Plan

  • Declare the clipper an identifier field on the two clipper-consuming search families (multi-start gradient and BFGS), so runs with different clippers get different output directories.
  • Leave the nested samplers, MCMC searches, and Drawer completely untouched — their identifiers must not re-key.
  • Pin the nested-sampler identifiers with regression tests captured on main before the change, plus a structural tripwire so a future refactor cannot silently put the clipper within their reach.
  • State plainly in the release notes that existing multi-start / (L)BFGS output directories re-key (orphaned on disk, not deleted).
Detailed implementation plan

Affected Repositories

  • PyAutoFit (primary, only)

Branch Survey

Repository Current Branch Dirty?
./PyAutoFit main @ c302f51 clean

Suggested branch: claude/clipper-search-identifier-4cowi2

Mechanism (verified at c302f51)

  • The identifier hashes [search, model] (+ unique_tag) — autofit/non_linear/paths/abstract.py:279-283.
  • An object declaring __identifier_fields__ contributes only those attributes plus its class name — autofit/mapper/identifier.py:113-121; an empty tuple contributes the class name only.
  • NonLinearSearch.__identifier_fields__ = tuple() (abstract_search.py:318) and neither AbstractMultiStartGradient nor AbstractBFGS overrides it — so today only their class name hashes, which is why the clipper (and everything else on them) never entered.
  • clipper is resolved on AbstractMLE.__init__ (mle/abstract_mle.py:16-22); the nested samplers and MCMC searches contain zero clipper references and declare their own explicit __identifier_fields__ tuples, so they are unchanged by construction.

Implementation Steps

  1. autofit/non_linear/search/mle/multi_start_gradient/search.py — add __identifier_fields__ = ("clipper",) to AbstractMultiStartGradient (covers MultiStartAdam/ADABelief/Lion/Prodigy).
  2. autofit/non_linear/search/mle/bfgs/search.py — add __identifier_fields__ = ("clipper",) to AbstractBFGS (covers BFGS/LBFGS).
  3. Drawer untouched: it inherits the clipper attribute but never uses it (zero references in mle/drawer/search.py) — a setting that cannot affect the result must not re-key it. Its __identifier_fields__ = ("total_draws",) stands.
  4. No change under nest/ or mcmc/.

Tests

  • test_autofit/database/identifier/test_identifiers.py::test_dynesty_static (line 389) pins DynestyStatic's exact hash list — must pass unchanged; it is the existing guard for the hard constraint.
  • Add sibling hash-list pins for Nautilus and DynestyDynamic (ideally Emcee/Zeus/NUTS too), captured on main before the change and asserted after.
  • Structural tripwire: nested samplers have no clipper attribute (assert not hasattr(af.Nautilus(), "clipper")) — trips if a refactor hoists clipper from AbstractMLE to NonLinearSearch.
  • MLE side: MultiStartAdam() vs explicit clipper=ClipperNone() identical; vs ClipperPriorBox() different; two ClipperPriorBox margins different; same trio on LBFGS.

Key Files

  • autofit/non_linear/search/mle/multi_start_gradient/search.py — first identifier-fields declaration
  • autofit/non_linear/search/mle/bfgs/search.py — second identifier-fields declaration
  • test_autofit/database/identifier/test_identifiers.py — pins + tripwire + MLE separation tests

Question for the human at PR time

Because these searches' __identifier_fields__ is empty today, n_starts, total_steps, learning_rate etc. also collide. This PR re-keys the MLE directories anyway, so widening the tuple now would pay the orphaning cost once instead of twice — but that widens scope beyond the recorded decision. Default is clipper-only; if widening is declined, it gets filed as its own prompt.

Accepted cost (for the release notes)

Every existing MultiStart* / BFGS / LBFGS output directory re-keys, including default-ClipperNone runs (the hash gains the clipper field itself). Stored results are orphaned on disk, not deleted; re-running recomputes into a fresh directory. Nested-sampler and MCMC results are unaffected.

Original Prompt

Click to expand starting prompt

PyAutoMind active/clipper_in_search_identifier.md (formerly draft/feature/autofit/) — the decided revision of the prompt, recorded 2026-08-18. See that file for the full decision record, the rejected options, and the grounding audit.

Decision, verbatim: "The clipper enters the search identifier — but only on the searches that consume one, and the nested samplers' identifiers must stay byte-for-byte identical."

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions