Work Classification

Library

Affected Repositories

• PyAutoFit (primary)

Branch Survey

Suggested branch: feature/search-conformance-metadata
Worktree root: ~/Code/PyAutoLabs-wt/search-conformance-metadata/

Implementation Steps

1. test_autofit/non_linear/search/conformance_roster.py: a searches_under_test() helper returning 15 entries (class_path string, short name, family, requires module list). Classes are resolved with importlib inside the test, never at collection. Roster: Emcee, Zeus, BlackJAXNUTS, SMC, DynestyStatic, DynestyDynamic, Nautilus, NSS, Drawer, BFGS, LBFGS, MultiStartAdam, MultiStartADABelief, MultiStartLion, MultiStartProdigy. The module docstring states the roster and the coverage contract.
2. Skip rule: a construction-dependent case skips only when its requires module is absent (NSS raises ImportError without blackjax ≥ 1.6; every other search constructs with defaults even when its backend is missing). No module-level importorskip.
3. test_conformance.py, parametrised per search:
   • test_default_construction: cls() succeeds.
   • test_identifier_golden: str(Identifier(search)) with PYAUTO_TEST_MODE removed from the environment (test mode changes Emcee and Zeus identifiers) equals the frozen golden table committed as data in the module. Also freezes each class's __identifier_fields__ tuple.
   • test_constructor_argument_set: the inspect.signature parameter names equal the frozen set.
   • test_search_json_round_trip: af.to_dict → json.dumps/loads → af.from_dict → af.to_dict equal. xfail(strict=True) for BlackJAXNUTS and SMC: the serialised inverse_mass_matrix is a kind string the constructor rejects. Repair goes to A0b and flips the xfail.
   • test_config_unchanged_by_construction: push a config override with output.search_internal: false (the test config has it true and masks the mutation), snapshot the config dict before and after construction, compare. xfail(strict=True) for Emcee, BlackJAXNUTS and SMC until A3.
   • test_family_base: each class subclasses the expected AbstractMCMC / AbstractNest / AbstractMLE.
4. Generate the golden table once from main, commit it, and verify a deliberate perturbation (one default argument) fails exactly one parametrised case.
5. Deferred to layer (ii): samples_info key sets (only obtainable from live sampler objects) and any backend execution.

Verification

• pytest test_autofit/non_linear green on the full-extras env.
• The same with jax, jaxlib, optax, blackjax blocked via an import hook (mirrors unittest-nojax, where nautilus, zeus, emcee and dynesty remain installed): NSS cases skip, the module still collects.

Key Files

• test_autofit/non_linear/search/test_conformance.py — new suite
• test_autofit/non_linear/search/conformance_roster.py — roster helper and golden tables
• autofit/mapper/identifier.py — identifier hashing read for reference (not edited)
• autofit/__init__.py — eager exports and lazy NSS/SMC (_LAZY_ATTRS) read for reference (not edited)

Search conformance suite, layer (i): metadata and serialization for all 15 searches on every CI leg (epic search-extensibility, phase A0a(i))

Type: test
Target: autofit
Repos:

• PyAutoFit
  Themes:
• searches
• jax
• ci
  Difficulty: medium
  Autonomy: safe
  Priority: high
  Epic: search-extensibility
  Status: draft
  Filed: 2026-10-07

Phase A0a(i) of the search-extensibility epic
(draft/research/autofit/search_extensibility_epic.md; plan in
search_extensibility_epic_report.md §4 A0a, decision D1 in §8.1). Tests
only. It is the safety net every later phase of track A runs against and the
only track-A dependency of B2.

Original request (verbatim from the epic plan)

"A0a, conformance suite in two layers. Repo: PyAutoFit, tests only. Layer
(i), metadata/serialization, all 15 searches, every CI leg (including
unittest-nojax; the module never imports af.NSS/af.SMC at collection
time): construction, search.json round trip and constructor-argument sets,
identifiers against a frozen golden table of all 15 default constructions,
declared capability attributes, and conf.instance unchanged by construction
(xfail for Emcee, NUTS and SMC until A3). Risk: none. Verify: pytest test_autofit/non_linear on a full-extras env and with jax/blackjax/optax
uninstalled. Witness: layer (i) green on all legs."

Scope

Add test_autofit/non_linear/search/test_conformance.py (plus a small
searches_under_test() helper module beside it) parametrised over every
public search exported from af:

• the 15 searches: Emcee, Zeus, NUTS, SMC (mcmc); DynestyStatic, DynestyDynamic,
  Nautilus, NSS (nest); LBFGS, BFGS, Drawer, MultiStartLBFGS/BFGS/Prodigy
  family (mle). Confirm the exact roster from autofit/non_linear/search/
  and the af exports before freezing the table, and record it in the module
  docstring.
• Collection never imports an optional backend: resolve search classes lazily
  by class_path string inside the test (importlib) so the module collects on
  the unittest-nojax leg; a search whose backend is absent is skipped by
  that fact alone for the construction-dependent checks, never by a
  module-level importorskip. Emulate the no-jax leg locally with an
  import-hook block before shipping (memory: nojaxSkip).
• Per search: default construction; search.json write → from_dict
  round trip equals the original (compare the serialised dicts); the
  constructor-argument set snapshot; identifier of the default construction
  against a frozen golden table committed as data in the test module (this
  table is the §3.2 invariant every later phase of the epic must hold);
  declared capability attributes present where they already exist today
  (assert only what exists; A1 adds the full set); conf.instance unchanged
  by construction, with pytest.mark.xfail(strict=True) for Emcee, NUTS and
  SMC until A3.
• Snapshot the per-search samples_info key sets only if obtainable without
  running a backend; otherwise defer to layer (ii).

Out of scope

Backend execution (layer (ii), after A0b), any source edit under autofit/,
the hygiene list (A0b) and the documentation repairs (A0c).

Verification

• pytest test_autofit/non_linear green in a full-extras env.
• The same green with jax/blackjax/optax blocked via an import hook (the
  unittest-nojax leg); JAX-only searches skip, the module still collects.
• Golden table: regenerate once, commit, and prove a deliberate identifier
  perturbation fails the test.

Witness

Layer (i) green on every CI leg; deleting a row from the golden table or
changing a default argument fails exactly one parametrised case.