Skip to content

test: search conformance suite layer (i) — metadata golden table for 15 searches - #1667

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/search-conformance-metadata
Oct 7, 2026
Merged

Jammy2211 merged 1 commit into
mainfrom
feature/search-conformance-metadata

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

Phase A0a(i) of the search-extensibility epic (closes #1666). A tests-only, parametrised metadata/serialization conformance suite over all 15 public searches, collecting and passing on every CI leg including Heart's unittest-nojax. It freezes a golden table per search (default-construction identifier, __identifier_fields__, constructor-argument names) generated from main 908ca61, so every later refactor phase of track A is checked against today's identifiers. No library source changes; no identifier changes.

Checks per search: default construction, family base class, identifier golden (with PYAUTO_TEST_MODE removed, since test mode changes Emcee/Zeus identifiers), constructor-argument set, search.json round trip (af.to_dict → JSON → af.from_dict → af.to_dict), and conf.instance unchanged by construction (under an output.search_internal: false override, because the test config's true masks the mutation).

Strict xfails (each pinned to one exception type with raises=), recording known defects that later epic phases repair:

  • round trip: BlackJAXNUTS, SMC — the serialised inverse_mass_matrix kind string is rejected by the constructor (phase A0b);
  • config unchanged: Emcee, BlackJAXNUTS, SMC — __init__ sets conf.instance["output"]["search_internal"] = True (phase A3).

Classes are resolved lazily by class-path string inside the tests; only NSS construction requires blackjax and skips when it is absent. No module-level importorskip.

API Changes

None. Tests only.

Test Plan

  • pytest test_autofit/non_linear/search/test_conformance.py: 101 passed, 5 xfailed (full env; also with PYAUTO_TEST_MODE=1)
  • pytest test_autofit/non_linear: 1023 passed, 1 skipped, 5 xfailed
  • no-jax emulation (import hook blocking jax, jaxlib, optax, blackjax; find_spec returns None for them): module collects, 97 passed, 4 skipped (NSS construction cases), 5 xfailed
  • perturbation proof: replacing Drawer's golden identifier with the md5 of Drawer(total_draws=51) fails exactly test_identifier_golden[Drawer] (1 failed, 100 passed, 5 xfailed); reverted
  • --runxfail confirms exactly the five listed cases fail and nothing else
  • CI: unittest (3.12, 3.13) and unittest-nojax legs green
Full API Changes (for automation & release notes)

Removed

  • none

Added

  • none (test modules only: test_autofit/non_linear/search/conformance_roster.py, test_autofit/non_linear/search/test_conformance.py)

Migration

  • none

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QJmrnXNQdq6HSruUt3MqVW

…15 searches (#1666)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJmrnXNQdq6HSruUt3MqVW
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Oct 7, 2026
@Jammy2211
Jammy2211 merged commit 81bfb8e into main Oct 7, 2026
3 checks passed
@Jammy2211
Jammy2211 deleted the feature/search-conformance-metadata branch October 7, 2026 20:37
@Jammy2211 Jammy2211 removed the pending-release PR queued for the next release build label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: search conformance suite layer (i) — metadata golden table for 15 searches

1 participant