Repository navigation
fix: A2/A3/A3b post-merge fixes from astra review 05 (search-extensibility) #1682
Description
Overview
Post-merge Codex gpt-6-astra review 05 of the search-extensibility A2/A3/A3b PRs (#1679/#1680/#1681) returned seven findings, four of them release-severity. They shipped in 2026.10.11.1 and 2026.10.12.1 by a deliberate human call, so they are now live bugs in released code. This task fixes all seven in one library PR before A4/B4 proceed. Epic ledger: PyAutoMind draft/research/autofit/search_extensibility_epic.md ("Resume"); review: search_extensibility_epic_reviews/05_codex_astra_a2_a3_a3b_postmerge.md.
Plan
- Reproduce each finding against current main with a failing test first; every earlier astra finding reproduced, so a non-reproducing one is a reason to stop and check, not to skip.
- Restore eager semantics for old
Fitnesspickles and share one resolved execution config between preflight and the sampler. - Finalize the archive before writing
.completed, and keep NSS resume state until the result exists. - Pass the analysis into the fallback
PoolFactory, close context-created pools on success, and persist the checkpoint strategy fromctx.update. - Make the golden diagnostics test value-sensitive while keeping the CSV tolerance.
- Run the disk-backed resume checks in a writable environment and quote the outputs on the PR.
Tier: judge — merge mode: human /prm
Detailed implementation plan
Affected Repositories
- PyAutoFit (primary, only)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./PyAutoFit | main | clean |
Suggested branch: feature/search-ext-a3-postmerge-fixes
Implementation Steps
- F1 old-pickle JIT —
autofit/non_linear/fitness.py__setstate__: when the state carries a legacyuse_jax_jitkey, map it tocompileinstead of popping it and defaultingcompile=True. Regression test: pickle aFitness(use_jax_jit=False)built from the A1-era attribute set (construct the legacy__dict__by hand), unpickle, assert the scalar objective is the eager callable (not ajax.jitwrapper). - F3 preflight resolved config —
autofit/non_linear/search/preflight.py: introduce one resolved-settings object (objective kind,compile,gradient_mode, batched) built by the same code path the search uses inmake_fitness/start; preflight traces that, not the class declaration. Tests:use_jax_jit=Falseeager likelihood passes preflight;gradient_mode="reverse"override passes; a genuinely untraceable likelihood still fails. - F2
.completedordering —autofit/non_linear/search/abstract_search.py(paths.completed()aftersave_results, and the second call near line 1349): move archive finalization (zip/paths.zip_remove-style step) beforepaths.completed(); on rerun, a folder with results but no.completedresumes instead of raising.autofit/non_linear/search/nest/nss/search.py:resume_state.discard(ctx.paths)moves after the result is written. Tests: simulate an archive-write failure aftersave_results→ no.completed; rerun completes. - F4 fork rule —
abstract_search.py~line 2012:PoolFactory(self, analysis)in the no-active-factory branch of_fit. Test: a JAX analysis withnumber_of_cores=2calling_fitdirectly asserts the factory refuses to fork (patchmultiprocessingstart / count pool creations). - F5/F6 FitContext contract —
autofit/non_linear/search/fit_context.py: track pools the context itself built and close them on the success path too (currently only on failure, ~line 139);ctx.update(~line 125) calls the checkpointer's save. Test with a resumable toy search on both exit paths. - F7 golden diagnostics —
test_autofit/.../test_golden_samples.py(~line 104): assert ESS/R-hat values within a tolerance of the fixture, and add a unit test on the diagnostics source (blackjax summary →BACKEND_DIAGNOSTIC_KEYS) that fails on sentinel -999. Keepassert_allclose(rtol=1e-12, atol=0)for the CSV. - Disk-backed checks — in the worktree (writable): load a 2026.10.7.1-era output folder, interrupt-and-resume Dynesty and Emcee, restore from a zipped output. Paste the commands and tails into the PR body.
- Emulate the
unittest-nojaxleg locally (block jax/blackjax imports) before ship; keep the Sphinx warning count at or below the CI limit.
Key Files
autofit/non_linear/fitness.py—__setstate__legacy mappingautofit/non_linear/search/preflight.py— resolved execution settingsautofit/non_linear/search/abstract_search.py—.completedordering,PoolFactoryfallbackautofit/non_linear/search/fit_context.py— pool close on success, checkpoint saveautofit/non_linear/search/nest/nss/search.py— resume-state discard timingtest_autofit/non_linear/search/test_golden_samples.py— value-sensitive diagnostics
Out of scope
- Review 05 item 8 (consolidating Drawer dill / Nautilus pool context / NSS checkpoint lifecycle) — later task.
- Prior/Message design (design: unify the Prior/Message hierarchy around a single-source density #1500) — untouched.
Original Prompt
Click to expand starting prompt
@PyAutoFit search-extensibility A2/A3/A3b post-merge fixes (astra review 05)
Type: bug
Target: autofit
Repos:
- PyAutoFit
Difficulty: large
Autonomy: supervised
Priority: high
Status: formalised
Consequence: judge
Epic: search-extensibility
Phase: 3c
Witness: (1) aFitnesspickled from the A1 release (compile-less,use_jax_jit=False) unpickles with an eager scalar objective — a regression test asserts the restored objective is not jitted; (2) preflight accepts an eageruse_jax_jit=Falselikelihood and agradient_mode="reverse"override by validating the same resolved settings the backend runs; (3) killing a search aftersave_resultsbut before the archive finalizes leaves no.completed, and a rerun resumes and completes; NSS keeps its resume state until the result is written; (4) calling_fitdirectly with a JAX analysis never forks (asserted in a test); (5) a resumable toy search run throughrun(ctx)closes context-created pools on both the success and failure exit paths, andctx.updatepersists the checkpoint strategy; (6) the golden diagnostics test fails when ESS/R-hat are replaced with sentinel values; (7) the disk-backed checks (loading a pre-release output folder, interrupt-and-resume, zipped restore) ran in a writable environment and their outputs are quoted on the PR.
Review-minutes: 30
Unattended: needs-slicing
Filed: 2026-10-14
Original request
Continue working on the PyAutoFit refactor epic. [...] I am happy to follow
your recommendation on everything else, try do in groups of tasks so theres
less input from me.
Recommendation adopted 2026-10-14: issue the review-05 fixes as one PyAutoFit
library task before A4/B4. The human ruled that releasing 2026.10.11.1 and
2026.10.12.1 with these findings in was deliberate, so they are now live bugs
in released code rather than release gates; the fixes go into the next release.
Source
draft/research/autofit/search_extensibility_epic_reviews/05_codex_astra_a2_a3_a3b_postmerge.md
(FINDINGS 7; §3 findings, §6 resume actions) and the ledger
draft/research/autofit/search_extensibility_epic.md "Resume (2026-10-08 night)".
Verify each finding against current main before fixing it; every earlier astra
finding reproduced. All seven are confirmed still present on main at
911fe05e1 (2026-10-14).
Items (ledger order)
- Old
Fitnesspickle silently enables JIT —fitness.py__setstate__
popsuse_jax_jitand defaultscompile=True. Keepuse_jax_jit=False
eager; add a pickle-from-A1 regression test. (F1, P1) - Preflight validates the declared config, not the resolved runtime config
—preflight.pytraces the declared objective kind, so it wrongly rejects
use_jax_jit=Falseeager likelihoods and thegradient_mode="reverse"
override. Resolve the execution settings once and share them between
preflight and sampling. (F3) .completedis written before the archive finalizes —
abstract_search.pywritespaths.completed()straight after
save_results; NSS discards its resume state during result processing
(nss/search.pyresume_state.discard). A failed archive write leaves an
unrecoverable "completed" fit. Finalize first, recover on rerun. Predates A2
but in scope. (F2, P1)- Direct
_fitbuildsPoolFactory(self)without the analysis
(abstract_search.py~line 2012), so Nautilus JAX can fork with 2 cores.
Pass the analysis; assert no fork happens. (F4) FitContextleaks context-created pools on success (fit_context.py
closes pools only on failure) andctx.updatenever saves the checkpoint
strategy. Both violate the frozenrun(ctx)contract. Test with a
resumable toy search on both exit paths. (F5, F6)- Shape-only golden diagnostics (
test_golden_samples.py) pass with
ESS/R-hat set to -999. Add value-sensitive tests of where the diagnostics
come from; keep the CSV tolerance (rtol 1e-12). (F7) - Disk-backed checks in a writable environment: loading pre-release output
folders, interruption and resume, zipped restore. Quote results on the PR.
Out of scope (carried to a later task): consolidating backend-specific
lifecycle paths (Drawer's dill, Nautilus's own pool context, NSS deleting its
checkpoint) — review 05 item 8, tagged [later].
Constraints
- Library only; no workspace edits. Downstream workspace adoption of A1/A2 was
already repaired on 2026-10-10 (complete/2026/10/heart-fitness-dispatch.md,
heart-hierarchical-backend.md) — do not redo it. - Emulate the
unittest-nojaxleg before ship (new tests that import jax or
blackjax needimportorskip). - Sphinx warning budget: do not push the count over the CI limit.
- Not the Prior/Message design (design: unify the Prior/Message hierarchy around a single-source density #1500): untouched.
Activity
Jammy2211 commentedon Oct 14, 2026
Shipped
PR #1683 merged 2026-10-14 (8437a3b), label
pending-release. All seven review-05 findings reproduced on main and were fixed test-first; both CI legs (unittest 3.12/3.13, nojax) and Docs green; the interrupt/resume and old-folder witnesses are on the PR. Record:PyAutoMind/complete/2026/10/search-ext-a3-postmerge-fixes.md. Review-05 item 8 (lifecycle consolidation) is carried in the epic ledger as a later task.