Skip to content

docs: search documentation and coverage repair, fit family + Brain (search-extensibility A0c part 1) #1668

Closed

Description

@Jammy2211

Overview

Phase A0c part 1 of the search-extensibility epic (PyAutoMind draft/research/autofit/search_extensibility_epic.md; plan in search_extensibility_epic_report.md §4, item list in search_extensibility_epic_surveys/03_search_docs.md §4). Repairs the search documentation and coverage where the search facts originate: PyAutoFit docs/citations, the autofit workspace family, the autofit assistant and the Brain samplers faculty + sampler_pipeline skill. Part 2 (downstream PySwarms/MultiNest ghosts and "LBFGS only" claims in the al/ag/cti family) is filed separately after this lands. Human rulings: RTD generic example stays DynestyStatic (2026-10-07); the superseded autofit_workspace_developer/searches/nss/ directory is deleted via Gut, not bannered (2026-10-08).

Plan

  • PyAutoFit: list all 15 searches in docs/api/searches.rst (Nautilus, NSS, Drawer, MultiStart×4 + autosummary stubs); add a Nautilus cookbook section; fix the stale config README, configs.md, installation/source.md and the citations files (drop MultiNest/pymultinest ghosts, add blackjax/optax/prodigy, add Nautilus, fix the sqlite key).
  • autofit_workspace: mirror the Nautilus cookbook section; fix the searches README roster (add BlackJAXNUTS, MultiStartAdam; drop NSS and the autofit[nss] block); drop NSS from llms.txt; reword the LBFGS/JAX line in mle.py; regenerate notebooks.
  • autofit_workspace_test: fix the config README; separate PR adds executable scripts/searches/{SMC,Drawer,BFGS,MultiStartADABelief,MultiStartLion}.py (on-demand, not in smoke_tests.txt).
  • autofit_workspace_developer: delete searches/nss/ via Gut (committed deletion, condemned.md entry with the pre-delete SHA) and fix the root README archive definition.
  • autofit_assistant: refresh skills/af_configure_search.md roster and stale claims, wiki/core pages and the config README.
  • PyAutoBrain, two PRs: (B) samplers faculty gap rule keyed on exported class names + unit test; (C) sampler_pipeline Boundary → autolens_inference, drop [nss]/config-mirroring lines, add "Stage 5 — documentation".
  • Shape: one PR per repo for prose repairs plus the three separate PRs named above (workspace_test scripts, Brain B, Brain C). Library (PyAutoFit) first; it is docs-only, no API change.

Tier: judge (undeclared) — merge mode: human /prm

Detailed implementation plan

Affected Repositories

  • PyAutoFit (primary, library; docs-only)
  • autofit_workspace
  • autofit_workspace_test
  • autofit_workspace_developer
  • autofit_assistant
  • PyAutoBrain

Branch Survey

Repository Current Branch Dirty?
fit/PyAutoFit main clean
fit/autofit_workspace main clean
fit/autofit_workspace_test main clean
fit/autofit_workspace_developer main clean
fit/autofit_assistant main 1 untracked (scripts/compose_model_gaussians_exponentials.py, unrelated, left alone)
organs/PyAutoBrain main clean

Suggested branch: feature/search-ext-a0c-fit-repair (worktree ~/Code/PyAutoLabs-wt/search-ext-a0c-fit-repair)

Implementation Steps

  1. PyAutoFit — docs/api/searches.rst:27-28,55-56: add Nautilus, NSS, Drawer, MultiStartAdam/ADABelief/Lion/… entries + 7 _autosummary stubs; docs/cookbooks/search.md: Nautilus section between DynestyStatic and LBFGS, soften "every search" at :20; autofit/config/non_linear/README.md (PyAutoLens → PyAutoFit; only GridSearch.yaml exists); docs/general/configs.md:3; docs/installation/source.md:18,37 → pip install -e PyAutoFit / -e "PyAutoFit[optional]"; files/citations.{bib,md,tex}: drop multinest/pymultinest, add blackjax (Cabezas+ 2024, arXiv:2402.10797), optax, prodigy (Mishchenko & Defazio 2023, arXiv:2306.06101), add Nautilus to .md/.tex, fix [@sqlite] → sqlite2020. Verify: GitHub docs legs green (RTD red on the autonerves floor, ignore); grep -ri "pyswarms\|multinest" docs/ files/ empty.
  2. autofit_workspace — scripts/cookbooks/search.py Nautilus mirror + contents bullet; scripts/searches/README.md:5-7 roster (+BlackJAXNUTS, +MultiStartAdam, −NSS) and delete :10-22; llms.txt:28 drop NSS; scripts/searches/mle.py:84-85: LBFGS runs under either backend (bfgs/search.py:298-306 calls fitness._jit; gradients finite-difference), use_jax=False here for speed, MultiStartAdam requires use_jax=True. Regenerate notebooks with the workspace generate.py. Verify: smoke searches/{mcmc,nest,mle}.py + cookbook under PYAUTO_TEST_MODE=1.
  3. autofit_workspace_test — PR a: config/non_linear/README.md:6-8. PR b: scripts/searches/{SMC,Drawer,BFGS,MultiStartADABelief,MultiStartLion}.py modelled on BlackJAXNUTS.py / LBFGS.py / MultiStartAdam.py; run each once under PYAUTO_TEST_MODE=1; not added to smoke_tests.txt.
  4. autofit_workspace_developer — git rm -r searches/nss (pre-delete SHA recorded at commit time; main is 7cb97c6 today); root README.md archive definition. Mind condemned.md entry autofit_workspace_developer/searches-nss (type file, merged yes, archive-ref: n/a — committed deletion; recoverable at pre-delete SHA), written at ship.
  5. autofit_assistant — skills/af_configure_search.md:3,21-26,103-105,117 (roster +NSS/SMC/BFGS/MultiStart×4; drop "until the core wiki lands"; drop the per-search YAML claim; "HowToFit chapter 2" → chapter_1_introduction/tutorial_3_non_linear_search); wiki/core/concepts/non_linear_search.md:57; wiki/core/stack/autonerves.md:18; config/non_linear/README.md:6-8.
  6. PyAutoBrain PR B — agents/faculties/samplers/_samplers.py: gap rule keyed on exported class names (parse autofit/__init__.py exports + _LAZY_ATTRS), class-vs-integration-stem match with _jax stripped, archive names that are also promoted excluded, searches_minimal variant suffixes (_mlp, _profile, _sweep) handled; unit test; verify bin/pyauto-brain samplers tier-gaps shows no false positive for MultiStart/SMC/BFGS and NSS once.
  7. PyAutoBrain PR C — skills/sampler_pipeline/{SKILL.md,reference.md}: Boundary → autolens_inference (SKILL.md:73,111-113, reference.md:98,113); drop [nss] extra and config-mirroring lines (reference.md:142,146-149); add "Stage 5 — documentation" after Stage 4 using surveys/03 §3 as the checklist; "Stages 3–5" at SKILL.md:88-91 and reference.md:184.
  8. Ship: /ship_library (PyAutoFit) → /ship_workspace (the rest, each its own PR); repos_sync --check clean; Mind active.md lists every PR.

Key Files

  • PyAutoFit/docs/api/searches.rst, docs/cookbooks/search.md, files/citations.{bib,md,tex}
  • autofit_workspace/scripts/searches/README.md, scripts/searches/mle.py, scripts/cookbooks/search.py, llms.txt
  • autofit_workspace_test/scripts/searches/ (new scripts), config/non_linear/README.md
  • autofit_workspace_developer/searches/nss/ (delete), README.md
  • autofit_assistant/skills/af_configure_search.md, wiki/core/concepts/non_linear_search.md
  • PyAutoBrain/agents/faculties/samplers/_samplers.py, skills/sampler_pipeline/{SKILL.md,reference.md}

Notes

  • Feature Agent: declared medium, derived too-large; the per-repo PR split keeps each review small. No public API change.
  • Nothing here touches search behaviour; A0b/A0a(ii)/A1 follow separately.

Original Prompt

Click to expand starting prompt

Search documentation and coverage repair, fit family + Brain (epic search-extensibility, phase A0c part 1)

Type: docs
Target: autofit
Repos:

  • PyAutoFit
  • autofit_workspace
  • autofit_workspace_test
  • autofit_workspace_developer
  • autofit_assistant
  • PyAutoBrain
    Themes:
  • searches
  • documentation
  • jax
    Difficulty: medium
    Autonomy: supervised
    Priority: high
    Epic: search-extensibility
    Status: draft
    Filed: 2026-10-07

Phase A0c of the search-extensibility epic (draft/research/autofit/search_extensibility_epic.md;
plan in search_extensibility_epic_report.md §4 A0c; item list in
search_extensibility_epic_surveys/03_search_docs.md §4, re-verified against main on
2026-10-07 by an Opus survey). A0c is split in two issues because it spans sixteen repos:
part 1 (this prompt) repairs the fit family and the Brain, where the search facts
originate; part 2 sweeps the downstream PySwarms/MultiNest ghosts and "LBFGS only"
claims in PyAutoLens/PyAutoGalaxy/PyAutoCTI docs, HowToLens/HowToGalaxy, autogalaxy_workspace,
autocti_workspace and the al/ag assistants. Human ruling 2026-10-07: the RTD generic
example stays DynestyStatic.

Original request (verbatim from the epic plan)

"A0c, documentation and coverage repair (surveys/03 phase 1; 'repair', not docs-only, D8).
Scope: the surveys/03 §4 list: (1–2) add Nautilus, NSS, Drawer and MultiStart×4 to
docs/api/searches.rst, and add a Nautilus cookbook section plus its workspace mirror; (3–4)
remove the stale NSS claims from the workspace README and llms.txt:28, and mark
autofit_workspace_developer/searches/nss as superseded; (5) delete the per-search-YAML claims
×7; (6–7) purge PySwarms and MultiNest ghosts from docs, CITATIONS and the bib, and add
blackjax, optax and prodigy to files/citations.bib; (8) fix the optional_requirements.txt
reference; (9) fix the staleness in af_configure_search.md; (10) fix the SamplerSurface
gap-rule bugs: false positive for MultiStart, false negatives for SMC and BFGS, and NSS
double-counted; (11) fix the sampler_pipeline Boundary → autolens_inference; (12) fix the
duplicated ag Start Point header and the HowToLens/Galaxy 'LBFGS only' claims; correct the
'use_jax=False is required by LBFGS' line (mle.py:84). Separate PRs within A0c:
workspace_test integration scripts for SMC, Drawer and BFGS (executable); the Brain
SamplerSurface class-keyed gap fix (item 10, parser code); and a 'Stage 5 — documentation'
in sampler_pipeline/reference.md, using the surveys/03 §3 blast radius as the interim
checklist. Shape: one PR per repo for the prose repairs, plus the separate PRs above."

Scope, part 1 (verified 2026-10-07 against main)

  • PyAutoFit (docs/api/searches.rst:27-28,55-56 + 7 _autosummary stubs; docs/cookbooks/search.md
    Nautilus section between DynestyStatic and LBFGS, soften "every search" at :20;
    autofit/config/non_linear/README.md (says PyAutoLens, lists mcmc/nest/mle.yaml — only GridSearch.yaml exists);
    docs/general/configs.md:3; docs/installation/source.md:18,37 (both requirements.txt files are missing →
    pip install -e PyAutoFit / -e "PyAutoFit[optional]"); files/citations.{bib,md,tex}: drop multinest/pymultinest,
    add blackjax (Cabezas+ 2024, arXiv:2402.10797), optax, prodigy (Mishchenko & Defazio 2023, arXiv:2306.06101),
    add Nautilus to .md/.tex, fix the [@sqlite]→sqlite2020 key).
  • autofit_workspace (scripts/cookbooks/search.py Nautilus mirror + contents bullet; scripts/searches/README.md:5-7
    roster (add BlackJAXNUTS, MultiStartAdam; drop NSS) and delete the :10-22 "When to use NSS"/autofit[nss] block;
    llms.txt:28 drop NSS; scripts/searches/mle.py:84-85 — LBFGS runs under JAX (bfgs/search.py:298-306 calls
    fitness._jit; gradients stay finite-difference), so reword as "either backend; use_jax=False here for speed;
    MultiStartAdam below requires use_jax=True"). Regenerate notebooks.
  • autofit_workspace_test (config/non_linear/README.md:6-8), and the separate scripts PR: executable
    scripts/searches/{SMC,Drawer,BFGS,MultiStartADABelief,MultiStartLion}.py modelled on BlackJAXNUTS.py/LBFGS.py/
    MultiStartAdam.py, run on demand, not added to smoke_tests.txt.
  • autofit_workspace_developer (searches/nss/: library copy is live and has diverged → superseded; root
    README.md archive definition).
  • autofit_assistant (skills/af_configure_search.md:3,21-26,103-105,117 — roster missing NSS/SMC/BFGS/MultiStart×4,
    stale "until the core wiki lands", per-search YAML claim, "HowToFit chapter 2" → chapter_1_introduction/tutorial_3_non_linear_search;
    wiki/core/concepts/non_linear_search.md:57; wiki/core/stack/autonerves.md:18; config/non_linear/README.md:6-8).
  • PyAutoBrain, two separate PRs: (B) agents/faculties/samplers/_samplers.py gap rule keyed on exported class names
    (parse autofit/__init__.py exports + _LAZY_ATTRS), class-vs-integration-stem match with _jax stripped, archive names
    that are also promoted excluded, searches_minimal variant suffixes (_mlp,_profile,_sweep) handled, plus a unit
    test; verify with bin/pyauto-brain samplers (there is no gaps verb). (C) skills/sampler_pipeline/{SKILL.md,reference.md}:
    Boundary → autolens_inference (SKILL.md:73,111-113, reference.md:98,113), drop the [nss] extra and
    config-mirroring lines (reference.md:142,146-149), add "Stage 5 — documentation" after Stage 4 using surveys/03 §3
    as the checklist, "Stages 3–5" at SKILL.md:88-91 and reference.md:184.

Out of scope (part 2)

PyAutoLens/PyAutoGalaxy/PyAutoCTI docs + CITATIONS ghosts, HowToLens/HowToGalaxy "LBFGS only", autogalaxy_workspace
duplicate __Start Point__ (guides/modeling/searches.py:336), autocti_workspace and al/ag assistant config claims.

Verification

GitHub docs legs (RTD floor is red on an autonerves pin — judge from the GitHub legs); grep -ri pyswarms\|multinest
over PyAutoFit docs/ and files/ empty; autofit_workspace smoke searches/{mcmc,nest,mle}.py + cookbook; the five new
workspace_test scripts run under PYAUTO_TEST_MODE=1; Brain tests + bin/pyauto-brain samplers tier-gaps section
reports only Drawer-style true gaps; repos_sync --check.

Witness

All 15 searches appear in docs/api/searches.rst; the Brain samplers tier-gaps section has no false positive for
MultiStart, SMC or BFGS and NSS appears once; grep -ri pyswarms over PyAutoFit docs/ returns nothing.

Activity

  1. Jammy2211 commented on Oct 8, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped to PR-open (2026-10-08). Nine PRs, nothing merged; tier judge, human /prm. Merge PyAutoFit#1669 first (library-first gate), then the rest; autofit_workspace_test#107 before #108.

    Repo PR Validation
    PyAutoFit (docs-only, pending-release) #1669 3066 passed / 2 skipped / 5 xfailed; Sphinx builds; no pyswarms/multinest left in docs/ or files/
    autofit_workspace autofit_workspace#168 smoke 4/4 (searches/{mcmc,nest,mle}, cookbooks/search) under test mode; notebooks regenerated
    autofit_workspace_test #107 (README), #108 (five scripts) scripts 5/5 in and out of test mode
    autofit_workspace_developer #29 searches/nss/ deleted via Gut (pre-delete 7cb97c6; Mind condemned.md entry autofit_workspace_developer/searches-nss)
    autofit_assistant #55 provenance 0 errors, API audit 0 missing; 1 test + the citation check fail identically on origin/main (pre-existing, not touched here)
    PyAutoBrain #502 (samplers gap rule), #503 (sampler_pipeline docs) samplers 10 passed; full suite 1273 passed

    Follow-ups filed in Mind: af.Drawer crashes under NullPaths (drawer/search.py:147, timer is None) — draft/bug/autofit/drawer_crashes_under_nullpaths_timer_none.md; the NSS docstring now renders a Sphinx block-quote warning (A0b). Part 2 (downstream ghost sweep) is filed after part 1 merges.

  2. Jammy2211 commented on Oct 8, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped and merged (2026-10-08). All nine PRs merged via human /prm: PyAutoFit#1669 (8bb5f6e, pending-release), autofit_workspace#168, autofit_workspace_test#107 and #108, autofit_workspace_developer#29 (human merge OK, no CI on that repo), autofit_assistant#55, PyAutoBrain#502 and #503. Two fixes landed during /prm: a blank line in the NSS docstring (the docs build started rendering it) and a neutral fixture directory name in the new Brain test (tenant firewall). Record: PyAutoMind complete/2026/10/search-ext-a0c-fit-repair.md. Next: A0c part 2 (downstream ghost sweep), A0b, A0a(ii).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions