Repository navigation
chore: clear the Nerves board's 79 workspace orphan keys (library-first) #441
Description
Overview
The Nerves board (PyAutoNerves#175) marks 14 workspace config files YELLOW for 79 "orphan" keys. An orphan is a key a workspace sets that no library's default YAML defines, which is not the same as a key nothing reads. A per-key classification, with the reading code checked, splits them into DELETE 51 / KEEP-WS 7 / PROMOTE 5 / FIX 16. The approved approach is library-first: fix the board (directory/file equivalence plus an autonerves allow-list), promote the 5 keys into PyAutoGalaxy, then delete the dead workspace keys. Target: 0 orphans.
Plan
- PyAutoNerves
scripts/board.py: when a library shipsX.yamland the workspace uses anX/directory (or the reverse), compare section by section, the same way autonerves merges them. This clears the 13 CTI false positives. Add aNERVES_OWNEDallow-list for theversion.*keys that autonerves itself reads, which clears the 7 KEEP-WS keys. - PyAutoGalaxy config: add the dPIE
ra/rsnotation label and label_format entries, and addareas_factor: Constant 0.5topriors/mesh/delaunay.yaml. The second change fixes "areas_factor Prior Missing" foraf.Model(al.mesh.Delaunay). Unit tests are included. - Ship the library half, then the workspaces, one PR per repo:
- Delete the 51 dead keys and the top-level
subplot_shape_to_figsize_factor. - Fix
fit_imaging {}:in autogalaxy_workspace. - Delete the 5 now-promoted keys from autolens_workspace.
- Delete the 51 dead keys and the top-level
- Witness: a local
python3 scripts/board.py --jsonreports 0 orphan files, and each workspace's start_here/smoke scripts still load their config.
Detailed implementation plan
Affected Repositories
- PyAutoNerves (library: board)
- PyAutoGalaxy (library: config promotion)
- autofit_workspace, autogalaxy_workspace, autolens_workspace, autocti_workspace (workspace deletes)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./organs/PyAutoNerves | main | clean |
| ./galaxy/PyAutoGalaxy | main | clean |
| ./fit/autofit_workspace | main | clean |
| ./galaxy/autogalaxy_workspace | main | clean |
| ./lens/autolens_workspace | main | clean |
| ./cti/autocti_workspace | main | clean |
Suggested branch: feature/workspace-config-cleanup, worktree ~/Code/PyAutoLabs-wt/workspace-config-cleanup/.
Parallel claim: PyAutoNerves is also claimed by eyes-organ-order (PyAutoMind#439). Its Nerves diff is only AGENTS.md, so it does not overlap with scripts/board.py. The human approved a parallel worktree on 2026-09-27.
Implementation Steps
- PyAutoNerves
scripts/board.py:- In
build_snapshot/diff_against_stack, handle file↔directory equivalence: a libraryX.yamlsection<f>corresponds to the workspace fileX/<f>.yaml, and the reverse. The semantics mirror autonervesRecursiveConfig. - Add the
NERVES_OWNEDallow-list forgeneral.yamlversion.{workspace_version, minimum_library_version, workspace_version_check, python_version_check}. These are excluded from orphans and noted "owned by autonerves". - Add tests: the equivalence clears a defined key, an owned key is not flagged, and an unknown key is still flagged.
- In
- PyAutoGalaxy:
autogalaxy/config/notation.yaml: addlabel.ra/rsandlabel_format.ra/rs, using the autolens_workspace values.autogalaxy/config/priors/mesh/delaunay.yaml: addareas_factoras a Constant 0.5, which matches theDelaunay(areas_factor=0.5)default.- Add unit tests: a Delaunay model has no missing prior, and the dPIE
ralabel is not the first-letter fallback.
- Ship the library half (
/ship_library). - Workspace edits:
- Delete the DELETE keys, the top-level
subplot_shape_to_figsize_factor(it equals the library default ofmat_plot.figure.*) and the PROMOTE keys in autolens_workspace. - Fix
fit_imaging: {}. - Leave the CTI and version keys alone.
- Before each delete, re-grep the workspace's scripts and notebooks. Mirror a change into notebooks or markdown only where they carry config snippets. Remove any YAML comment that describes a deleted key.
- Delete the DELETE keys, the top-level
- Ship the workspaces (
/ship_workspace), then/prm, then dispatchnerves_board.ymland re-check the published board.
Per-key verdicts (79)
| File | Key(s) | Verdict | Reason |
|---|---|---|---|
| autofit_workspace/config/general.yaml | hpc.live_visual_update, hpc.quick_update_background | DELETE | PyAutoFit reads updates.* (abstract_search.py:240-251), not hpc.* |
| autofit_workspace/config/general.yaml | test.check_preloads, test.preloads_check_threshold | DELETE | preloads check removed; nothing reads them |
| autofit_workspace/config/general.yaml | version.minimum_library_version, version.workspace_version_check | KEEP-WS | read by autonerves (workspace.py / init.py); board allow-list |
| autofit_workspace/config/visualize/plots_search.yaml | mle.corner_cornerpy | DELETE | not read |
| autogalaxy_workspace/config/general.yaml | hpc.live_visual_update, hpc.quick_update_background | DELETE | read from updates.* |
| autogalaxy_workspace/config/general.yaml | test.disable_positions_lh_inversion_check | DELETE | only PyAutoLens reads it |
| autogalaxy_workspace/config/general.yaml | version.minimum_library_version, version.workspace_version_check | KEEP-WS | autonerves; allow-list |
| autogalaxy_workspace/config/notation.yaml | label.contribution_factor, label_format.contribution_factor | DELETE | param no longer exists |
| autogalaxy_workspace/config/visualize/general.yaml | subplot_shape.{1,2,4,6,9,12,16,20,36,49,64,81,100} (13) | DELETE | table not read |
| autogalaxy_workspace/config/visualize/general.yaml | subplot_shape_to_figsize_factor | FIX (delete) | top level is never read; PyAutoArray reads mat_plot.figure.*, which equals the library default |
| autogalaxy_workspace/config/visualize/plots.yaml:30 | fit_imaging {}: |
FIX | malformed key → fit_imaging: {} |
| autolens_workspace/config/general.yaml | hpc.live_visual_update, hpc.quick_update_background | DELETE | read from updates.* |
| autolens_workspace/config/general.yaml | version.minimum_library_version, version.python_version_check, version.workspace_version_check | KEEP-WS | autonerves; allow-list |
| autolens_workspace/config/notation.yaml | label.contribution_factor, label_format.contribution_factor, label.sigma_scale | DELETE | params no longer exist |
| autolens_workspace/config/notation.yaml | superscript.hyperbackgroundnoise, hypergalaxy, hyperimagesky | DELETE | hyper classes removed |
| autolens_workspace/config/notation.yaml | label.ra, label.rs, label_format.ra, label_format.rs | PROMOTE | dPIE params read by the library; add to PyAutoGalaxy notation.yaml, then delete here |
| autolens_workspace/config/priors/mesh/delaunay.yaml | Delaunay.areas_factor | PROMOTE | the PyAutoGalaxy prior file is a bare Delaunay:, so "areas_factor Prior Missing" |
| autolens_workspace/config/visualize/general.yaml | subplot_shape.* (13) | DELETE | not read |
| autolens_workspace/config/visualize/general.yaml | subplot_shape_to_figsize_factor | FIX (delete) | as above |
| autocti_workspace/config/general.yaml | analysis.n_cores, analysis.preload_attempts, profiling.should_profile, test.check_preloads, test.disable_positions_lh_inversion_check, test.preloads_check_threshold | DELETE | not read by PyAutoCTI |
| autocti_workspace/config/visualize/general.yaml | general.subplot_ascending_fpr, general.symmetric_cmap_value | FIX (board) | false positive: PyAutoCTI ships a single visualize.yaml and autonerves merges the directory form |
| autocti_workspace/config/visualize/plots.yaml | combined_only, dataset.{data,data_binned,data_logy,fpr_non_uniformity,subplot_dataset_regions}, fit.{data_logy,fits_fit,residual_map_logy,subplot_fit_regions}, subplot_format (11) | FIX (board) | same false positive |
| autocti_workspace/config/visualize/plots_search.yaml | mle.corner_cornerpy | DELETE | not read |
Totals: DELETE 51, KEEP-WS 7, PROMOTE 5, FIX 16 (13 CTI board false positives, 2 × subplot_shape_to_figsize_factor, 1 fit_imaging typo).
Out of scope: library-side unused keys (human-review-gated, NOT removed here)
- PyAutoLens:
config/non_linear.yaml: the dead 57-keynest.DynestyStatic/nest.DynestyDynamicblock.general.yaml:output.fit_dill.
- PyAutoFit:
general.yaml:output.log_level,output.log_to_file,output.log_file,output.search_internalandprofiling.repeats. These may be read dynamically.non_linear/GridSearch.yaml:parallel,parallel.number_of_coresandparallel.step_size.
- PyAutoArray
general.yaml:numba.use_numba. - PyAutoGalaxy
general.yaml:adapt.adapt_noise_limit. - PyAutoCTI
general.yaml:fits.flip_for_ds9,hpc.iterations_per_update,model.ignore_prior_limits,output.log_file,output.log_levelandoutput.log_to_file.
Remove any of these only on explicit human confirmation, as a separate library prompt.
Key Files
PyAutoNerves/scripts/board.py: board snapshot, stack diff, orphansPyAutoNerves/autonerves/directory_config.py,autonerves/conf.py: the merge semantics to mirrorPyAutoGalaxy/autogalaxy/config/notation.yaml,autogalaxy/config/priors/mesh/delaunay.yaml- The workspace
config/trees listed in the table
Original Prompt
Click to expand starting prompt
Workspace config cleanup: remove orphan config keys the Nerves board flags
Type: maintenance
Target: workspaces
Repos:
- autofit_workspace
- autogalaxy_workspace
- autolens_workspace
- autocti_workspace
Difficulty: small
Autonomy: supervised
Priority: normal
Status: formalised
Consequence: glance
Witness: after the change, a local Nerves board render (python3 scripts/board.py --jsonin PyAutoNerves) reports 0 workspace files with orphan keys (down from 14 files / 79 keys), except any key this task keeps with a recorded reason; each touched workspace's start_here/smoke scripts still load their config with no config-key error.
Review-minutes: 3
Unattended: ready
Filed: 2026-09-27
Epic: organ-cockpit
Lane: local-dev
Why
The Nerves board (PyAutoNerves#175, shipped 2026-09-27 in the cockpit-followups bundle) now lists
workspace config keys that no library in the stack defines. On the 2026-09-27 render
(https://pyautolabs.github.io/PyAutoNerves/) the board is YELLOW with 14 workspace config files
carrying 79 orphan keys. They are dead settings that mislead users who edit them expecting an effect.
The orphan keys (workspace side — in scope)
- autofit_workspace/config/general.yaml (L14, 6): hpc.live_visual_update, hpc.quick_update_background, test.check_preloads, test.preloads_check_threshold, version.minimum_library_version, version.workspace_version_check
- autofit_workspace/config/visualize/plots_search.yaml (L6, 1): mle.corner_cornerpy
- autogalaxy_workspace/config/general.yaml (L26, 5): hpc.live_visual_update, hpc.quick_update_background, test.disable_positions_lh_inversion_check, version.minimum_library_version, version.workspace_version_check
- autogalaxy_workspace/config/notation.yaml (L24, 2): label.label.contribution_factor, label_format.format.contribution_factor
- autogalaxy_workspace/config/visualize/general.yaml (L14, 14): subplot_shape.1, subplot_shape.100, subplot_shape.12, subplot_shape.16, subplot_shape.2, subplot_shape.20, subplot_shape.36, subplot_shape.4, subplot_shape.49, subplot_shape.6, subplot_shape.64, subplot_shape.81, subplot_shape.9, subplot_shape_to_figsize_factor
- autogalaxy_workspace/config/visualize/plots.yaml (L30, 1): fit_imaging {}
- autolens_workspace/config/general.yaml (L25, 5): hpc.live_visual_update, hpc.quick_update_background, version.minimum_library_version, version.python_version_check, version.workspace_version_check
- autolens_workspace/config/notation.yaml (L24, 10): label.label.contribution_factor, label.label.ra, label.label.rs, label.label.sigma_scale, label.superscript.hyperbackgroundnoise, label.superscript.hypergalaxy, label.superscript.hyperimagesky, label_format.format.contribution_factor, label_format.format.ra, label_format.format.rs
- autolens_workspace/config/priors/mesh/delaunay.yaml (L2, 1): delaunay.areas_factor
- autolens_workspace/config/visualize/general.yaml (L14, 14): subplot_shape.1, subplot_shape.100, subplot_shape.12, subplot_shape.16, subplot_shape.2, subplot_shape.20, subplot_shape.36, subplot_shape.4, subplot_shape.49, subplot_shape.6, subplot_shape.64, subplot_shape.81, subplot_shape.9, subplot_shape_to_figsize_factor
- autocti_workspace/config/general.yaml (L2, 6): analysis.n_cores, analysis.preload_attempts, profiling.should_profile, test.check_preloads, test.disable_positions_lh_inversion_check, test.preloads_check_threshold
- autocti_workspace/config/visualize/general.yaml (L5, 2): general.subplot_ascending_fpr, general.symmetric_cmap_value
- autocti_workspace/config/visualize/plots.yaml (L1, 11): combined_only, dataset.data, dataset.data_binned, dataset.data_logy, dataset.fpr_non_uniformity, dataset.subplot_dataset_regions, fit.data_logy, fit.fits_fit, fit.residual_map_logy, fit.subplot_fit_regions, subplot_format
- autocti_workspace/config/visualize/plots_search.yaml (L6, 1): mle.corner_cornerpy
Before deleting each key, grep the workspace itself (scripts, notebooks, start_here.py, any
workspace-local helper) for a reader: the board only checks library code, so a key read by the
workspace's own code (possibly the version.* keys via the version handshake in autonerves, or
subplot_shape.* read by an integer lookup) is not an orphan. Keep any such key and record why.
Mirror the removals into the matching notebooks/markdown only if they carry config snippets.
The typo
autogalaxy_workspace/config/visualize/plots.yaml:30 has fit_imaging {}: — a malformed key (the
{} looks like a leftover flow-mapping), so the fit_imaging section it was meant to open is not
read. Fix it to the intended fit_imaging: section (compare with autolens_workspace's plots.yaml),
not just delete it.
Separate, human-review-gated: library-side unused keys (NOT in scope to remove without review)
The same scan flags library config keys no library code reads. These need a human to confirm
before removal, because a key may be read dynamically (string-built lookups the scan cannot see):
- PyAutoLens
config/non_linear.yaml: a dead 57-keynest.DynestyStatic/nest.DynestyDynamic
block (Dynesty searches live in PyAutoFit's config; PyAutoLens's copy is not read). - PyAutoLens
general.yaml:output.fit_dill. - PyAutoFit
general.yaml:output.log_level,output.log_to_file,output.log_file,
output.search_internal,profiling.repeats(may be read dynamically — needs review);
non_linear/GridSearch.yaml:parallel,parallel.number_of_cores,parallel.step_size. - PyAutoArray
general.yaml:numba.use_numba; PyAutoGalaxygeneral.yaml:adapt.adapt_noise_limit. - PyAutoCTI
general.yaml:fits.flip_for_ds9,hpc.iterations_per_update,model.ignore_prior_limits,
output.log_file,output.log_level,output.log_to_file.
Present this list to the human; remove only what they confirm, as a library change shipped before
the workspace edits (library-first), or re-file it as its own library prompt.
Related prompts
draft/maintenance/workspaces/config_key_mirror_drift.md— the opposite direction (library keys
missing from workspace configs); do both in one pass if picked up together, and do not re-add a
key this task removes.draft/maintenance/workspaces/sync_remaining_workspace_config_priors_copies.md.
Activity
Jammy2211 commentedon Sep 27, 2026
Parallel claim: PyAutoNerves is also claimed by
eyes-organ-order(PyAutoMind#439). That task's Nerves diff is onlyAGENTS.md. This task touches onlyscripts/board.pyand its tests, so the file sets don't overlap. The human approved a parallel worktree (~/Code/PyAutoLabs-wt/workspace-config-cleanup) on 2026-09-27.prompt: file issue for workspace-config-cleanup (#441); register libr…
prompt: workspace-config-cleanup claims PyAutoLens (#441 scope change)
Jammy2211 commentedon Sep 27, 2026
Scope change (human-approved 2026-09-27):
feature/workspace-config-cleanup). ConfiguringDelaunay.areas_factorin PyAutoGalaxy turns two model-figure acceptance tests red:test_autolens/model_figure/test_acceptance_lens.pytest__mge__areas_factor_is_missing_on_the_meshandtest__mge__counts. They used that parameter as the real-class witness of the "missing" state. The witness moves to a class that is still unconfigured.autogalaxy_workspace/config/priors/mesh/delaunay.yaml, a bareDelaunay:file. After the promotion it is the board's only new orphan. It has no runtime effect, becauseareas_factoralready resolves to 0.5 from the library.Jammy2211 commentedon Sep 27, 2026
Library PR Created
Delaunay.areas_factorconfigured asConstant0.5; dPIEra/rsnotation labels. Merge first.Delaunay.areas_factor(needs Galaxy#630; merge together with / after it).NERVES_OWNEDallow-list (orphans 14/79 → 12/59).API changes: no Python API change. Config-default change in PyAutoGalaxy: Delaunay models now get
areas_factor=0.5Constant from library config (previously needed a workspace prior or reported Missing). Lens/Nerves: internal/test-only.Heart YELLOW (85) acknowledged by human: manifest drift (hub organism blurb 7, organism-map blocks 1, workspace checkouts 1); release validation stale (PyAutoFit, PyAutoArray, PyAutoGalaxy, PyAutoLens).
Next: workspace PRs (autofit_workspace, autogalaxy_workspace, autolens_workspace, autocti_workspace) —
/ship_workspacein the same task.prompt: ship workspace-config-cleanup libraries (Galaxy#630, Lens#751…
prompt: ship workspace-config-cleanup workspaces (autofit#164, autoga…
Jammy2211 commentedon Sep 27, 2026
Workspace PRs Created — 2026-09-27
Library PRs
Workspace PRs
Merge order
priors/mesh/delaunay.yamland the dPIEra/rsnotation, relying on the new PyAutoGalaxy defaults. Recorded asrelease-gate: PyAutoGalaxyin active.md.Evidence
pyauto-heart smoke --root <wt>): autofit 10/10, autogalaxy 16/16, autolens 43/43. autocti: smoke env not buildable locally (no GSL headers for arcticpy); fallback.github/scripts/run_smoke.pybase env 3/3 — relying on PR CI.build_util.py_to_notebook+inject_colab_setup); output byte-identical to the committed notebooks, no local paths; no markdown mirror exists for these guides.Task stays open; merge + close-out via
/prm.prompt: workspace-config-cleanup 6/7 merged, autolens_workspace#578 h…
Jammy2211 commentedon Sep 27, 2026
/prm — 6 of 7 merged (2026-09-27)
All legs green, mergeable CLEAN, Heart freeze clear. Merged in library-first order:
cbd89ced1e6372fd0b6e7c78aa7361df7fd1953d7aa79ac6Held: autolens_workspace#578 — release-gate PyAutoGalaxy (it drops the notation
ra/rsand Delaunayareas_factorentries that only the new PyAutoGalaxy defaults supply). Merge with/prmonce PyAutoGalaxy with #630 is on PyPI; full close-out (records, this issue, worktree) follows then.nerves_board.ymlre-dispatched. Until #578 merges, the published board will still list autolens_workspace's dead keys (the 21 DELETE keys + top-levelsubplot_shape_to_figsize_factor); the 5 PROMOTE keys stop counting as orphans now that PyAutoGalaxy defines them. Every other workspace should read 0.Jammy2211 commentedon Sep 30, 2026
Shipped. All 7 PRs merged: PyAutoGalaxy#630 (cbd89ced), PyAutoLens#751 (1e6372fd), PyAutoNerves#176 (0b6e7c78), autofit_workspace#164 (aa7361df), autogalaxy_workspace#250 (7fd1953d), autocti_workspace#34 (7aa79ac6) on 2026-09-27, and autolens_workspace#578 (bf77575d) on 2026-09-30. Library PRs remain pending release (PyAutoGalaxy, PyAutoLens, PyAutoNerves).
prompt: ecosystem-layers PR #441 awaiting merge; record RED override
complete: ecosystem-layers research (Brain #441)