From dfe0e197e3c8f07c121e2a501f48bebe657491aa Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 18 Aug 2026 12:20:42 +0000 Subject: [PATCH 1/3] Register the HowTo setup_notebook audit and record its result MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The prompt advances draft/bug/howto/ -> active/: the audit has been run and the fix pushed as branches in six repos, so it is in flight rather than not started. The audit itself is recorded in the prompt file. Every .py under scripts/ (plus each HowTo repo's root start_here.py) was checked across the three HowTo repos and all five user-facing workspaces; 39 scripts were missing the setup_notebook() line and all 39 are fixed, with the matching generated notebook updated. autocti_workspace was already clean. autoreduce_workspace and the *_workspace_test / *_workspace_developer repos are out of scope and the prompt says why: none of them generates notebooks, so the convention does not apply there yet. No issue was filed and no PR opened — the registry entry says so and names the branch and per-repo commit, so whoever picks it up knows exactly what is and is not on main. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0126SmBwHMzP4okcjhjPoLFZ --- active.md | 36 +++++++++ active/missing_setup_notebook_audit.md | 76 +++++++++++++++++++ dashboard.md | 11 ++- .../bug/howto/missing_setup_notebook_audit.md | 28 ------- 4 files changed, 117 insertions(+), 34 deletions(-) create mode 100644 active/missing_setup_notebook_audit.md delete mode 100644 draft/bug/howto/missing_setup_notebook_audit.md diff --git a/active.md b/active.md index 213a55ea..272c0082 100644 --- a/active.md +++ b/active.md @@ -113,3 +113,39 @@ - PyAutoGalaxy: feature/version-stamp-sync-guards - PyAutoLens: feature/version-stamp-sync-guards - PyAutoHands: feature/version-stamp-sync-guards + +## howto-setup-notebook-audit +- issue: (none — cloud session, no issue filed) +- prompt: active/missing_setup_notebook_audit.md +- session: cloud (Claude Code on the web), no local worktree +- status: branches pushed, no PRs opened +- what that means: six repos carry `claude/howto-setup-notebook-audit-dm2j9e`, each one commit on + top of the main it was cloned from. Nothing is merged and no PR exists — opening them (and the + pending-release gate, if the workspace legs need one) is the next step. +- repos-none-claimed: this entry claims NO repos as 2-space ` - Repo` bullets — the work ran + in a cloud container with no local worktree, so `worktree_check_conflict` must not read it as + a live claim. The branches are listed as prose below. +- branches: HowToFit `fa6c03f` / HowToGalaxy `9aa173c` / HowToLens `1ff2fa0` / + autofit_workspace `6180480` / autogalaxy_workspace `d3fb5b5` / autolens_workspace `181694c`, + all on `claude/howto-setup-notebook-audit-dm2j9e`. +- audit result: every `.py` under `scripts/` (plus each HowTo's root `start_here.py`) was checked + in all three HowTo repos and all five user-facing workspaces. 39 scripts were missing the + `# from auto* import setup_notebook; setup_notebook()` line — HowToFit 3, HowToGalaxy 2, + HowToLens 6, autofit_workspace 1, autogalaxy_workspace 5, autolens_workspace 22. All 39 fixed, + with the matching generated notebook updated to the uncommented form PyAutoHands emits. +- clean: autocti_workspace (79 scripts, 0 missing). +- out of scope, deliberately: autoreduce_workspace (30 scripts, all missing) has NO `notebooks/` + directory, so the line is not part of its convention yet — same for every `*_workspace_test` + and `*_workspace_developer` repo (checked by tree listing; none generates notebooks). If + autoreduce_workspace ever starts generating notebooks, its 30 scripts need this sweep. +- placement: after the module docstring, or after the `from auto* import jax_wrapper` line where + a script has one (workspace convention). One outlier, autolens_workspace + `scripts/guides/units/mass_to_light_ratio_units.py`, opens on imports rather than a docstring — + the line went at the top of the file, still ahead of every import. +- not regenerated by PyAutoHands: the notebooks were patched by hand to the exact shape the + generator emits (verified against already-correct sibling pairs, and `json.dumps(nb, indent=1)` + round-trips byte-identically). A real `generate.py` run should be a no-op on these; confirm that + before merging. +- adjacent, NOT done here: `draft/maintenance/workspaces/notebook_setup_notebook_drift_siblings.md` + is the separate regeneration-drift sweep (notebooks carrying the COMMENTED form). Different + failure, different task — untouched. diff --git a/active/missing_setup_notebook_audit.md b/active/missing_setup_notebook_audit.md new file mode 100644 index 00000000..ba3e80e0 --- /dev/null +++ b/active/missing_setup_notebook_audit.md @@ -0,0 +1,76 @@ +# Audit HowTo tutorials for missing setup_notebook() line + +Type: bug +Target: HowToFit +Repos: +- @HowToFit +- @HowToGalaxy +- @HowToLens +- @autofit_workspace +- @autogalaxy_workspace +- @autolens_workspace +Difficulty: small +Autonomy: safe +Priority: normal +Status: formalised + +Discovered during the batch-2b markdown rollout ([[markdown-example-renderings]]). +Several HowTo tutorials are missing the standard +`# from autoconf import setup_notebook; setup_notebook()` line that every other +workspace/tutorial script carries (it chdir's to the workspace root + enables +inline plotting). Without it, a tutorial that loads data via a relative path +(or runs a simulator subprocess) FAILS when executed by nbconvert (which runs +with CWD = the notebook's own directory), and only works interactively if the +user happens to launch jupyter from the repo root. + +Confirmed missing in chapter_1_introduction alone: HowToFit start_here.py + +tutorial_1_models.py; HowToGalaxy tutorial_4_methods.py; (HowToGalaxy +tutorial_3_fitting.py + HowToLens tutorial_7_fitting.py were fixed in batch 2b's +PRs since they blocked rendering). The 3 still-missing pass today only because +they don't load data by relative path. + +Fix: audit ALL chapters of all three HowTo repos (and re-check the workspaces) +for scripts lacking the setup_notebook line; add it right after the opening +docstring, matching the sibling convention; regenerate the affected notebooks. +Low-risk boilerplate consistency fix. + +## Audit result (2026-08-18) + +Every `.py` under `scripts/` — plus each HowTo repo's root `start_here.py` — was checked in the +three HowTo repos and all five user-facing workspaces. 39 scripts were missing the line: + +| Repo | Missing | Notes | +|---|---|---| +| HowToFit | 3 | `chapter_1_introduction/{start_here,tutorial_1_models}.py`, root `start_here.py` | +| HowToGalaxy | 2 | `chapter_2_modeling/tutorial_8_need_for_speed.py`, root `start_here.py` | +| HowToLens | 6 | `chapter_2_lens_modeling/{tutorial_8_need_for_speed,tutorial_11_slam}.py`, `chapter_3_pixelizations/tutorial_9_model_fit.py`, `simulator/{lens_x2,lens_x3}.py`, root `start_here.py` | +| autofit_workspace | 1 | `overview/overview_3_statistical_methods.py` | +| autogalaxy_workspace | 5 | under `guides/`, `imaging/data_preparation/`, `interferometer/` | +| autolens_workspace | 22 | mostly `cluster/*`, `guides/*` and feature `simulator.py` scripts | +| autocti_workspace | 0 | clean | + +`tutorial_4_methods.py` in HowToGalaxy, named in the original report, already carries the line — +it was fixed before this audit ran. + +The omissions are not a deliberate per-subtree policy: they sit alongside siblings in the same +directory that do carry the line (e.g. `autolens_workspace/scripts/cluster/` has 6 without and 5 +with). + +The two genuinely load-bearing cases are `HowToLens/scripts/simulator/lens_x{2,3}.py`, which write +to a relative `dataset/` path; the rest are prose-only or import-only scripts that pass today for +the reason the report gives. + +**Out of scope, deliberately:** `autoreduce_workspace` has 30 scripts and none carry the line, but +it has no `notebooks/` directory — the convention does not apply there yet. Same for every +`*_workspace_test` and `*_workspace_developer` repo (checked by tree listing; none generates +notebooks). + +**Placement:** after the module docstring, or after the `from auto* import jax_wrapper` line where +a script has one. One outlier — `autolens_workspace/scripts/guides/units/mass_to_light_ratio_units.py` +opens on imports rather than a docstring, so the line went at the top of the file. + +**Notebooks:** patched by hand to the exact shape PyAutoHands emits (uncommented, same cell, +`json.dumps(nb, indent=1)` round-trips byte-identically), because the generator is not available in +a cloud session. A real `generate.py` run should be a no-op on these — confirm before merging. + +Branch `claude/howto-setup-notebook-audit-dm2j9e` in all six repos. Not merged, no PRs opened. diff --git a/dashboard.md b/dashboard.md index 882b4d9f..ddb46dd6 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,10 +8,10 @@ Tasks only — the organism's health lives with the Heart (`/health`), not here. | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 4 | +| [In flight](#in-flight) (`active/`) | 5 | | [Parked](#parked) (`parked.md`) | 1 | | [Planned](#planned) (`planned.md`) | 7 | -| [Backlog](#backlog) (`draft/`) | 139 | +| [Backlog](#backlog) (`draft/`) | 138 | Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3Aissue+is%3Aopen&type=issues) · [open pull requests](https://github.com/search?q=org%3APyAutoLabs+is%3Apr+is%3Aopen&type=prs) @@ -34,7 +34,6 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A **Quick wins** (small enough, and safe enough to run unattended) -- [Audit HowTo tutorials for missing setup_notebook() line](draft/bug/howto/missing_setup_notebook_audit.md) — howto · small · safe · normal - [PyAutoFit CLI-noise batch: unclosed search.log handler + four small warning](draft/maintenance/pyautofit/cli_noise_pyautofit_batch.md) — pyautofit · small · safe · normal - [Tenant firewall: release_run.py carries an unlisted 'PyAutoLabs' instance fact](draft/bug/pyautoheart/tenant_firewall_release_run_instance_fact.md) — pyautoheart · small · safe · normal - [Silence the three autonerves-rooted CLI-noise sources (fits leak, pytest collection,](draft/maintenance/pyautonerves/cli_noise_autonerves_batch.md) — pyautonerves · small · safe · normal @@ -49,6 +48,7 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A Issued — each has an open GitHub issue and usually a branch. The full record for each is in [`active.md`](active.md). - [Commit the inference-methods programme plan + knowledge ledger into autolens_profiling](active/inference_programme_ledger.md) — [issue #134](https://github.com/PyAutoLabs/autolens_profiling/issues/134) — pr-open (https://github.com/PyAutoLabs/autolens_profiling/pull/135) +- [Audit HowTo tutorials for missing setup_notebook() line](active/missing_setup_notebook_audit.md) — branches pushed, no PRs opened - [PyAutoReduce validation: slacs1430+4105 ACS reduction vs trusted legacy dataset](active/pyautoreduce_slacs1430_acs_comparison.md) - [Reconstructing a stored sample raises through `ignore_assertions=True`](active/to_instance_guard_gap.md) — [issue #1486](https://github.com/PyAutoLabs/PyAutoFit/issues/1486) — library-dev — WORKSPACE HALF SHIPPED; the PyAutoFit hardening (#1486) is what remains - [Version-stamp sync to 2026.8.17.1 + release-sed guards](active/version_stamp_sync_and_release_sed_guards.md) — [issue #235](https://github.com/PyAutoLabs/PyAutoHands/issues/235) — pr-open (https://github.com/PyAutoLabs/PyAutoLens/pull/700) @@ -83,10 +83,10 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**139** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). +**138** filed prompts, not started. Each section is sorted most-pickable first (priority, then size).
-bug — 40 +bug — 39 - [pre_build stages untracked files, publishing uncommitted human work](draft/bug/pyautohands/pre_build_stages_untracked_wip.md) — pyautohands · small · supervised · high - [UniformPrior bounds are not enforced in the objective on the](draft/bug/autofit/uniform_prior_bounds_unenforced_on_numpy_path.md) — autofit · medium · human-required · high @@ -109,7 +109,6 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. - [TEST_MODE bypass crashes on ordered-parameter assertion ties](draft/bug/autofit/test_mode_bypass_ordered_assertion_ties.md) — autofit · small · supervised · normal - [point.py JAX-vmap parity assert is non-deterministic under the smoke env](draft/bug/autolens/point_jax_vmap_parity_nondeterministic.md) — autolens · small · supervised · normal - [Scripts derive geometry from a hardcoded pixel_scale while the dataset](draft/bug/autolens_workspace/script_local_pixel_scale_vs_dataset_pixel_scales.md) — autolens_workspace · small · supervised · normal -- [Audit HowTo tutorials for missing setup_notebook() line](draft/bug/howto/missing_setup_notebook_audit.md) — howto · small · safe · normal - [HowToGalaxy small API drifts: ellipse kwargs + plot_grid_lines (parked NEEDS_FIX)](draft/bug/howtogalaxy/small_api_drift_ellipse_and_plot_grid_lines.md) — howtogalaxy · small · supervised · normal - [`generate.py` deletes notebooks/ before rejecting an unknown project](draft/bug/pyautohands/generate_rejects_autocti_after_deleting_notebooks.md) — pyautohands · small · supervised · normal - [Tenant firewall: release_run.py carries an unlisted 'PyAutoLabs' instance fact](draft/bug/pyautoheart/tenant_firewall_release_run_instance_fact.md) — pyautoheart · small · safe · normal diff --git a/draft/bug/howto/missing_setup_notebook_audit.md b/draft/bug/howto/missing_setup_notebook_audit.md deleted file mode 100644 index d16b5165..00000000 --- a/draft/bug/howto/missing_setup_notebook_audit.md +++ /dev/null @@ -1,28 +0,0 @@ -# Audit HowTo tutorials for missing setup_notebook() line - -Type: bug -Target: HowToFit -Difficulty: small -Autonomy: safe -Priority: normal -Status: formalised - -Discovered during the batch-2b markdown rollout ([[markdown-example-renderings]]). -Several HowTo tutorials are missing the standard -`# from autoconf import setup_notebook; setup_notebook()` line that every other -workspace/tutorial script carries (it chdir's to the workspace root + enables -inline plotting). Without it, a tutorial that loads data via a relative path -(or runs a simulator subprocess) FAILS when executed by nbconvert (which runs -with CWD = the notebook's own directory), and only works interactively if the -user happens to launch jupyter from the repo root. - -Confirmed missing in chapter_1_introduction alone: HowToFit start_here.py + -tutorial_1_models.py; HowToGalaxy tutorial_4_methods.py; (HowToGalaxy -tutorial_3_fitting.py + HowToLens tutorial_7_fitting.py were fixed in batch 2b's -PRs since they blocked rendering). The 3 still-missing pass today only because -they don't load data by relative path. - -Fix: audit ALL chapters of all three HowTo repos (and re-check the workspaces) -for scripts lacking the setup_notebook line; add it right after the opening -docstring, matching the sibling convention; regenerate the affected notebooks. -Low-risk boilerplate consistency fix. From 03c2274772ea60d057c26ceaa1bdc4dec26fd5b2 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 18 Aug 2026 13:22:12 +0000 Subject: [PATCH 2/3] Record the six PR numbers on the setup_notebook audit entry The entry was written before the PRs existed and said so. They are open now, so it names them instead: HowToFit#45, HowToGalaxy#66, HowToLens#70, autofit_workspace#138, autogalaxy_workspace#211, autolens_workspace#485. Also records why the three workspace legs are not behind the library-first merge gate: setup_notebook is long-shipped in the released libraries and is already called by every other example script in those repos, so there is no unreleased API for a pending-release label to guard. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0126SmBwHMzP4okcjhjPoLFZ --- active.md | 13 ++++++++----- active/missing_setup_notebook_audit.md | 4 +++- dashboard.md | 2 +- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/active.md b/active.md index 272c0082..71d7e6ff 100644 --- a/active.md +++ b/active.md @@ -115,13 +115,16 @@ - PyAutoHands: feature/version-stamp-sync-guards ## howto-setup-notebook-audit -- issue: (none — cloud session, no issue filed) +- issue: (none — cloud session, no issue filed; the six PRs below carry the record) - prompt: active/missing_setup_notebook_audit.md - session: cloud (Claude Code on the web), no local worktree -- status: branches pushed, no PRs opened -- what that means: six repos carry `claude/howto-setup-notebook-audit-dm2j9e`, each one commit on - top of the main it was cloned from. Nothing is merged and no PR exists — opening them (and the - pending-release gate, if the workspace legs need one) is the next step. +- status: pr-open (six PRs, awaiting CI) +- PRs: HowToFit#45 / HowToGalaxy#66 / HowToLens#70 / autofit_workspace#138 / + autogalaxy_workspace#211 / autolens_workspace#485 — each one commit on the branch below, on + top of the main it was cloned from. +- no pending-release gate: `setup_notebook` is long-shipped in released autofit/autogalaxy/autolens + and is already called by every other example script in these repos, so the library-first merge + gate does not apply to the three workspace legs. - repos-none-claimed: this entry claims NO repos as 2-space ` - Repo` bullets — the work ran in a cloud container with no local worktree, so `worktree_check_conflict` must not read it as a live claim. The branches are listed as prose below. diff --git a/active/missing_setup_notebook_audit.md b/active/missing_setup_notebook_audit.md index ba3e80e0..9eae24f6 100644 --- a/active/missing_setup_notebook_audit.md +++ b/active/missing_setup_notebook_audit.md @@ -73,4 +73,6 @@ opens on imports rather than a docstring, so the line went at the top of the fil `json.dumps(nb, indent=1)` round-trips byte-identically), because the generator is not available in a cloud session. A real `generate.py` run should be a no-op on these — confirm before merging. -Branch `claude/howto-setup-notebook-audit-dm2j9e` in all six repos. Not merged, no PRs opened. +Branch `claude/howto-setup-notebook-audit-dm2j9e` in all six repos, one commit each: +HowToFit#45, HowToGalaxy#66, HowToLens#70, autofit_workspace#138, autogalaxy_workspace#211, +autolens_workspace#485. diff --git a/dashboard.md b/dashboard.md index ddb46dd6..b2ff5980 100644 --- a/dashboard.md +++ b/dashboard.md @@ -48,7 +48,7 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A Issued — each has an open GitHub issue and usually a branch. The full record for each is in [`active.md`](active.md). - [Commit the inference-methods programme plan + knowledge ledger into autolens_profiling](active/inference_programme_ledger.md) — [issue #134](https://github.com/PyAutoLabs/autolens_profiling/issues/134) — pr-open (https://github.com/PyAutoLabs/autolens_profiling/pull/135) -- [Audit HowTo tutorials for missing setup_notebook() line](active/missing_setup_notebook_audit.md) — branches pushed, no PRs opened +- [Audit HowTo tutorials for missing setup_notebook() line](active/missing_setup_notebook_audit.md) — pr-open (six PRs, awaiting CI) - [PyAutoReduce validation: slacs1430+4105 ACS reduction vs trusted legacy dataset](active/pyautoreduce_slacs1430_acs_comparison.md) - [Reconstructing a stored sample raises through `ignore_assertions=True`](active/to_instance_guard_gap.md) — [issue #1486](https://github.com/PyAutoLabs/PyAutoFit/issues/1486) — library-dev — WORKSPACE HALF SHIPPED; the PyAutoFit hardening (#1486) is what remains - [Version-stamp sync to 2026.8.17.1 + release-sed guards](active/version_stamp_sync_and_release_sed_guards.md) — [issue #235](https://github.com/PyAutoLabs/PyAutoHands/issues/235) — pr-open (https://github.com/PyAutoLabs/PyAutoLens/pull/700) From 02aeead04bddfaf2ef88a007e69bb3a9549dae2f Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 18 Aug 2026 13:35:08 +0000 Subject: [PATCH 3/3] Ship the setup_notebook audit: record it complete All six PRs merged 2026-08-18 with every check green, so the prompt advances active/ -> complete/2026/08/howto-setup-notebook-audit.md and its active.md entry is removed. lifecycle check + index check both pass; dashboard regenerated. The record keeps what the next person would otherwise re-derive: that the omissions were not a per-subtree policy (siblings in the same directory carry the line), that autoreduce_workspace and the *_test / *_developer repos are out of scope because none of them generates notebooks, the two placement forms plus the one import-first outlier, and the CRLF file in HowToFit that turns a naive mechanical sweep into a 547-line whitespace diff. Two findings are logged as unclaimed rather than silently fixed: fifteen start_here.ipynb files across the two big workspaces carry JSON formatting that differs from the generator's output, and the sibling regeneration-drift task stays open as a separate failure. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0126SmBwHMzP4okcjhjPoLFZ --- active.md | 39 ----- active/missing_setup_notebook_audit.md | 78 ---------- .../2026/08/howto-setup-notebook-audit.md | 134 ++++++++++++++++++ complete/index.md | 3 +- dashboard.md | 3 +- 5 files changed, 137 insertions(+), 120 deletions(-) delete mode 100644 active/missing_setup_notebook_audit.md create mode 100644 complete/2026/08/howto-setup-notebook-audit.md diff --git a/active.md b/active.md index 71d7e6ff..213a55ea 100644 --- a/active.md +++ b/active.md @@ -113,42 +113,3 @@ - PyAutoGalaxy: feature/version-stamp-sync-guards - PyAutoLens: feature/version-stamp-sync-guards - PyAutoHands: feature/version-stamp-sync-guards - -## howto-setup-notebook-audit -- issue: (none — cloud session, no issue filed; the six PRs below carry the record) -- prompt: active/missing_setup_notebook_audit.md -- session: cloud (Claude Code on the web), no local worktree -- status: pr-open (six PRs, awaiting CI) -- PRs: HowToFit#45 / HowToGalaxy#66 / HowToLens#70 / autofit_workspace#138 / - autogalaxy_workspace#211 / autolens_workspace#485 — each one commit on the branch below, on - top of the main it was cloned from. -- no pending-release gate: `setup_notebook` is long-shipped in released autofit/autogalaxy/autolens - and is already called by every other example script in these repos, so the library-first merge - gate does not apply to the three workspace legs. -- repos-none-claimed: this entry claims NO repos as 2-space ` - Repo` bullets — the work ran - in a cloud container with no local worktree, so `worktree_check_conflict` must not read it as - a live claim. The branches are listed as prose below. -- branches: HowToFit `fa6c03f` / HowToGalaxy `9aa173c` / HowToLens `1ff2fa0` / - autofit_workspace `6180480` / autogalaxy_workspace `d3fb5b5` / autolens_workspace `181694c`, - all on `claude/howto-setup-notebook-audit-dm2j9e`. -- audit result: every `.py` under `scripts/` (plus each HowTo's root `start_here.py`) was checked - in all three HowTo repos and all five user-facing workspaces. 39 scripts were missing the - `# from auto* import setup_notebook; setup_notebook()` line — HowToFit 3, HowToGalaxy 2, - HowToLens 6, autofit_workspace 1, autogalaxy_workspace 5, autolens_workspace 22. All 39 fixed, - with the matching generated notebook updated to the uncommented form PyAutoHands emits. -- clean: autocti_workspace (79 scripts, 0 missing). -- out of scope, deliberately: autoreduce_workspace (30 scripts, all missing) has NO `notebooks/` - directory, so the line is not part of its convention yet — same for every `*_workspace_test` - and `*_workspace_developer` repo (checked by tree listing; none generates notebooks). If - autoreduce_workspace ever starts generating notebooks, its 30 scripts need this sweep. -- placement: after the module docstring, or after the `from auto* import jax_wrapper` line where - a script has one (workspace convention). One outlier, autolens_workspace - `scripts/guides/units/mass_to_light_ratio_units.py`, opens on imports rather than a docstring — - the line went at the top of the file, still ahead of every import. -- not regenerated by PyAutoHands: the notebooks were patched by hand to the exact shape the - generator emits (verified against already-correct sibling pairs, and `json.dumps(nb, indent=1)` - round-trips byte-identically). A real `generate.py` run should be a no-op on these; confirm that - before merging. -- adjacent, NOT done here: `draft/maintenance/workspaces/notebook_setup_notebook_drift_siblings.md` - is the separate regeneration-drift sweep (notebooks carrying the COMMENTED form). Different - failure, different task — untouched. diff --git a/active/missing_setup_notebook_audit.md b/active/missing_setup_notebook_audit.md deleted file mode 100644 index 9eae24f6..00000000 --- a/active/missing_setup_notebook_audit.md +++ /dev/null @@ -1,78 +0,0 @@ -# Audit HowTo tutorials for missing setup_notebook() line - -Type: bug -Target: HowToFit -Repos: -- @HowToFit -- @HowToGalaxy -- @HowToLens -- @autofit_workspace -- @autogalaxy_workspace -- @autolens_workspace -Difficulty: small -Autonomy: safe -Priority: normal -Status: formalised - -Discovered during the batch-2b markdown rollout ([[markdown-example-renderings]]). -Several HowTo tutorials are missing the standard -`# from autoconf import setup_notebook; setup_notebook()` line that every other -workspace/tutorial script carries (it chdir's to the workspace root + enables -inline plotting). Without it, a tutorial that loads data via a relative path -(or runs a simulator subprocess) FAILS when executed by nbconvert (which runs -with CWD = the notebook's own directory), and only works interactively if the -user happens to launch jupyter from the repo root. - -Confirmed missing in chapter_1_introduction alone: HowToFit start_here.py + -tutorial_1_models.py; HowToGalaxy tutorial_4_methods.py; (HowToGalaxy -tutorial_3_fitting.py + HowToLens tutorial_7_fitting.py were fixed in batch 2b's -PRs since they blocked rendering). The 3 still-missing pass today only because -they don't load data by relative path. - -Fix: audit ALL chapters of all three HowTo repos (and re-check the workspaces) -for scripts lacking the setup_notebook line; add it right after the opening -docstring, matching the sibling convention; regenerate the affected notebooks. -Low-risk boilerplate consistency fix. - -## Audit result (2026-08-18) - -Every `.py` under `scripts/` — plus each HowTo repo's root `start_here.py` — was checked in the -three HowTo repos and all five user-facing workspaces. 39 scripts were missing the line: - -| Repo | Missing | Notes | -|---|---|---| -| HowToFit | 3 | `chapter_1_introduction/{start_here,tutorial_1_models}.py`, root `start_here.py` | -| HowToGalaxy | 2 | `chapter_2_modeling/tutorial_8_need_for_speed.py`, root `start_here.py` | -| HowToLens | 6 | `chapter_2_lens_modeling/{tutorial_8_need_for_speed,tutorial_11_slam}.py`, `chapter_3_pixelizations/tutorial_9_model_fit.py`, `simulator/{lens_x2,lens_x3}.py`, root `start_here.py` | -| autofit_workspace | 1 | `overview/overview_3_statistical_methods.py` | -| autogalaxy_workspace | 5 | under `guides/`, `imaging/data_preparation/`, `interferometer/` | -| autolens_workspace | 22 | mostly `cluster/*`, `guides/*` and feature `simulator.py` scripts | -| autocti_workspace | 0 | clean | - -`tutorial_4_methods.py` in HowToGalaxy, named in the original report, already carries the line — -it was fixed before this audit ran. - -The omissions are not a deliberate per-subtree policy: they sit alongside siblings in the same -directory that do carry the line (e.g. `autolens_workspace/scripts/cluster/` has 6 without and 5 -with). - -The two genuinely load-bearing cases are `HowToLens/scripts/simulator/lens_x{2,3}.py`, which write -to a relative `dataset/` path; the rest are prose-only or import-only scripts that pass today for -the reason the report gives. - -**Out of scope, deliberately:** `autoreduce_workspace` has 30 scripts and none carry the line, but -it has no `notebooks/` directory — the convention does not apply there yet. Same for every -`*_workspace_test` and `*_workspace_developer` repo (checked by tree listing; none generates -notebooks). - -**Placement:** after the module docstring, or after the `from auto* import jax_wrapper` line where -a script has one. One outlier — `autolens_workspace/scripts/guides/units/mass_to_light_ratio_units.py` -opens on imports rather than a docstring, so the line went at the top of the file. - -**Notebooks:** patched by hand to the exact shape PyAutoHands emits (uncommented, same cell, -`json.dumps(nb, indent=1)` round-trips byte-identically), because the generator is not available in -a cloud session. A real `generate.py` run should be a no-op on these — confirm before merging. - -Branch `claude/howto-setup-notebook-audit-dm2j9e` in all six repos, one commit each: -HowToFit#45, HowToGalaxy#66, HowToLens#70, autofit_workspace#138, autogalaxy_workspace#211, -autolens_workspace#485. diff --git a/complete/2026/08/howto-setup-notebook-audit.md b/complete/2026/08/howto-setup-notebook-audit.md new file mode 100644 index 00000000..d64b39bf --- /dev/null +++ b/complete/2026/08/howto-setup-notebook-audit.md @@ -0,0 +1,134 @@ +- issue: (none — run end-to-end from a cloud session; the six PRs are the record) +- completed: 2026-08-18 +- prs: HowToFit#45, HowToGalaxy#66, HowToLens#70, autofit_workspace#138, + autogalaxy_workspace#211, autolens_workspace#485 — all squash-merged 2026-08-18, + every check green on each. +- merge-commits: HowToFit `1a4e7132`, HowToGalaxy `d973bc57`, HowToLens `e0fbdf61`, + autofit_workspace `fa55c438`, autogalaxy_workspace `47227c79`, autolens_workspace `c5960f69`. +- summary: Full audit of the `# from auto* import setup_notebook; setup_notebook()` boilerplate + across every notebook-generating repo. 39 scripts were missing it — HowToFit 3, HowToGalaxy 2, + HowToLens 6, autofit_workspace 1, autogalaxy_workspace 5, autolens_workspace 22 — and all 39 + were fixed with their generated notebooks updated to the uncommented form PyAutoHands emits. + autocti_workspace was already clean (79 scripts, 0 missing). The line chdir's to the workspace + root and enables inline plotting; without it a notebook that loads data by a relative path + fails under nbconvert, which runs with CWD set to the notebook's own directory. +- the only live breakage: `HowToLens/scripts/simulator/lens_x{2,3}.py`, which resolve + `dataset_path = Path("dataset", ...)` against CWD and write FITS output there. Everything else + in the sweep is prose-only or import-only and passed today for exactly the reason the original + report predicted — they never load data by relative path. +- the omissions were not policy: they sit alongside siblings in the SAME directory that do carry + the line (`autolens_workspace/scripts/cluster/` has 6 without and 5 with). Anyone re-auditing + should not read a missing line as a deliberate per-subtree exemption. +- placement rule (two forms, both in the tree already): after the module docstring, OR after the + `from auto* import jax_wrapper` line where a script has one — jax_wrapper must set the JAX + environment before anything else. One outlier, + `autolens_workspace/scripts/guides/units/mass_to_light_ratio_units.py`, opens on imports rather + than a docstring; the line went at the top of the file, still ahead of every import. +- SCOPE BOUNDARY worth keeping: `autoreduce_workspace` has 30 scripts and NONE carry the line, + which looks alarming until you notice it has no `notebooks/` directory — nothing generates + notebooks there, so the convention does not apply yet. Same for every `*_workspace_test` and + `*_workspace_developer` repo (verified by tree listing, not assumption). If autoreduce_workspace + ever starts generating notebooks, those 30 scripts need this sweep. +- notebooks were NOT regenerated by PyAutoHands: it is not available in a cloud session, so the + `.ipynb` files were patched directly to the exact shape the generator emits. Verified three ways + before pushing — the edit reproduces already-correct sibling pairs byte-for-byte, + `json.dumps(nb, indent=1)` round-trips every touched file unchanged, and every `.py` still + parses. A real `generate.py` run should be a no-op on these; if it is not, that is the thing to + look at first. +- CRLF trap: `HowToFit/scripts/chapter_1_introduction/tutorial_1_models.py` is committed with CRLF + line endings while everything around it is LF. A naive read/split/write rewrote all 547 lines as + a whitespace-only diff. The fix preserves the file's existing terminator. Anything doing a + mechanical sweep across these repos will hit this file. +- pre-existing drift found, deliberately NOT touched: nine `start_here.ipynb` files in + autolens_workspace and six in autogalaxy_workspace are stored with a JSON formatting that + differs from the generator's `indent=1` output. Flagged in both PR bodies. Unrelated to this + task, real, and unclaimed. +- no pending-release gate applied to the three workspace legs: `setup_notebook` is long-shipped in + the released libraries and is already called by every other example script in those repos, so + there is no unreleased API for the library-first merge gate to guard. +- adjacent but distinct: `draft/maintenance/workspaces/notebook_setup_notebook_drift_siblings.md` + covers notebooks that carry the COMMENTED form and need regenerating. Different failure, still + open, untouched here. +- `HowToGalaxy/chapter_1_introduction/tutorial_4_methods.py`, named in the original report as + missing, already carried the line — fixed before this audit ran. + +## Original prompt + +# Audit HowTo tutorials for missing setup_notebook() line + +Type: bug +Target: HowToFit +Repos: +- @HowToFit +- @HowToGalaxy +- @HowToLens +- @autofit_workspace +- @autogalaxy_workspace +- @autolens_workspace +Difficulty: small +Autonomy: safe +Priority: normal +Status: formalised + +Discovered during the batch-2b markdown rollout ([[markdown-example-renderings]]). +Several HowTo tutorials are missing the standard +`# from autoconf import setup_notebook; setup_notebook()` line that every other +workspace/tutorial script carries (it chdir's to the workspace root + enables +inline plotting). Without it, a tutorial that loads data via a relative path +(or runs a simulator subprocess) FAILS when executed by nbconvert (which runs +with CWD = the notebook's own directory), and only works interactively if the +user happens to launch jupyter from the repo root. + +Confirmed missing in chapter_1_introduction alone: HowToFit start_here.py + +tutorial_1_models.py; HowToGalaxy tutorial_4_methods.py; (HowToGalaxy +tutorial_3_fitting.py + HowToLens tutorial_7_fitting.py were fixed in batch 2b's +PRs since they blocked rendering). The 3 still-missing pass today only because +they don't load data by relative path. + +Fix: audit ALL chapters of all three HowTo repos (and re-check the workspaces) +for scripts lacking the setup_notebook line; add it right after the opening +docstring, matching the sibling convention; regenerate the affected notebooks. +Low-risk boilerplate consistency fix. + +## Audit result (2026-08-18) + +Every `.py` under `scripts/` — plus each HowTo repo's root `start_here.py` — was checked in the +three HowTo repos and all five user-facing workspaces. 39 scripts were missing the line: + +| Repo | Missing | Notes | +|---|---|---| +| HowToFit | 3 | `chapter_1_introduction/{start_here,tutorial_1_models}.py`, root `start_here.py` | +| HowToGalaxy | 2 | `chapter_2_modeling/tutorial_8_need_for_speed.py`, root `start_here.py` | +| HowToLens | 6 | `chapter_2_lens_modeling/{tutorial_8_need_for_speed,tutorial_11_slam}.py`, `chapter_3_pixelizations/tutorial_9_model_fit.py`, `simulator/{lens_x2,lens_x3}.py`, root `start_here.py` | +| autofit_workspace | 1 | `overview/overview_3_statistical_methods.py` | +| autogalaxy_workspace | 5 | under `guides/`, `imaging/data_preparation/`, `interferometer/` | +| autolens_workspace | 22 | mostly `cluster/*`, `guides/*` and feature `simulator.py` scripts | +| autocti_workspace | 0 | clean | + +`tutorial_4_methods.py` in HowToGalaxy, named in the original report, already carries the line — +it was fixed before this audit ran. + +The omissions are not a deliberate per-subtree policy: they sit alongside siblings in the same +directory that do carry the line (e.g. `autolens_workspace/scripts/cluster/` has 6 without and 5 +with). + +The two genuinely load-bearing cases are `HowToLens/scripts/simulator/lens_x{2,3}.py`, which write +to a relative `dataset/` path; the rest are prose-only or import-only scripts that pass today for +the reason the report gives. + +**Out of scope, deliberately:** `autoreduce_workspace` has 30 scripts and none carry the line, but +it has no `notebooks/` directory — the convention does not apply there yet. Same for every +`*_workspace_test` and `*_workspace_developer` repo (checked by tree listing; none generates +notebooks). + +**Placement:** after the module docstring, or after the `from auto* import jax_wrapper` line where +a script has one. One outlier — `autolens_workspace/scripts/guides/units/mass_to_light_ratio_units.py` +opens on imports rather than a docstring, so the line went at the top of the file. + +**Notebooks:** patched by hand to the exact shape PyAutoHands emits (uncommented, same cell, +`json.dumps(nb, indent=1)` round-trips byte-identically), because the generator is not available in +a cloud session. A real `generate.py` run should be a no-op on these — confirm before merging. + +Branch `claude/howto-setup-notebook-audit-dm2j9e` in all six repos, one commit each: +HowToFit#45, HowToGalaxy#66, HowToLens#70, autofit_workspace#138, autogalaxy_workspace#211, +autolens_workspace#485. diff --git a/complete/index.md b/complete/index.md index 3c51183d..a0a54eb9 100644 --- a/complete/index.md +++ b/complete/index.md @@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema: only then grep a dated bucket. Curators: edit the band between the CURATED markers; everything below GENERATED is rebuilt. -1005 records across 7 buckets. +1006 records across 7 buckets. ## Highlights @@ -59,6 +59,7 @@ _(curate hard-won records here — survives regeneration.)_ - [hazard-profiling-likelihood-tier](2026/08/hazard-profiling-likelihood-tier.md) — Tier-2 likelihood profiling landed with five persistent findings and corrected NNLS continuity semantics. - [health-conductor-stale-verdict](2026/08/health-conductor-stale-verdict.md) - [heart-red-guarded-sample-escape](2026/08/heart-red-guarded-sample-escape.md) — Closed the escape path that kept release validation RED after +- [howto-setup-notebook-audit](2026/08/howto-setup-notebook-audit.md) — none — run end-to-end from a cloud session; the six PRs are the record - [howto-smoke-all-tutorials](2026/08/howto-smoke-all-tutorials.md) - [howtofit-stub-fork-cleanup](2026/08/howtofit-stub-fork-cleanup.md) - [hpc-sync-first-push-race](2026/08/hpc-sync-first-push-race.md) — `hpc/sync push()` fired the `CODE_DIRS` rsyncs in parallel before diff --git a/dashboard.md b/dashboard.md index b2ff5980..5a67e707 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,7 +8,7 @@ Tasks only — the organism's health lives with the Heart (`/health`), not here. | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 5 | +| [In flight](#in-flight) (`active/`) | 4 | | [Parked](#parked) (`parked.md`) | 1 | | [Planned](#planned) (`planned.md`) | 7 | | [Backlog](#backlog) (`draft/`) | 138 | @@ -48,7 +48,6 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A Issued — each has an open GitHub issue and usually a branch. The full record for each is in [`active.md`](active.md). - [Commit the inference-methods programme plan + knowledge ledger into autolens_profiling](active/inference_programme_ledger.md) — [issue #134](https://github.com/PyAutoLabs/autolens_profiling/issues/134) — pr-open (https://github.com/PyAutoLabs/autolens_profiling/pull/135) -- [Audit HowTo tutorials for missing setup_notebook() line](active/missing_setup_notebook_audit.md) — pr-open (six PRs, awaiting CI) - [PyAutoReduce validation: slacs1430+4105 ACS reduction vs trusted legacy dataset](active/pyautoreduce_slacs1430_acs_comparison.md) - [Reconstructing a stored sample raises through `ignore_assertions=True`](active/to_instance_guard_gap.md) — [issue #1486](https://github.com/PyAutoLabs/PyAutoFit/issues/1486) — library-dev — WORKSPACE HALF SHIPPED; the PyAutoFit hardening (#1486) is what remains - [Version-stamp sync to 2026.8.17.1 + release-sed guards](active/version_stamp_sync_and_release_sed_guards.md) — [issue #235](https://github.com/PyAutoLabs/PyAutoHands/issues/235) — pr-open (https://github.com/PyAutoLabs/PyAutoLens/pull/700)