Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@
</head>
<body>
<header class="hero"><span class="orb"><svg class="mark" viewBox="0 0 48 48" fill="none" stroke="currentColor" stroke-width="1.3" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><circle cx="24" cy="24" r="20.4"/><path d="M20.2,37.6 L20.2,33.6 C20.2,32.4 19.4,31.6 18.4,30.8 C16.8,29.6 15.8,28.2 15.6,26.6 C15.5,25.6 14.9,25.2 13.9,24.9 C12.6,24.5 12.3,23.7 13.0,22.7 C13.8,21.6 14.5,20.6 14.6,19.4 C15.0,15.4 18.2,12.2 22.4,11.8 C27.2,11.3 31.6,14.6 32.3,19.2 C32.8,22.4 31.6,25.0 30.6,27.0 C29.9,28.4 29.6,29.6 29.6,31.2 L29.6,37.6"/><circle cx="24.6" cy="19.6" r="4.4"/><path d="M22.5,19.7 L24.0,21.3 L26.8,18.1"/><path d="M34.2,16.5 L38.8,16.5 M35.2,21.0 L39.4,21.0 M33.6,25.5 L37.2,25.5"/><g fill="currentColor" stroke="none"><circle cx="40.1" cy="16.5" r="1.4"/><circle cx="40.7" cy="21.0" r="1.4"/><circle cx="38.5" cy="25.5" r="1.4"/></g></svg></span><h1>PyAuto<b>Mind</b><span class="kind">Dashboard</span></h1><div class="rule"></div><p class="tag">Intent. Priority. Flow.</p></header><p class="lede">Every task the Mind is holding. Tap a task's 📋 and its <code>/start_dev</code> command is on your clipboard — paste it into a Claude Code chat to route Claude straight to that task. <a href="#recent">Recent</a> is the same work by date — what has been happening rather than what to do next.</p>
<ul class="stats"><li><b>5</b><span>In flight</span></li><li><b>2</b><span>Human review</span></li><li><b>4</b><span>Parked</span></li><li><b>6</b><span>Planned</span></li><li><b>216</b><span>Backlog</span></li></ul>
<ul class="stats"><li><b>5</b><span>In flight</span></li><li><b>2</b><span>Human review</span></li><li><b>4</b><span>Parked</span></li><li><b>6</b><span>Planned</span></li><li><b>217</b><span>Backlog</span></li></ul>
<div class="fresh"><p><b>Last updated 2026-09-24.</b> This page is generated from <code>active/</code>, <code>draft/</code> and the registry files, so it is only as current as they are. <code>dashboard_refresh.yml</code> re-renders it on every push to <code>main</code> — that heals a stale page, but not a stale prompt: a task that shipped without its prompt advancing to <code>complete/</code> keeps rendering here as pickable backlog. Reconciling those is the refresh below.</p>
<div class="task"><button class="copy" data-cmd="Bring the PyAutoMind dashboard up to date. Work in the PyAutoMind checkout:

Expand Down Expand Up @@ -375,7 +375,7 @@ <h2>Planned</h2>
<div class="task"><button class="copy" data-cmd="/route start the planned PyAutoMind task latent-nan-guard-honest-run — its record is in planned.md" aria-label="Copy the Claude command">📋</button><p><b>latent-nan-guard-honest-run</b><span class="facets"> — planned 2026-07-22</span></p></div>
</details>
<h2>Backlog</h2>
<p class="muted">216 filed prompts, not started — sorted most-pickable first (priority, then size). 38 of them belong to an epic and are listed only under Epics below.</p>
<p class="muted">217 filed prompts, not started — sorted most-pickable first (priority, then size). 39 of them belong to an epic and are listed only under Epics below.</p>
<details>
<summary>bug — 66</summary>
<div class="task"><button class="copy" data-cmd="/start_dev draft/bug/euclid/sed_chain_waveband_fit_can_spin_forever.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/bug/euclid/sed_chain_waveband_fit_can_spin_forever.md">SED chain waveband fit can spin forever in Nautilus exploration…</a><span class="tags"><span class="pill w">🐛 bug</span><span class="pill">euclid</span><span class="pill n">small</span><span class="pill n">supervised</span><span class="pill r">high</span></span></p></div>
Expand Down Expand Up @@ -1256,14 +1256,15 @@ <h2>Epics</h2>
<div class="task"><button class="copy" data-cmd="/start_dev draft/feature/autolens/source_cluster_arc.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/feature/autolens/source_cluster_arc.md">Source &amp; Cluster arc — magnification science, PointSolver trust, cluster extended sources</a><span class="tags"><span class="pill w">✨ feature</span><span class="pill">autolens</span><span class="pill r">too-large</span><span class="pill n">supervised</span><span class="pill r">high</span></span></p></div>
</details>
<details>
<summary>Expectation propagation (EP) campaign — 8 queued prompt(s), in order</summary>
<summary>Expectation propagation (EP) campaign — 9 queued prompt(s), in order</summary>
<div class="task"><button class="copy" data-cmd="Continue the 'Expectation propagation (EP) campaign' epic. Its canonical state lives in draft/research/graphical_ep/ep_campaign.md — read that ledger (and any DECISIONS/RESULTS files beside it) first. Cross-check this epic's entry in PyAutoMind/epics.md, any related rows in PyAutoMind/active.md, and the referenced repos' open issues and PRs, to work out the last completed phase and what is currently in flight. Then pick the next logical step and continue it through the normal workflow (/start_dev — filing the phase's prompt first if none exists), updating the ledger as the work advances. Note: umbrella phase map — each phase's real content lives in its own prompt under draft/research/graphical_ep/; the campaign file itself is never issued. Science half: PyAutoCortex `projects/{analytic_gaussian,ep_toy_gaussian,slope_hierarchy_scale,ic50_workspace}.md` (campaign phases 1b, 3 and 4)." aria-label="Copy the Claude command">📋</button><p><b>Expectation propagation (EP) campaign</b> — <span class="facets">ledger: <code>draft/research/graphical_ep/ep_campaign.md</code></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/research/graphical_ep/graphical_scoping.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/research/graphical_ep/graphical_scoping.md">Graphical Model Scale-Up — Scoping</a><span class="tags"><span class="pill w">🔬 research</span><span class="pill">graphical_ep</span><span class="pill r">too-large</span><span class="pill r">human-required</span><span class="pill r">high</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/feature/autofit/ep_lbfgs_jax.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/feature/autofit/ep_lbfgs_jax.md">The project @z_projects/ic50_workspace is our IC50 use case which we</a><span class="tags"><span class="pill w">✨ feature</span><span class="pill">autofit</span><span class="pill n">medium</span><span class="pill g">safe</span><span class="pill n">normal</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/bug/autofit/ep_factors_end_the_run_with_zero.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/bug/autofit/ep_factors_end_the_run_with_zero.md">EP factors end the run with zero SUCCESS updates on the exactly-Gaussian…</a><span class="tags"><span class="pill w">🐛 bug</span><span class="pill">autofit</span><span class="pill n">medium</span><span class="pill n">supervised</span><span class="pill r">high</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/bug/autofit/nan_in_ep_evidence_accumulation_invalid_value.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/bug/autofit/nan_in_ep_evidence_accumulation_invalid_value.md">NaN in EP evidence accumulation: <code>invalid value encountered in add</code> in ep_mean_field.py…</a><span class="tags"><span class="pill w">🐛 bug</span><span class="pill">autofit</span><span class="pill n">medium</span><span class="pill n">supervised</span><span class="pill n">medium</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/bug/graphical_ep/analytic_gaussian_unseeded_graphical_column.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/bug/graphical_ep/analytic_gaussian_unseeded_graphical_column.md"><code>analytic_gaussian.py</code>'s graphical column is an unseeded run, so its PARITY count…</a><span class="tags"><span class="pill w">🐛 bug</span><span class="pill">graphical_ep</span><span class="pill n">small</span><span class="pill g">safe</span><span class="pill n">low</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/feature/autofit/ep_hierarchical_scatter_moment_matching.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/feature/autofit/ep_hierarchical_scatter_moment_matching.md">EP: moment-matching projection for the hierarchical scatter…</a><span class="tags"><span class="pill w">✨ feature</span><span class="pill">autofit</span><span class="pill n">medium</span><span class="pill n">supervised</span><span class="pill n">normal</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/refactor/autofit/ep_factor_search_wrapper_overhead.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/refactor/autofit/ep_factor_search_wrapper_overhead.md">EP: cut the per-factor-search wrapper overhead…</a><span class="tags"><span class="pill w">♻️ refactor</span><span class="pill">autofit</span><span class="pill n">medium</span><span class="pill n">supervised</span><span class="pill r">high</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/research/graphical_ep/ep_campaign.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/research/graphical_ep/ep_campaign.md">EP campaign — phase map for the 2026 Q3 graphical/EP push</a><span class="tags"><span class="pill w">🔬 research</span><span class="pill">graphical_ep</span><span class="pill r">too-large</span><span class="pill r">human-required</span><span class="pill r">high</span></span></p></div>
<div class="task"><button class="copy" data-cmd="/start_dev draft/research/graphical_ep/ep_scoping.md" aria-label="Copy the Claude command">📋</button><p><a href="draft/research/graphical_ep/ep_scoping.md">Expectation Propagation Scale-Up — Scoping</a><span class="tags"><span class="pill w">🔬 research</span><span class="pill">graphical_ep</span><span class="pill r">too-large</span><span class="pill r">human-required</span><span class="pill r">high</span></span></p></div>
</details>
Expand Down
14 changes: 11 additions & 3 deletions dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ anything you could not verify.
| [Human review](#human-review) (`draft/human_review/`) | 2 |
| [Parked](#parked) (`parked.md`) | 4 |
| [Planned](#planned) (`planned.md`) | 6 |
| [Backlog](#backlog) (`draft/`) | 216 |
| [Backlog](#backlog) (`draft/`) | 217 |

> **No batch in flight.**

Expand Down Expand Up @@ -588,7 +588,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.

## Backlog

**216** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **38** of them belong to an epic and are listed only under [Epics](#epics) below.
**217** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **39** of them belong to an epic and are listed only under [Epics](#epics) below.

<details>
<summary><b>bug</b> — 66</summary>
Expand Down Expand Up @@ -2536,7 +2536,7 @@ Continue the 'Cluster strong lensing — Source & Cluster arc' epic. Its canonic
</details>

<details>
<summary><b>Expectation propagation (EP) campaign</b> — 8 queued prompt(s), in order</summary>
<summary><b>Expectation propagation (EP) campaign</b> — 9 queued prompt(s), in order</summary>

<details><summary>📋 <b>Expectation propagation (EP) campaign</b> — ledger: `draft/research/graphical_ep/ep_campaign.md`</summary>

Expand Down Expand Up @@ -2594,6 +2594,14 @@ Continue the 'Expectation propagation (EP) campaign' epic. Its canonical state l

</details>

<details><summary>📋 <a href="draft/refactor/autofit/ep_factor_search_wrapper_overhead.md">EP: cut the per-factor-search wrapper overhead…</a> — autofit · medium · supervised · high</summary>

```
/start_dev draft/refactor/autofit/ep_factor_search_wrapper_overhead.md
```

</details>

<details><summary>📋 <a href="draft/research/graphical_ep/ep_campaign.md">EP campaign — phase map for the 2026 Q3 graphical/EP push</a> — graphical_ep · too-large · human-required · high</summary>

```
Expand Down
35 changes: 35 additions & 0 deletions draft/feature/autofit/ep_hierarchical_scatter_moment_matching.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,38 @@ A moment-matching projection option for `_HierarchicalFactor` updates
- Mechanism records: `complete/2026/09/ep-prior-id-zero.md`, `ep-message-support.md`, `ep-laplace-hessian.md`
- PyAutoFit#1405 (umbrella), #1561 (Hessian), autofit_workspace_test#91 (referee)
- README caveat: `PyAutoFit/autofit/graphical/README.md` §3.5

## 2026-09-24 — slope_hierarchy_scale 343299: hierarchical factor 0/50 SUCCESS at N=25

RAL job 343299 (PyAutoCortex `projects/slope_hierarchy_scale.md`) completed
2026-09-16 02:03 BST: wall 5h20m, `MAX_STEPS=2`, 25 lenses, JAX-on-CPU, no
pool. It logged 50 "jit compiling vectorized" lines — one compile per factor
search (2 steps x 25), confirming `draft/refactor/autofit/ep_analysis_level_compile_cache.md`.
Every dataset factor reached SUCCESS (2/2 each). **The hierarchical factor
never did:** `HierarchicalFactor0` has 50 rows = 27 `BAD_PROJECTION` + 23
`FAILURE`, zero SUCCESS (`PriorFactor239` and `PriorFactor263` also stale),
and the library's own STALE FACTORS warning fired. The parent recovery printed
exactly the prior: mean 2.0000 ± 0.7071, sigma 0.5000 ± 0.3536 (priors
TG(2,1) / TG(0.5,0.5); truth sigma 0.1).

Under the Laplace optimiser's semantics `BAD_PROJECTION` = the Hessian at the
mode is not finite or not negative-definite (the scale parameter is driven to
a limit) and `FAILURE` = the line search failed and the mean field was handed
back unchanged. This is the Laplace-on-scatter caveat
(`PyAutoFit/autofit/graphical/README.md` §3.5, `analytic_gaussian` leg B) at
100 % on the lensing model: per-lens slope widths of 0.002-0.05 are far
tighter than the scatter prior, so the tilted density in sigma sits at the
boundary. The EP arm of campaign phase 3 therefore cannot estimate the slope
scatter with the Laplace projection; this prompt is now the blocking decision
for it (human call, `Autonomy: supervised`). EP output (local):
`output/sample_n25_seed42/ep/expectation_propagation/a7ec60f07f4261afcb98b2adef484199/`
(`ep_history.csv`, `ep_diagnostics.results`).

Cross-reference — `draft/bug/autofit/ep_factors_end_the_run_with_zero.md`:
that bug records a `HierarchicalFactor1` ending with only `BAD_PROJECTION` and
never a SUCCESS on 6/200 leg-B seeds of the analytic toy (seeds 29, 71, 106,
152, 178, 193). The same shape now reproduces at 100 % on a real lensing model
(every hierarchical update over 2 steps x 25 factors), so it is no longer a
rare-seed edge case: whether a zero-SUCCESS hierarchical factor is legitimate
(the bug's open question) and whether moment matching removes it should be
judged together.
88 changes: 88 additions & 0 deletions draft/refactor/autofit/ep_factor_search_wrapper_overhead.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# EP: cut the per-factor-search wrapper overhead (redundant second run_nested pass, per-search plots, instance_from_vector fast path)

Type: refactor
Target: PyAutoFit
Repos:
- PyAutoFit
Themes:
- graphical-ep
Difficulty: medium
Autonomy: supervised
Priority: high
Status: formalised
Consequence: judge
Witness: on the ep_toy_gaussian N=5, max_steps=3 toy, `run_nested` is called once per Dynesty factor search (15 calls for 15 searches, not 30) and per-search wall drops from ~2.45 s to <= 1.5 s at identical parent moments (mean and sigma within 1e-6 of a control run with the same seeds), with the full `test_autofit` suite green run serially.
Review-minutes: 5
Unattended: ready
Epic: graphical-ep
Filed: 2026-09-24

## Why

Every EP factor update is a full autofit non-linear search. On the
`ep_toy_gaussian` #1405 collapse toy (PyAutoCortex
`projects/ep_toy_gaussian.md`) a 1-parameter Dynesty factor search spends
about as much time in the autofit wrapper as in the sampler. A 1000-search
N=50 repeat carries ~15-20 min of pure overhead; best case on the toy is ~2x.

Profile (2026-09-24, laptop, PyAutoFit at the RAL mirror a73684012, N=5,
max_steps=3, 15 factor searches, cProfile total 109 s; files in
`ep_factor_search_wrapper_overhead_assets/`):

- **Per search 2.45 s = ~1.47 s dynesty `run_nested` + ~0.98 s autofit wrapper**
(plots, a redundant second `run_nested` pass, samples writes).
- **Redundant second `run_nested` pass (~5.5 s / 13 %).** 30 `run_nested`
calls for 15 searches. `run_search_internal`
(`autofit/non_linear/search/nest/dynesty/search/abstract.py:413-455`)
returns `finished = total_iterations == iterations_after_run`, which is
never true on a first pass that converges, so every search does a full
`perform_update(during_analysis=True)` + checkpoint restore + a no-op second
`run_nested`.
- **Per-search visuals.** `corner_anesthetic` + `model_fit.png` 7.7 s / 18 %;
`data.png` redrawn every search 2.3 s / 5 % although the data never changes.
- **`instance_from_vector`** (`autofit/mapper/prior_model/abstract.py:990`)
is 57 % of `fitness.call`, i.e. ~25-30 % of `run_nested` for a 1-parameter
model: `autofit/mapper/model.py:30` `cache` 486k calls,
`autofit/mapper/prior_model/attribute_pair.py` wrapper 481k calls,
`prior_model.py:483 _instance_for_arguments` 2.7 s tottime.
- **`truncated_normal.log_partition` / `kl`** call `scipy.stats.norm.cdf`
18.6k times / 3.45 s — use `scipy.special.ndtr` / `log_ndtr`.
- **`manage_quick_update`** 1.8 s while quick-update is disabled (61k calls).
- **`gc.collect` per search** 75 ms (1.78 s over 15).

## What (phases)

1. **Dynesty finished-detection fix** so a converged first pass is recognised
and the second `perform_update` + checkpoint restore + `run_nested` is
skipped; unit test that counts `run_nested` calls (mock the sampler) and
asserts one per converged search.
2. **EP factor searches skip per-search visuals** — a config switch, default
off under EP (corner/model_fit per search), and `data.png` drawn once per
analysis rather than per search.
3. **Mapper fast path for fixed-structure models** in `instance_from_vector`
(precompute the prior-to-attribute map once per model; bypass the
`cache` / `attribute_pair` wrappers on the hot path).
4. **`ndtr` swap** in `autofit/messages/truncated_normal.py` (and
`mapper/prior/_erf_helpers.py` if it pays) — `scipy.special.ndtr` /
`log_ndtr` in place of `scipy.stats.norm.cdf`. Also look at the disabled
`manage_quick_update` early-out and the per-search `gc.collect`.

Each phase should be measured on the same toy (per-search wall, `run_nested`
call count, parent moments against a same-seed control).

## Side observation (reproducibility)

Three runs of `ep_repeat.py --repeat 0` gave parent means 51.86 / 51.33 /
51.61: the repeat seed does not make EP reproducible at a73684012. Known cause
— the Laplace refine draws from global `np.random` and depends on variable-id
ordering (PyAutoFit#1352). The witness's "same seeds" control therefore needs
the global RNG seeded (or the #1352 fix) before the 1e-6 moment comparison is
meaningful; if it cannot be made deterministic, compare moments across
matched repeats within their spread and say so.

## Links

- Campaign ledger: `draft/research/graphical_ep/ep_campaign.md` (row 1b; phase 6 epic 2, EP profiling)
- Sibling: `draft/refactor/autofit/ep_analysis_level_compile_cache.md` (the JAX recompile per factor search)
- EMFILE fix that preceded this profile: PyAutoFit#1632 (dynesty single-core no pool), #1634 (release factor search internals)
- Evidence: `ep_factor_search_wrapper_overhead_assets/ep_profile_report.txt`, `ep_pstats_cumulative.txt`, `ep_pstats_tottime.txt` (paths shortened: `SP/` = site-packages, `PyAutoFit/`, `STDLIB/`, `SCRATCH/` = the profiling scratch dir)
Loading