Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 9 additions & 15 deletions campaigns.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
version: 1
last_checkin: '2026-10-17T15:12:59Z'
last_checkin: '2026-10-23T15:00:32Z'
campaigns:
- id: setup-baseline
title: Setup baseline collection and scientific acceptance
Expand Down Expand Up @@ -47,13 +47,9 @@ campaigns:
- id: cluster-pointsolver
title: Cluster PointSolver robustness and performance
status: active
updated: '2026-10-17'
next: 'No campaign wiki/results change or new matching implementation claim found. Representative cluster baseline
and observable-overflow policy remain unstarted. Choose one bounded phase; inherited single-source results
do not qualify a representative cluster baseline. Check-in 2026-10-17: no project wiki/results/issue change
since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and
2026.10.17.1 supply no new measurement.'
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/cluster_pointsolver.md
updated: '2026-10-23'
next: 'Human decision 2026-10-23 (chat): keep the 1.0″ starting grid as the production default (decision record cluster-pointsolver-starting-grid; closes the phase-1 deferral). Inputs: phase-1 simple sweep (#438), A2744 pilot job 412573 (1.0″ misses fewest, 2/69), starting-grid follow-up autolens_profiling#439 → #440 merged 03d36c81 (no fixed scale finds every image; no separation rule). Mitigation: the workspace validation check. Next bounded task: spurious-core candidates library task (start_dev in progress), then the validation check. No default changed.'
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/03d36c810b2a4f7d31342d9a75f46cd7fe10f43a/results/notes/cluster_pointsolver_campaign.md
- id: interferometer
title: Interferometer likelihood and streaming
status: parked
Expand Down Expand Up @@ -285,12 +281,11 @@ tasks:
- id: cluster_pointsolver_speed
campaign: cluster-pointsolver
title: Cluster PointSolver — robustness, analysis settings and performance
status: ready
status: active
priority: normal
path: tasks/cluster_pointsolver_speed.md
updated: '2026-10-13'
next: 'Unissued, no claim. Pick one bounded phase per Mind epics.md: observable-overflow invalid-result policy
+ CI witness, or a representative-data baseline cell.'
updated: '2026-10-23'
next: 'Decision taken 2026-10-23 (human): keep 1.0″ + validation check; #439/#440 merged (03d36c81). Next: spurious-core candidates library task (start_dev in progress), then the workspace validation check.'
- id: imaging_production_over_sampling
campaign: imaging
title: Profile imaging pixelizations at production over-sampling
Expand Down Expand Up @@ -384,9 +379,8 @@ tasks:
status: ready
priority: normal
path: tasks/point_solver_profiling_cells.md
updated: '2026-10-13'
next: 'Unissued, no claim. Candidate first cell: a representative-data cluster likelihood_runtime baseline
(single-source preset).'
updated: '2026-10-23'
next: 'Unissued, no claim. Candidate first cell: a representative-data cluster likelihood_runtime baseline (single-source preset). Reconcile onto the phase-1 harness (scripts/cluster/image_plane/robust_settings_sweep.py) when the next cell is issued; no duplicate cell.'
- id: point_source_image_plane_gpu_breakdown
campaign: point-source-image-gpu
title: 'Point-source A100 speed-up campaign: profile and optimize with the shared breakdown'
Expand Down
33 changes: 33 additions & 0 deletions checkins/2026-10-23.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Profiling check-in — 2026-10-23 (directed: cluster-pointsolver)

Reviewed at 2026-10-23T13:54:43Z. This check-in covered only the **cluster-pointsolver** campaign at the
human's direction; other campaigns were not reviewed and their ledger rows are unchanged.
Review time and board generation are not measurement times. It authorizes no compute, default
change, resubmission, scientific acceptance, release or merge.

## Verified availability

- RAL was reachable (SSH `euclid_jump`). `sacct` for job 412573 and its log (1951 lines) were
read. No compute was run and no results were collected into the project.
- GitHub: autolens_profiling#438 merged at `c8c9abb3` (2026-10-22); Mind record
`complete/2026/10/cluster-pointsolver-robust-settings.md`.

## Campaign review

| Campaign | Changes since 2026-10-17 | Current state | Blockers | Next bounded step |
|---|---|---|---|---|
| [Cluster PointSolver robustness and performance](../tasks/cluster_pointsolver_speed.md) | Phase 1 merged ([#438](https://github.com/PyAutoLabs/autolens_profiling/pull/438), `c8c9abb3`); human decision DEFERRED 2026-10-22 (default stays 1.0″); Knowledge block landed. A2744 pilot (job 412573) pulled: **FAILED** after 16 diagnostic rows (03:33:07, MaxRSS 85.5 GB) on LLVM `Cannot allocate memory`; no JSON written. A2744 coverage: 1.0″ 67/69, 0.5″ 66/69, 0.7″ 64/69, 2.0″ 61/69, truth-bbox 58–60/69 — opposite ordering to `simple`, so the scale ranking is dataset-dependent. RSS 8.3 → 39.4 GB with live arrays ~0: retained compiled executables plus compile transients. | active | The A2744 sweep harness must release executables between rows (or run one row per process) before any A2744 full sweep; resubmission needs explicit human authorization (none given). Phase 2 CI witnesses: nothing accepted to distil. | Starting-grid / close-member-image Mind draft (`draft/research/autolens_profiling/cluster_pointsolver_starting_grid_close_member_image.md`) via start_dev; one task, no bulk queue. |

## Task changes

| Task | State | Note |
|---|---|---|
| [cluster_pointsolver_speed](../tasks/cluster_pointsolver_speed.md) | ready → active | Check-in 2026-10-23 section, phase-1 Knowledge block and A2744 pilot paragraph added. |
| [point_solver_profiling_cells](../tasks/point_solver_profiling_cells.md) | ready | Reconciles onto phase 1's harness (`scripts/cluster/image_plane/robust_settings_sweep.py`) when the next cell is issued; no duplicate cell. |

## Unavailable or unverified

- The A2744 model is a synthetic prior-median model with 3 judged draws; its rows are diagnostic
only and carry no speed claim. The backend rows and timing rounds were never reached.
- The pilot log is not yet tracked in autolens_profiling; landing it is deliverable 0 of the
starting-grid draft.
2 changes: 1 addition & 1 deletion dashboard.html

Large diffs are not rendered by default.

15 changes: 8 additions & 7 deletions dashboard.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PyAutoPulse — profiling dashboard

<!-- pulse-campaigns:4e7b5e1126147b4593a7e870235fd9a5d367256202aaeb9408c0254298791675 -->
<!-- pulse-campaigns:b2279cce088381119a25ae69e4a1a493c347db477e562fe28b38596ca10f6197 -->
## Profiling Check In

<details><summary>Full check-in prompt</summary>
Expand All @@ -25,7 +25,7 @@ After taking action, report what changed, what the evidence supports and what re

</details>

Last check-in: 2026-10-17T15:12:59Z (review date, not measurement freshness).
Last check-in: 2026-10-23T15:00:32Z (review date, not measurement freshness).

## Active campaigns

Expand Down Expand Up @@ -68,14 +68,14 @@ Reviewed 2026-10-17. Parked as research, human decision 2026-10-17 (chat) (&quot

<details><summary>Cluster PointSolver robustness and performance — active</summary>

Reviewed 2026-10-17. No campaign wiki/results change or new matching implementation claim found. Representative cluster baseline and observable-overflow policy remain unstarted. Choose one bounded phase; inherited single-source results do not qualify a representative cluster baseline. Check-in 2026-10-17: no project wiki/results/issue change since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and 2026.10.17.1 supply no new measurement.
Reviewed 2026-10-23. Human decision 2026-10-23 (chat): keep the 1.0″ starting grid as the production default (decision record cluster-pointsolver-starting-grid; closes the phase-1 deferral). Inputs: phase-1 simple sweep (#438), A2744 pilot job 412573 (1.0″ misses fewest, 2/69), starting-grid follow-up autolens_profiling#439 → #440 merged 03d36c81 (no fixed scale finds every image; no separation rule). Mitigation: the workspace validation check. Next bounded task: spurious-core candidates library task (start_dev in progress), then the validation check. No default changed.

[Campaign evidence](https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/cluster_pointsolver.md)
[Campaign evidence](https://github.com/PyAutoLabs/autolens_profiling/blob/03d36c810b2a4f7d31342d9a75f46cd7fe10f43a/results/notes/cluster_pointsolver_campaign.md)

### Active tasks

- [Cluster PointSolver — robustness, analysis settings and performance](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/cluster_pointsolver_speed.md) — ready: Unissued, no claim. Pick one bounded phase per Mind epics.md: observable-overflow invalid-result policy + CI witness, or a representative-data baseline cell.
- [PointSolver profiling cells: lensed quasar → cluster runtime tier → single/multi-source → multiplane](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/point_solver_profiling_cells.md) — ready: Unissued, no claim. Candidate first cell: a representative-data cluster likelihood_runtime baseline (single-source preset).
- [Cluster PointSolver — robustness, analysis settings and performance](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/cluster_pointsolver_speed.md) — active: Decision taken 2026-10-23 (human): keep 1.0″ + validation check; #439/#440 merged (03d36c81). Next: spurious-core candidates library task (start_dev in progress), then the workspace validation check.
- [PointSolver profiling cells: lensed quasar → cluster runtime tier → single/multi-source → multiplane](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/point_solver_profiling_cells.md) — ready: Unissued, no claim. Candidate first cell: a representative-data cluster likelihood_runtime baseline (single-source preset). Reconcile onto the phase-1 harness (scripts/cluster/image_plane/robust_settings_sweep.py) when the next cell is issued; no duplicate cell.

</details>

Expand Down Expand Up @@ -420,9 +420,10 @@ Project `autolens_inference`, scope `None`, read from [PyAutoLabs/autolens_infer
</details>


<!-- decision-history:731ca0edc0c01337b75841f88d79bd1838b31ba05910928a6353dc43150ca0ea -->
<!-- decision-history:4d88bb773a36f774562f9e06d52a024b839cc44d2aa91282311891f6064c43c3 -->
## Decision History

- <a href="https://github.com/PyAutoLabs/PyAutoPulse/blob/main/decisions/cluster-pointsolver-starting-grid.md">Keep the 1.0″ PointSolver starting grid for cluster fits, with a validation check</a>
- <a href="https://github.com/PyAutoLabs/PyAutoPulse/blob/main/decisions/direct-conv-symmetric-gate.md">Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column ordering</a>
- <a href="https://github.com/PyAutoLabs/PyAutoPulse/blob/main/decisions/interferometer-route-gate-cost-model.md">Route interferometer curvature by a calibrated two-constant cost model</a>
- <a href="https://github.com/PyAutoLabs/PyAutoPulse/blob/main/decisions/adaptsplit-zero-weight-delaunay-only.md">Exclude zero-weight mappings from AdaptSplit pixel signals on Delaunay meshes only</a>
Expand Down
4 changes: 4 additions & 0 deletions decisions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,7 @@ decisions:
title: Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column ordering
date: '2026-10-16'
url: https://github.com/PyAutoLabs/PyAutoPulse/blob/main/decisions/direct-conv-symmetric-gate.md
- id: cluster-pointsolver-starting-grid
title: Keep the 1.0″ PointSolver starting grid for cluster fits, with a validation check
date: '2026-10-23'
url: https://github.com/PyAutoLabs/PyAutoPulse/blob/main/decisions/cluster-pointsolver-starting-grid.md
80 changes: 80 additions & 0 deletions decisions/cluster-pointsolver-starting-grid.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Keep the 1.0″ PointSolver starting grid for cluster fits, with a validation check

Date: 2026-10-23
Decision maker: James Nightingale
Decision source: Human decision in chat on 2026-10-23 ("keep the 1.0″ starting grid"), taken after the phase-1 decision was DEFERRED on 2026-10-22 pending the starting-grid follow-up and the A2744 pull. Recorded in Mind [complete/2026/10/cluster-pointsolver-starting-grid.md](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/cluster-pointsolver-starting-grid.md).

## Campaign

This decision belongs to the cluster PointSolver campaign, task
[`cluster_pointsolver_speed`](../tasks/cluster_pointsolver_speed.md). Phase 1
([autolens_profiling#436](https://github.com/PyAutoLabs/autolens_profiling/issues/436) →
[#438](https://github.com/PyAutoLabs/autolens_profiling/pull/438)) measured which triangle-solver
settings find every multiple image of cluster-scale point sources. The open question was the
step-0 initial grid scale: the production 1.0″ against a finer start.

Every result below is judged against an independent oracle that samples the multi-plane
lens-equation residual densely and Newton-polishes each root (calibrated to 2.84e-8″). It is not
the triangle solver.

## Evidence

Ledger: [results/notes/cluster_pointsolver_campaign.md at `03d36c81`](https://github.com/PyAutoLabs/autolens_profiling/blob/03d36c810b2a4f7d31342d9a75f46cd7fe10f43a/results/notes/cluster_pointsolver_campaign.md).
The three inputs use different models and hardware; their numbers are not interchangeable.

- **`simple` sweep (phase 1).** Synthetic `dataset/cluster/simple`: 13 components, 3 planes,
2 sources, 101 judged draws. RAL job 412572, Xeon 8490H, CPU fp64, production extent
100×100 @ 1.0″. All rows cover 306/306 prior roots. Of 300 stress roots, 0.5″ and 0.7″ miss 2,
the production 1.0″ misses 5 and 2.0″ misses 6. Warmed two-source solve: 1.0″ 5.29 ms, 0.5″
6.82 ms (0.78× speed), 0.7″ 7.02 ms (0.75×). No row is admissible. Mind record
[cluster-pointsolver-robust-settings.md](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/cluster-pointsolver-robust-settings.md).
- **A2744 pilot.** A2744-geometry synthetic model at prior medians (not a fitted cluster model),
7 sources, 3 judged draws, 69 oracle roots. RAL job 412573, CPU fp64, pulled 2026-10-23. The job
FAILED after 16 diagnostic rows on LLVM memory exhaustion, so there is no timing claim. Coverage:
1.0″ misses 2/69, 0.5″ 3, 0.7″ 5, 2.0″ 8. The ordering is the opposite of `simple`.
- **Starting-grid follow-up** ([#439](https://github.com/PyAutoLabs/autolens_profiling/issues/439) →
[#440](https://github.com/PyAutoLabs/autolens_profiling/pull/440), merged `03d36c81`; Mind
[cluster-pointsolver-starting-grid.md](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/cluster-pointsolver-starting-grid.md)).
autolens_visualization cluster model, laptop CPU fp64, 17 judged draws, diagnostic timings.
The workspace 1.0″ grid loses `point_0`'s central image 0.39″ from `main_1`'s core, and so does
every scale down to 0.3″. Only 0.2″ finds it (60.5 ms vs 7.4 ms warmed at 1.0″, ~8×), and 0.2″
still misses 3/108 roots. Six draw-sources are found at a coarser scale and lost at a finer one.
No single-separation rule holds over 236 draw-sources. The solver also returns spurious
candidates (residual 0.04″) on `r_core = 0` member cores.

## Options considered

- **0.5″ start.** Fewest misses on `simple` (2) at ~0.77× speed, but 3 misses on A2744 (more than
1.0″) and it does not find the viz model's central image.
- **Fixed 0.2″ start.** ~8× the per-solve cost (laptop) and still misses 3/108 viz roots.
- **Separation-based scale rule.** Unsupported: "scale ≲ separation / 1.06" is contradicted, and
the only uncontradicted form rests on two synthetic models.
- **Keep 1.0″ and add a validation check.** Every observed image must have a solved counterpart;
otherwise re-solve on a small fine grid around it and say so. Drop candidates whose
source-plane residual exceeds the solver tolerance.

## Decision

Keep the 1.0″ starting grid as the production default, and adopt the validation check as the
mitigation for missed images. This closes the phase-1 decision deferred on 2026-10-22.

## Implications

No library or workspace default is changed by this record. The validation check is a Mind draft
(`draft/feature/autolens_workspace/cluster_point_solver_validation_check.md`, unissued). The
spurious-core candidates are a separate library task
(`draft/feature/autolens/point_solver_spurious_core_candidates.md`, start_dev in progress), next.
The solver is not in the likelihood (`FitPositionsSourceSolved` traces the observed images), so a
costly check at validation/visualization time does not slow fits.

## Revisit conditions

Evidence comes from synthetic models only; there is no fitted A2744 reference tracer, and the
A2744 pilot covers 3 draws without timings. A full A2744 sweep (needs a reference tracer and
human authorization), a fitted cluster model, or a validation check that fires often in practice
would reopen the choice.

## History

First record for this choice. It follows the phase-1 deferral (2026-10-22) recorded in the task
and the profiling ledger.
Loading
Loading