Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# PyAutoPulse — agent instructions

## Profiling campaign front door

Read `CHECKIN.md` for all-campaign updates and one-chat operation. Pulse now owns
profiling campaign intent and pending tasks in `campaigns.yaml` and `tasks/`,
including prompts migrated from Mind (`migration.yaml`). The board displays
the editable check-in prompt, active campaigns, open tasks, then measurements.
Mind retains bounded development issue/PR lifecycle and repository claims;
project repos retain measurement evidence. This explicit ownership update takes
precedence over older descriptions of Pulse as only a summary reader.


This file is for AI coding agents (Claude Code, Codex, Cursor, etc.) and humans
discovering this repository. PyAutoPulse is the **Pulse** organ of the PyAuto
organism — where the organism feels how fast it runs. It is the cross-project
Expand Down
58 changes: 58 additions & 0 deletions CHECKIN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# All profiling work in one chat

The editable prompt at the top of the dashboard is the entry point. Direction
supplied before or after it has the same meaning: focus that campaign or assess
that idea while retaining the overall sweep. No direction means check all open
campaigns and tasks. This static page does not run jobs or monitor them itself.

## Check-in procedure

1. Fetch current Pulse, Mind and relevant project branches; read their agent
instructions. Read `campaigns.yaml`, task files and `migration.yaml` when an
old Mind path is referenced. Pulse is the canonical profiling queue.
2. Read each campaign's current project wiki/ledger, results and issue/PR state.
Evidence URLs pin the last reviewed source; inspect the current default
branch for subsequent changes too. Consult Brain's profiling conductor for
scientific interpretation. With HPC access, use project pull/status commands;
otherwise report which jobs/results remain unverified. Never infer completion
from an old job id or absent data.
3. Compare with the last check-in and report one table: changes, current state,
blockers, next bounded step. Keep timing, correctness, compile cost, memory
and hardware distinct. Respect task contracts, human decisions, release gates
and Mind's current repository claims.
4. Incorporate user direction into priorities and suggested next steps. An idea
is a hypothesis until tested. Choose one bounded phase before execution and
use existing development/compute authorization rules. The check-in itself
authorizes no job submission, default/baseline change, release or merge.
5. Update `campaigns.yaml` with verified facts, reviewed dates, next steps and
source links. Preserve parked/blocked work. Append decisions and their
evidence to the task or project ledger. Record `last_checkin` as an ISO UTC
timestamp after the sweep, noting unavailable sources in the affected tasks.
A board refresh alone must never stamp a check-in.
6. Run `bin/pyauto-pulse board --offline` (or live ingest when available), then
`bin/pyauto-pulse check --offline`; persist through the repo development
workflow. Report the PR and whether it merged. Continue subsequent profiling
requests in this conversation; a fresh chat resumes from these records.

## Ownership and development handoff

Pulse owns campaign intent and pending tasks. File new profiling work under
`tasks/` with a row in `campaigns.yaml`. Each task has a stable ID, campaign,
title, status, priority, path, reviewed date and next step. Campaigns carry a
title, status, reviewed date, next step and evidence URL. Completed/superseded
rows remain in the ledger but leave the open table. Parked and blocked work
stays visible. A campaign is not a job: active does not mean running.

For a library/workspace implementation phase, file a small Mind development
prompt referencing the Pulse task, then use start_dev and the normal issue/PR
and claim lifecycle. Reuse existing issues (including autolens_profiling#362 for
timing-noise audit). Do not copy entire campaigns into Mind or bulk issue their
phases. After close-out, link the completion record from the Pulse task and
update its next step. Mind retains repository claims and implementation PR
lifecycle; mixed science/library work stays there with Pulse links.

Projects retain detailed campaign evidence, producers, results, pins and drift
policy. Pulse scheduling metadata never changes measurement qualification or
Heart readiness. Migration preserves original text and provenance; historical
headers inside those texts are not a live queue. Resolve old Mind paths via
`migration.yaml`; use the pinned source commit for unmigrated historical paths.
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# PyAutoPulse

The [campaign control room](CHECKIN.md) provides one editable check-in prompt,
active campaigns and Pulse-owned tasks above the detailed profiling evidence.
`campaigns.yaml` is the scheduling ledger; `migration.yaml` records original Mind
prompts and their source commit. Board refreshes never change task status or stamp
a check-in. Run `bin/pyauto-pulse board --offline` after ledger edits.

The **Pulse** organ of the [PyAuto organism](https://github.com/PyAutoLabs/PyAutoBrain/blob/main/ORGANISM.md):
where the organism feels how fast it runs.

Expand Down
6 changes: 6 additions & 0 deletions REFERENCE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# PyAutoPulse — reference

The [campaign control room](CHECKIN.md) provides one editable check-in prompt,
active campaigns and Pulse-owned tasks above the detailed profiling evidence.
`campaigns.yaml` is the scheduling ledger; `migration.yaml` records original Mind
prompts and their source commit. Board refreshes never change task status or stamp
a check-in. Run `bin/pyauto-pulse board --offline` after ledger edits.

The registry schema, the `profiling-summary` v1 contract as this organ reads
it, and the formats of the receipts, snapshots, board markers and cockpit feed.
Prose boundaries live in [AGENTS.md](AGENTS.md); the design is
Expand Down
285 changes: 285 additions & 0 deletions campaigns.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,285 @@
version: 1
last_checkin: null
campaigns:
- id: point-source-image-cpu
title: Point-source image plane · CPU
status: active
updated: '2026-10-03'
next: Phases 1–4 shipped; reconcile carried cleanup and test-placement work.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/point_source_image_plane_cpu.md
- id: point-source-image-gpu
title: Point-source image plane · A100
status: needs-decision
updated: '2026-10-03'
next: Measure an image-plane fit for the admission decision; retain forward-gradient correctness findings.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/point_source_gpu_breakdown.md
- id: point-source-source
title: Point-source source plane
status: parked
updated: '2026-10-03'
next: Core complete; gradient-sampler admission required before blackjax work. Warm-up measurement fix remains
open.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/point_source_source_plane.md
- id: cluster-pointsolver
title: Cluster PointSolver robustness and performance
status: active
updated: '2026-10-03'
next: Reconcile the expanded robustness contract; select one bounded phase and current representative baseline.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/cluster_pointsolver.md
- id: interferometer
title: Interferometer likelihood and streaming
status: active
updated: '2026-10-03'
next: Collect the final CPU matrix cell; scope streaming scaling, curvature reuse and FFT levers separately.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/interferometer_likelihood.md
- id: linear-solver
title: Linear-solver accuracy and cost
status: blocked
updated: '2026-10-03'
next: Verify release containing Array#595 and RAL sync before GPU/vmap parity and timing.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/linear_solver_accuracy.md
- id: imaging
title: Production imaging and likelihood breakdown
status: active
updated: '2026-10-03'
next: Reconcile measured GPU/CPU coverage, then profile production over-sampling and remaining costs.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/post_certified_breakdown.md
- id: measurement-tools
title: Measurement reliability and profiling tools
status: active
updated: '2026-10-03'
next: Audit timing noise; repair warm-up, cumulative accounting, memo blindness and compile-cell construction.
evidence: https://github.com/PyAutoLabs/PyAutoMind/tree/f21898e042be67afd86cc9c512879b3ad1c9ca9a/draft
- id: autofit
title: PyAutoFit profiling bootstrap
status: needs-decision
updated: '2026-10-03'
next: Confirm new repository creation before porting baseline packages; no empty producer registration.
evidence: https://github.com/PyAutoLabs/PyAutoMind/tree/f21898e042be67afd86cc9c512879b3ad1c9ca9a/draft
- id: critical-curves
title: Critical curves and evaluation grids
status: blocked
updated: '2026-10-03'
next: Galaxy#646 and profiling#365 merged; workspace#343 remains release-gated in Mind.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/critical_curves.md
- id: certified-solver
title: Certified positive solver
status: parked
updated: '2026-10-03'
next: Retain the parked opt-in solver study; read the project ledger before proposing new work.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/efcece3ea39c622e62cc61518f58394ee3986d4c/wiki/campaigns/certified_positive_solver.md
tasks:
- id: runtime_cell_single_jit_gpu_warmup
campaign: point-source-source
title: Runtime cells' A100 `single_jit` includes the post-compile warm-up — source-plane 0.642 ms vs a steady
0.267 ms on the same node
status: needs-decision
priority: low
path: tasks/runtime_cell_single_jit_gpu_warmup.md
updated: '2026-10-03'
next: Read the task and resolve its explicit human decision before execution.
- id: timing_noise_audit
campaign: measurement-tools
title: Audit timing tests and profiling gates for measurement noise
status: ready
priority: high
path: tasks/timing_noise_audit.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
issue: https://github.com/PyAutoLabs/autolens_profiling/issues/362
- id: mge_likelihood_breakdown_steps_are_cumulative_an
campaign: measurement-tools
title: MGE likelihood_breakdown steps are cumulative and `linear_gaussians` is reported as 0
status: ready
priority: low
path: tasks/mge_likelihood_breakdown_steps_are_cumulative_an.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: gradient_cost_probe
campaign: measurement-tools
title: 'A gradient-cost probe: forward vs `value_and_grad` ms/eval and a strict FD check, on any registry cell'
status: ready
priority: low
path: tasks/gradient_cost_probe.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: numba_breakdown_harness_memo_blind
campaign: measurement-tools
title: 'Numba breakdown harness: perturb the instance so the operated-matrix memo cannot hide a step'
status: ready
priority: medium
path: tasks/numba_breakdown_harness_memo_blind.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: search_settings_estimation_infrastructure
campaign: measurement-tools
title: Search settings-estimation + profiling infrastructure (n_starts / batch_size / n_batch)
status: ready
priority: normal
path: tasks/search_settings_estimation_infrastructure.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: jax_compile_probe_needs_own_cell_builder
campaign: measurement-tools
title: jax_compile/probe.py lost its cell builder with the searches tier — give profiling its own
status: ready
priority: normal
path: tasks/jax_compile_probe_needs_own_cell_builder.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: mass_field_flat_adoption_science_repos
campaign: measurement-tools
title: 'Remaining code consumers move to the flat `fields=` form: profiling, inference, JOSS benchmarks, Reduce
prototypes'
status: superseded
priority: normal
path: tasks/mass_field_flat_adoption_science_repos.md
updated: '2026-10-03'
next: Historical umbrella; shipped work retained. Do not issue it again.
- id: mge_nnls_fix_pyautoarray_571_slam_60
campaign: linear-solver
title: 'Linear-solver programme phase 3: GPU/vmap/A100 timing and parity rows for the solver corpus'
status: blocked
priority: medium
path: tasks/mge_nnls_fix_pyautoarray_571_slam_60.md
updated: '2026-10-03'
next: a release shipping the phase-2 fix (PyAutoArray#595, merged 2026-09-30 pending release; record complete/2026/09/raw-pdip-forward-polish.md)
AND the RAL mirror synced via HPCPullPyAuto
- id: autofit_profiling_bootstrap
campaign: autofit
title: 'autofit_profiling: bootstrap the repo + general PyAutoFit profiling epic'
status: needs-decision
priority: normal
path: tasks/autofit_profiling_bootstrap.md
updated: '2026-10-03'
next: Read the task and resolve its explicit human decision before execution.
- id: cluster_pointsolver_speed
campaign: cluster-pointsolver
title: Cluster PointSolver — robustness, analysis settings and performance
status: ready
priority: normal
path: tasks/cluster_pointsolver_speed.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: imaging_production_over_sampling
campaign: imaging
title: Profile imaging pixelizations at production over-sampling
status: ready
priority: normal
path: tasks/imaging_production_over_sampling.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: interferometer_decision_matrix_last_cell
campaign: interferometer
title: Interferometer decision matrix — fill the last cell (CPU rect 39² at alma_high r5.0) and flip the wiki
row to shipped
status: needs-decision
priority: normal
path: tasks/interferometer_decision_matrix_last_cell.md
updated: '2026-10-03'
next: Read the task and resolve its explicit human decision before execution.
- id: interferometer_fixed_mapper_curvature_preload
campaign: interferometer
title: 'Interferometer fixed-mapper searches: reuse the W~ curvature matrix across likelihood calls via `preloads.curvature_matrix`'
status: needs-slicing
priority: normal
path: tasks/interferometer_fixed_mapper_curvature_preload.md
updated: '2026-10-03'
next: Scope one bounded phase from the full task contract.
- id: interferometer_nnls_memo_scattered_stream_guard
campaign: interferometer
title: 'fnnls warm-start memo: stop it slowing scattered evaluation streams (interferometer CPU, autolens_profiling#332)'
status: ready
priority: medium
path: tasks/interferometer_nnls_memo_scattered_stream_guard.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: interferometer_streaming_scaling
campaign: interferometer
title: 'Campaign: interferometer streaming (array-free) vs in-memory — memory and time scaling to 2e8 visibilities'
status: ready
priority: high
path: tasks/interferometer_streaming_scaling.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: interferometer_w_tilde_fft_size_levers
campaign: interferometer
title: 'Interferometer W~ curvature matrix is FFT-bound on the mask extent: pruned padded FFT and real-space pixel
scale on the A100'
status: ready
priority: normal
path: tasks/interferometer_w_tilde_fft_size_levers.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: point_solver_profiling_cells
campaign: cluster-pointsolver
title: 'PointSolver profiling cells: lensed quasar → cluster runtime tier → single/multi-source → multiplane'
status: ready
priority: normal
path: tasks/point_solver_profiling_cells.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: point_source_image_plane_gpu_breakdown
campaign: point-source-image-gpu
title: 'Point-source A100 speed-up campaign: profile and optimize with the shared breakdown'
status: needs-decision
priority: high
path: tasks/point_source_image_plane_gpu_breakdown.md
updated: '2026-10-03'
next: Read the task and resolve its explicit human decision before execution.
- id: point_source_source_plane_chi_squared_speed
campaign: point-source-source
title: Point-source source-plane chi-squared speed-up campaign — remaining candidates (phases 1–2e shipped)
status: blocked
priority: low
path: tasks/point_source_source_plane_chi_squared_speed.md
updated: '2026-10-03'
next: Await gradient-sampler admission evidence; core campaign already complete.
- id: pointsolver_cpu_speed_campaign_remainder
campaign: point-source-image-cpu
title: Point-source (single-source) CPU campaign — carried leftovers and completion evidence
status: ready
priority: low
path: tasks/pointsolver_cpu_speed_campaign_remainder.md
updated: '2026-10-03'
next: Read the task and select the next bounded step; check current repository claims.
- id: post_certified_solver_likelihood_breakdown
campaign: imaging
title: Assess remaining likelihood bottlenecks after certified solver integration on CPU and GPUs
status: needs-slicing
priority: normal
path: tasks/post_certified_solver_likelihood_breakdown.md
updated: '2026-10-03'
next: Scope one bounded phase from the full task contract.
- id: profile_lens_aggregator_needs_config_dir
campaign: measurement-tools
title: 'profile_lens_aggregator.py cannot run from the autolens_workspace_developer root: no config/ directory'
status: ready
priority: low
path: tasks/profile_lens_aggregator_needs_config_dir.md
updated: '2026-10-03'
next: Read the task, reconcile current evidence and select one bounded step.
- id: gradient_pixelization_pin_residual_drift
campaign: measurement-tools
title: 'jax_profiling/gradient/imaging/pixelization.py: 3.2% of its pin move is unattributed'
status: ready
priority: normal
path: tasks/gradient_pixelization_pin_residual_drift.md
updated: '2026-10-03'
next: Read the task, reconcile current evidence and select one bounded step.
- id: pair_jax_xla_env_vars_with_measured
campaign: measurement-tools
title: Pair JAX/XLA env vars with measured compile and run times, per backend
status: ready
priority: high
path: tasks/pair_jax_xla_env_vars_with_measured.md
updated: '2026-10-03'
next: Read the task, reconcile current evidence and select one bounded step.
- id: quick_update_plotting_cost
campaign: imaging
title: Quick-update plotting cost — minutes per update, and it is not JAX compile
status: ready
priority: medium
path: tasks/quick_update_plotting_cost.md
updated: '2026-10-03'
next: Read the task, reconcile current evidence and select one bounded step.
Loading
Loading