Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions campaigns.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
version: 1
last_checkin: '2026-10-04T17:18:38Z'
campaigns:
- id: setup-baseline
title: Setup baseline collection and scientific acceptance
status: needs-decision
updated: '2026-10-06'
next: 'Specification is draft (autolens_profiling#384); no baseline collection or acceptance authorized.
Resolve immutable software and dependency lock, dataset hashes/settings, hardware/threads/load and
timing/cache/memory methodology/witnesses; freeze the manifest before separately authorizing collection.
Compile probe builder remains unavailable; archived timings are context only.'
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/62a6ad1e82735e360c7cf25be62253a286493b17/wiki/campaigns/setup_baseline.md
- id: point-source-image-cpu
title: Point-source image plane · CPU
status: active
Expand Down Expand Up @@ -83,6 +92,17 @@ campaigns:
in Mind. Retain parked; read the project ledger before proposing new work.
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/a93f37a7ade16fcb7673a7777dfde505183728ed/wiki/campaigns/certified_positive_solver.md
tasks:
- id: setup_baseline_campaign
campaign: setup-baseline
title: Freeze the setup baseline specification, then collect and review evidence
status: needs-decision
priority: high
path: tasks/setup_baseline_campaign.md
updated: '2026-10-06'
next: 'Human resolves draft manifest choices and freezes the specification; collection requires separate
compute authorization. The report prepares evidence for human scientific review and never promotes
baselines. CPU production arrays use ral only; compile capability gap remains open.'
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/62a6ad1e82735e360c7cf25be62253a286493b17/baseline/campaign.json
- id: runtime_cell_single_jit_gpu_warmup
campaign: point-source-source
title: Runtime cells' A100 `single_jit` includes the post-compile warm-up — source-plane 0.642 ms vs a steady
Expand Down
2 changes: 1 addition & 1 deletion dashboard.html

Large diffs are not rendered by default.

14 changes: 13 additions & 1 deletion dashboard.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PyAutoPulse — profiling dashboard

<!-- pulse-campaigns:77ca7c41ddc4b75d3a81b678afe72bb4a5d261fa62678fa94e5331b020b7b65c -->
<!-- pulse-campaigns:cbea1eba629dc0064077ad5d1af613005d3a58f8a6d17b54b253eb6468e6d379 -->
## Profiling Check In

<details><summary>Full check-in prompt</summary>
Expand All @@ -15,6 +15,18 @@ Last check-in: 2026-10-04T17:18:38Z (review date, not measurement freshness).

## Active campaigns

<details><summary>Setup baseline collection and scientific acceptance — needs-decision</summary>

Reviewed 2026-10-06. Specification is draft (autolens_profiling#384); no baseline collection or acceptance authorized. Resolve immutable software and dependency lock, dataset hashes/settings, hardware/threads/load and timing/cache/memory methodology/witnesses; freeze the manifest before separately authorizing collection. Compile probe builder remains unavailable; archived timings are context only.

[Campaign evidence](https://github.com/PyAutoLabs/autolens_profiling/blob/62a6ad1e82735e360c7cf25be62253a286493b17/wiki/campaigns/setup_baseline.md)

### Active tasks

- [Freeze the setup baseline specification, then collect and review evidence](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/setup_baseline_campaign.md) — needs-decision: Human resolves draft manifest choices and freezes the specification; collection requires separate compute authorization. The report prepares evidence for human scientific review and never promotes baselines. CPU production arrays use ral only; compile capability gap remains open.

</details>

<details><summary>Point-source image plane · CPU — active</summary>

Reviewed 2026-10-04. Phases 1–4 shipped; PyAutoLens#764 (extent warning) released in 2026.10.4.1. Next: docs-only autolens_profiling PR writing campaign completion evidence and fixing stale wiki/index lines (#763 &#x27;in flight&#x27; → #764 merged). RAL leftover folders present; mirror sync unverified.
Expand Down
71 changes: 71 additions & 0 deletions tasks/setup_baseline_campaign.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Freeze the setup baseline specification, then collect and review evidence

Campaign: setup-baseline
Task: setup_baseline_campaign
Reviewed: 2026-10-06

Current scheduling state lives in `campaigns.yaml`. This is a pending scientific
campaign task, not authorization to run jobs or accept measurements. The project
specification is being implemented under
[autolens_profiling#384](https://github.com/PyAutoLabs/autolens_profiling/issues/384);
those specification changes do not complete baseline collection or scientific
acceptance.

## Project specification and evidence

- [Draft campaign manifest](https://github.com/PyAutoLabs/autolens_profiling/blob/62a6ad1e82735e360c7cf25be62253a286493b17/baseline/campaign.json)
- [Baseline procedure and report contract](https://github.com/PyAutoLabs/autolens_profiling/blob/62a6ad1e82735e360c7cf25be62253a286493b17/baseline/README.md)
- [Project campaign ledger](https://github.com/PyAutoLabs/autolens_profiling/blob/62a6ad1e82735e360c7cf25be62253a286493b17/wiki/campaigns/setup_baseline.md)
- [Compile route capability](https://github.com/PyAutoLabs/autolens_profiling/blob/62a6ad1e82735e360c7cf25be62253a286493b17/catalogue/script_routes.json)
- [Pending compile-builder task](jax_compile_probe_needs_own_cell_builder.md)
- [Measurement noise audit](timing_noise_audit.md)

The specification links pin the reviewed Phase 6 implementation commit. This
is still a draft, not the later frozen campaign. Record the separately reviewed
frozen manifest identity and project revision before collection. Existing archived measurements remain historical context, without
establishing a fresh campaign baseline or a temporal performance trend. Pulse
keeps its existing receipts and snapshots; this task creates no new measurements.

## Human decisions before freezing

1. Choose full immutable revisions for the profiling project and every library,
and an immutable dependency lock including Python, JAX/jaxlib and numerical
dependencies. Record how the executable environment reproduces that lock.
2. Specify each selected setup's dataset content hashes and full scientific
settings, including model, mask/grid, precision, solver, sampling and all
relevant defaults. Decide which cells and evidence axes are required.
3. Name the exact CPU/GPU hardware and backend, thread allocation, process
placement and host-load conditions for each collection leg. Production CPU
arrays must use `--partition=ral` only. GPU collection needs its own explicit
device/resource allocation and authorization.
4. Fix timing, synchronization, repetition and uncertainty methodology;
distinguish compile, warm-up and steady runtime, declare cache state, memory
metric/measurement boundaries and the necessary correctness/parity witnesses.
Keep time, compile cost, memory and correctness as separate evidence axes.
5. Decide how to handle unavailable capabilities before collection. The compile
probe's per-cell builder is explicitly unavailable in `catalogue/script_routes.json`.
Archive coverage does not make the probe runnable. Repair it through its
separate bounded development task, or record the gap and obtain a human scope
decision; never mark compile evidence complete from archived records.

## Next bounded steps and acceptance

1. Have the human resolve the choices above in the project's draft manifest and
procedure, review its validation and record the frozen manifest identity plus
the immutable project specification revision. Preserve the decisions in the
project campaign ledger. A draft or a freeze alone authorizes no compute.
2. Request separate human compute authorization for the named cells, resources
and measurement methods. Then use the project's drivers to collect against
that frozen specification, preserving result identities, provenance and
incomplete/invalid evidence. Record actual execution outcomes in the project.
3. Run the project's report against that frozen manifest and collected evidence.
Treat its output as readiness for human scientific review; report generation
never promotes a baseline, moves a pin or grants scientific acceptance.
4. Have the human judge the matched evidence and record acceptance, rejection or
requested follow-up for each selected setup and evidence axis in the project
ledger. Only then update this pending task from the recorded decision. Any
later baseline/pin promotion requires its own explicit authorization.

Until those steps are complete, retain `needs-decision`: no campaign is reported
as running or accepted, and specification implementation remains distinct from
the later collection and review deliverables.
Loading