Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions campaigns.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,11 @@ campaigns:
title: Measurement reliability and profiling tools
status: active
updated: '2026-10-08'
next: 'timing_noise_audit phase 1 (inventory) started 2026-10-08 against autolens_profiling#362 (worktree
timing-noise-audit-p1-inventory); related Mind draft call_accounting_ci_timing_threshold.md becomes a fix phase the
inventory ranks. Other tasks unchanged and unissued.'
evidence: https://github.com/PyAutoLabs/autolens_profiling/issues/362
next: 'timing_noise_audit phase 1 merged (autolens_profiling#402): inventory of 31 timing assertions/gates, 10 UNSAFE-SILENT
including the dashboard qualify() flag Pulse relies on (pulse/catalogue.py requires qualified: true). Fix phase (2)
will change the profiling-summary contract - Pulse must coordinate. New campaign page wiki/campaigns/measurement_tools.md.
Other tasks unchanged and unissued.'
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/main/wiki/campaigns/measurement_tools.md
- id: autofit
title: PyAutoFit profiling bootstrap
status: ready
Expand Down Expand Up @@ -138,11 +139,13 @@ tasks:
priority: high
path: tasks/timing_noise_audit.md
updated: '2026-10-08'
next: 'Phase 1 (inventory, audit only) STARTED 2026-10-08 on the human''s direction after the linear-solver programme
closed: issue autolens_profiling#362 reused (plan comment posted), Mind active/timing_noise_audit_phase1_inventory.md,
worktree timing-noise-audit-p1-inventory, Opus executing. Deliverable: results/notes/timing_noise_audit_2026_10.md
(inventory + PASS/FAIL/INCONCLUSIVE semantics + fix phases) and a read-only assertion lister. Next: /prm on its PR.'
next: 'Phase 1 (inventory) MERGED 2026-10-08 as autolens_profiling#402 (main un-reddened first by #403, a Brain theme
drift); Mind record complete/2026/10/timing-noise-audit-p1-inventory.md. 31 gates: 11 SOUND / 10 FRAGILE / 10 UNSAFE-SILENT;
worst is dashboard qualify() (laptop / no-loadavg / no-host rows qualify, and Pulse accepts evidence on that flag).
Next: file phase 2 = fix (1) shared overhead verdict for the #361 test + production cell (reuse #362); then fix (2)
dashboard qualification + drift wording, which changes profiling-summary and needs Pulse coordination.'
issue: https://github.com/PyAutoLabs/autolens_profiling/issues/362
evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/main/results/notes/timing_noise_audit_2026_10.md
- id: mge_likelihood_breakdown_steps_are_cumulative_an
campaign: measurement-tools
title: MGE likelihood_breakdown steps are cumulative and `linear_gaussians` is reported as 0
Expand Down
2 changes: 1 addition & 1 deletion dashboard.html

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions dashboard.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PyAutoPulse — profiling dashboard

<!-- pulse-campaigns:938f09acacf48ab93bf2de5c0e3ef4f84a61f526f0466765a8e579ca80210082 -->
<!-- pulse-campaigns:161570e1ff092d4786d763e82ad44cc4dd4fff6307169d73eb409c75fa36d052 -->
## Profiling Check In

<details><summary>Full check-in prompt</summary>
Expand Down Expand Up @@ -133,13 +133,13 @@ Reviewed 2026-10-04. No new results since the pin. Over-sampling phase 1: add --

<details><summary>Measurement reliability and profiling tools — active</summary>

Reviewed 2026-10-08. timing_noise_audit phase 1 (inventory) started 2026-10-08 against autolens_profiling#362 (worktree timing-noise-audit-p1-inventory); related Mind draft call_accounting_ci_timing_threshold.md becomes a fix phase the inventory ranks. Other tasks unchanged and unissued.
Reviewed 2026-10-08. timing_noise_audit phase 1 merged (autolens_profiling#402): inventory of 31 timing assertions/gates, 10 UNSAFE-SILENT including the dashboard qualify() flag Pulse relies on (pulse/catalogue.py requires qualified: true). Fix phase (2) will change the profiling-summary contract - Pulse must coordinate. New campaign page wiki/campaigns/measurement_tools.md. Other tasks unchanged and unissued.

[Campaign evidence](https://github.com/PyAutoLabs/autolens_profiling/issues/362)
[Campaign evidence](https://github.com/PyAutoLabs/autolens_profiling/blob/main/wiki/campaigns/measurement_tools.md)

### Active tasks

- [Audit timing tests and profiling gates for measurement noise](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/timing_noise_audit.md) — active: Phase 1 (inventory, audit only) STARTED 2026-10-08 on the human&#x27;s direction after the linear-solver programme closed: issue autolens_profiling#362 reused (plan comment posted), Mind active/timing_noise_audit_phase1_inventory.md, worktree timing-noise-audit-p1-inventory, Opus executing. Deliverable: results/notes/timing_noise_audit_2026_10.md (inventory + PASS/FAIL/INCONCLUSIVE semantics + fix phases) and a read-only assertion lister. Next: /prm on its PR.
- [Audit timing tests and profiling gates for measurement noise](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/timing_noise_audit.md) — active: Phase 1 (inventory) MERGED 2026-10-08 as autolens_profiling#402 (main un-reddened first by #403, a Brain theme drift); Mind record complete/2026/10/timing-noise-audit-p1-inventory.md. 31 gates: 11 SOUND / 10 FRAGILE / 10 UNSAFE-SILENT; worst is dashboard qualify() (laptop / no-loadavg / no-host rows qualify, and Pulse accepts evidence on that flag). Next: file phase 2 = fix (1) shared overhead verdict for the #361 test + production cell (reuse #362); then fix (2) dashboard qualification + drift wording, which changes profiling-summary and needs Pulse coordination.
- [MGE likelihood_breakdown steps are cumulative and `linear_gaussians` is reported as 0](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/mge_likelihood_breakdown_steps_are_cumulative_an.md) — ready: Unchanged and unissued (no GitHub refs, 2026-10-04). Select one bounded step when measurement-tools is prioritised.
- [A gradient-cost probe: forward vs `value_and_grad` ms/eval and a strict FD check, on any registry cell](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/gradient_cost_probe.md) — ready: Unissued. New context 2026-10-07: Heart unit-test timing flagged PyAutoLens forward-gradient tests +150%/+96% after PyAutoLens#768 rewrote that test file; not a controlled comparison, and this probe is the right instrument. Select one bounded step when prioritised.
- [Numba breakdown harness: perturb the instance so the operated-matrix memo cannot hide a step](https://github.com/PyAutoLabs/PyAutoPulse/blob/main/tasks/numba_breakdown_harness_memo_blind.md) — ready: Unchanged and unissued (PyAutoArray#496 closed 2026-08-27). Select one bounded step when prioritised.
Expand Down
8 changes: 8 additions & 0 deletions tasks/timing_noise_audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ The original task below is retained verbatim. Current scheduling state lives in
- Human direction (Pulse check-in, after the linear-solver programme closed): "move on to the next Pulse task"; phase 1 plan approved in chat. Issue #362 reused (plan comment posted 2026-10-08); Mind `active/timing_noise_audit_phase1_inventory.md` (a6e8aa1a); worktree `timing-noise-audit-p1-inventory`, branch `feature/timing-noise-audit-p1-inventory`; Opus executes.
- Scope: inventory of every timing assertion and production gate with estimator, samples, warm-up, pairing, clock, host qualification, cutoff, noise model, FP/FN risk and owner; PASS/FAIL/INCONCLUSIVE semantics; ranked fix phases with synthetic witnesses; read-only lister with `--check`. No gate or tolerance changes.


### Phase 1 shipped 2026-10-08

- [autolens_profiling#402](https://github.com/PyAutoLabs/autolens_profiling/pull/402) merged 2026-10-08 (lint green after main was un-reddened by #403, a PyAutoBrain theme drift). Mind record `complete/2026/10/timing-noise-audit-p1-inventory.md`. Issue #362 stays open for the fix phases.
- Inventory `results/notes/timing_noise_audit_2026_10.md`: 31 rows, 11 SOUND / 10 FRAGILE / 10 UNSAFE-SILENT. UNSAFE-SILENT includes dashboard `qualify()` (laptop rows with provenance, rows without load average and HPC rows without a host all qualify) — **Pulse-relevant**: `pulse/catalogue.py` accepts evidence on that flag, so today's "qualified" counts on the board are not a measurement-quality statement. The #361 trigger test is FRAGILE (near-zero power; INCONCLUSIVE silent under `pytest -q`; passed at a ratio of 0.79).
- Lister `scripts/misc/tooling/list_timing_assertions.py --check` is in lint.yml; campaign page `wiki/campaigns/measurement_tools.md` created.
- Proposed fix phases: (1) shared overhead verdict (test + cell, interval vs excess over budget, visible INCONCLUSIVE); (2) dashboard qualification + drift wording (contract change → Pulse coordination); (3) INCONCLUSIVE for go / lever rules, tie sets instead of argmax. Next: file phase 2 = fix (1) from this task, reusing #362; retire the subsumed Mind draft `call_accounting_ci_timing_threshold.md` when it is filed.

---

# Audit timing tests and profiling gates for measurement noise
Expand Down
Loading