Repository navigation
Description
Overview
Phase 0 sub-issue 0B of the likelihood-lifecycle epic (ledger PyAutoMind/draft/research/autolens_profiling/likelihood_lifecycle_epic.md). Restructures autolens_profiling's results/ into the §3.7 mirrored tree from a human-reviewed day-zero migration map, then upgrades the existing catalogue builder to emit profiling-summary@3 (contract shipped in PyAutoPulse#70) so Pulse can re-pin lens to @3. This is the last open phase-0 sub-issue (0A, 0C, 0D, 0E merged). Rulings R1/R2/R3/R7 are decided; nothing is re-measured.
Plan
- Migration-map PR —
catalogue/migration_day_zero.json: old → new path + sha256 for all 1,481 trackedresults/files,results/notes/ledgers →wiki/campaigns/<dataset_type>/<variant>/, dead experiments →archive/YYYY/MM/<slug>/. Nothing moves; the human reviews the map first. - Move-only PR —
git mvinto the §3.7 layout +archive/+ README pointers +results/README.mdredirect table. Held until docs(cluster): record the human decision — keep the 1.0″ starting grid (#439) #441 merges and no live RAL job runs on a RAL checkout of this repo. - Catalogue builder PR — extend
scripts/misc/tooling/build_catalogue.pyto emitprofiling-summary@3(class/status/superseded_by,planned_cells,figures[],archive_record_count), mirror test,--checkcurrent. - Pulse PR —
registry.yamllenspin@2 → @3, receipt refetched (unblocks 0C'spredicted_ms_per_eval). scripts/misc/grouping PR —misc/test/split by group,misc/searches/orphan removed, one-offs archived, README per group.- Coverage matrix + parity convention PR —
coverage_matrix.py(likelihood × class × measurement → current/superseded/missing/stale);parity.jsonper cell read by the builder asmeasurement: parity.
Tier: judge — merge mode: human /prm for every PR, in order. Autonomy: supervised.
Detailed implementation plan
Affected Repositories
- autolens_profiling (primary)
- PyAutoPulse (one-line registry re-pin after the builder merges)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| lens/autolens_profiling | main (03d36c8) | untracked dataset/abell_1201/ only |
| organs/PyAutoPulse | main | clean (unregistered worktree checkin/2026-10-17, unrelated) |
Claim guard worktree_check_conflict likelihood-lifecycle-p0-profiling autolens_profiling PyAutoPulse → exit 0.
Suggested branch: feature/likelihood-lifecycle-p0-profiling (PRs 1–6 as stacked/sequential branches off it, e.g. -map, -move, -builder, -misc, -coverage).
Decisions settled in the plan
- R3 provisional Optimal rows — Pulse REFERENCE.md:
class: optimalrequires adecision_id; none exists before phase 3. Emitclass: secondarywith"provisional_optimal: true"inlimitations; phase 3 flips them when designations are written. - Mirror test location — repo tests live under
scripts/misc/test/; the mirror test (test_results_mirror_scripts.py: everyresults/<dataset_type>/<variant>/hasscripts/<dataset_type>/<variant>/and vice versa,archive/excluded) goes there; PR 5 re-homes it with its group. - Move PR vs. path indexes — the move breaks path lists in
results/provenance_grandfathered.txt, result destinations incatalogue/script_routes.json/wiki_bindings.json, andALLOWED_NOTES_DIRSincheck_results_layout.py. Regenerate these mechanically from the map inside the move PR (paths only, no measurement content; listed in the PR body) so main never goes red between PRs.
Implementation Steps
- Map generator (script under
scripts/misc/tooling/), emittingcatalogue/migration_day_zero.json; retire legacy names in paths (pixelization→rectangular,*_numba→ R1setup_id, host/tier →host_class). Archive classes: dated suffixes,_lever,_probe, pre-2026-09 series,baselines/PreOptimizationTimes, frozen packs. sha256 recomputed after docs(cluster): record the human decision — keep the 1.0″ starting grid (#439) #441 merges. - Move PR applies the map with
git mvonly (plus map-derived path-index rewrites);check_results_layout.py --checkgreen. build_catalogue.py→ v3 shape per PyAutoPulse REFERENCE.md "profiling-summaryv3" (metric/unit/values; nosetups[]/selections[]); §3.6 status rule (29planned_baseline+ 3reference_candidate→currentwhere the host rule holds; others on mirrored keys →superseded+superseded_by; archived → counted); R7local_*nevercurrent;dashboard/catalogue.jsonregenerated;test_build_catalogue.pyextended.- PyAutoPulse
registry.yamllens→profiling-summary@3;pyauto-pulse checkaccepts it; receipt refetched. scripts/misc/polish.coverage_matrix.py+parity.jsonconvention documented inresults/README.md.
Witness
test_results_mirror_scripts.py passes; build_catalogue --check current, every record carries class/status, exactly one current per key; pyauto-pulse check accepts the re-pinned lens at @3 and its receipt refetches; the coverage matrix lists current/superseded/missing/stale cells per likelihood × class × measurement.
Key Files
scripts/misc/tooling/build_catalogue.py— v2 emitter → v3scripts/misc/tooling/check_results_layout.py,results/provenance_grandfathered.txt— layout gatecatalogue/registry.json,catalogue/script_routes.json,catalogue/wiki_bindings.json,_script_routes.pyresults/README.md— artefact policy + redirect table- PyAutoPulse
registry.yaml,REFERENCE.md
Original Prompt
Click to expand starting prompt
Likelihood lifecycle phase 0B: autolens_profiling mirrored tree, day-zero migration map, profiling-summary@3 catalogue builder, misc grouping, coverage matrix, parity convention
Type: feature
Target: autolens_profiling
Repos:
- autolens_profiling
- PyAutoPulse
Themes: - profiling
- dashboard
Difficulty: large
Autonomy: supervised
Priority: high
Status: formalised
Consequence: judge
Witness:tests/test_results_mirror_scripts.pypasses (everyresults/<dataset_type>/<variant>/has ascripts/<dataset_type>/<variant>/and vice versa,archive/excluded);build_catalogue --checkis current and emitsprofiling-summary@3with every record carryingclass/statusand exactly onecurrentper key;pyauto-pulse checkaccepts the re-pinnedlensinstance at @3 and its receipt refetches; the coverage matrix tool lists current / superseded / missing / stale cells per likelihood × class × measurement
Review-minutes: 40
Unattended: never
Epic: likelihood-lifecycle
Phase: 0
Parent: draft/research/autolens_profiling/likelihood_lifecycle_epic.md
Filed: 2026-10-21
Phase 0 sub-issue 0B of the likelihood-lifecycle epic (ledger draft/research/autolens_profiling/likelihood_lifecycle_epic.md; report §3.1, §3.6, §3.7, §4 phase 0; survey likelihood_lifecycle_epic_surveys/01_profiling_inference_layout.md). 0A (PyAutoPulse#70, merged 2026-10-21) defined the contract this issue emits: v3 records carry likelihood, instrument, regime, per-record setup + setup_id, host_class, model_variant, measurement, metric/unit/values, class, status, superseded_by, day_zero, figures[], decision_id, release_pin, measured_at; there are no setups[]/selections[] — planned_cells (with class) replace selections; archive_record_count is required (PyAutoPulse REFERENCE.md "profiling-summary v3").
Rulings applied (confirmed 2026-10-21)
R1 setup_id = <backend>_<device>_<operator>_<precision>[_<transformer>], host/tier moves to host_class; R2 label-only history cut — nothing deleted, nothing re-run; R3 the provisional designations of report §3.3 are the phase-0 class labels (provisional, confirmed by phases 2–3; no decision_id exists yet, so every provisional Optimal row is emitted class: optimal only once a designation decision exists — until then class: secondary with a provisional_optimal: true note in limitations, or the builder refuses; decide in the plan); R7 local_* rows are never current.
What, as separate PRs in this order (data guard: moves and content never in one PR)
- Commit nothing new first; migration map —
catalogue/migration_day_zero.json: old path → new path (+ sha256) for every tracked file underresults/(1,456 files, 326 MB,results/lens/solver/corpus165 MB) andresults/notes/ledgers →wiki/campaigns/<dataset_type>/<variant>/; dead experiments (dated suffixes,_lever,_probe, pre-2026-09 series,baselines/PreOptimizationTimes, frozen packs) →archive/YYYY/MM/<slug>/. Legacy model names retired in paths (pixelization→rectangular,*_numba→ thesetup_id). The map is reviewed by the human before any move. - Move-only PR (git mv only, reviewed as renames): the §3.7 layout
results/<dataset_type>/<variant>/<instrument>/<setup_id>/<host_class>/{runtime,breakdown,gradient,fit,parity,provenance}.*withhistory/<release_pin>_<date>/for superseded files;results/lens/<component>/,results/misc/<group>/;wiki/campaigns/<dataset_type>/<variant>/;archive/YYYY/MM/<slug>/with a README pointer at each old parent;results/README.mdredirect table. Never under a live RAL job on a RAL checkout of this repo. - Catalogue builder (
scripts/misc/tooling/build_catalogue.py,catalogue/registry.json,_script_routes.py): reads the mirrored tree, emitsprofiling-summary@3, converts the config grammar, assignsclass/statusby the §3.6 rule (the 29planned_baseline+ 3reference_candidate→currentprovisional where the host rule holds; other records on mirrored keys →supersededwithsuperseded_by; archived → counted),planned_cellswithclass,figures[]wherefit.pngexists;build_catalogue --check;dashboard/catalogue.jsonregenerated; Pulseregistry.yamllenspin →profiling-summary@3(one-line PyAutoPulse PR after the builder merges) and the Pulse receipt refetched. scripts/misc/grouping polish:misc/test/split by group,misc/searches/orphan removed, one-off scripts toarchive/, a README per group.- Coverage matrix tool (extend the profiling conductor's
campaigngrid diff or addscripts/misc/tooling/coverage_matrix.py): likelihood × class × measurement → current / superseded / missing / stale, reading the v3 catalogue and the §3.3 provisional table. - Parity witness convention:
parity.jsonper cell (reference log-likelihood, tolerance, status ∈ match / diverged / not_assessed) documented inresults/README.mdand read by the builder intomeasurement: parityrecords.
Out of scope
Re-measuring anything (phase 3); designation decisions (phase 3); fit.png rendering in _profile_cli.py (phase 1B); runtime_estimates.yaml (phase 5E); autolens_inference (0C); Brain conductor grid (0E).
Blocked-by (operational, not a header gate)
(Resolved 2026-10-22: that task merged as autolens_profiling#438 and the claim is released.) autolens_profiling was claimed by cluster-pointsolver-robust-settings (issue #436, branch feature/cluster-pointsolver-robust-settings, 5 unmerged commits, RAL sweeps live on 2026-10-21). The move-only PR cannot land under that branch or under live RAL jobs on a RAL checkout; start after that task ships (or the human releases the claim) — /start_dev registers this in planned.md until then.
Activity
Merge pull request #444 from PyAutoLabs/feature/likelihood-lifecycle-…
Jammy2211 commentedon Oct 23, 2026
PR 1 of 6 (day-zero migration map) MERGED via /prm: #444 → aeef275. Map: 1,509 files, 23 unresolved (setup axis not recorded), human §3.7 rulings applied (model_variant segment, scans/experiments, hpc_ral_cpu for CPU-on-GPU-node, _wtilde). Next: PR 2 move-only — regenerate the map first if #441 has merged; never under live RAL jobs on a RAL checkout. Issue stays open.
chore(lint): exclude archive/ from ruff like results/ (#442 PR 2)
Merge pull request #447 from PyAutoLabs/feature/likelihood-lifecycle-…
Jammy2211 commentedon Oct 24, 2026
PR 2 merged; PR 3 and PR 4 opened (2026-10-24)
archive/(archived notes keep their exact contents).profiling-summary@3: 126 current and 319 superseded records, 99 archived, 246 planned cells. R3 and R7 are applied. Under the size rule, 112 current cells fall back to the nearest size, because nothing is at 1250 or 1225 yet. Undated rows take the date their file was first committed (human ruling).results_layout.pyshim (Mind draft filed).Remaining after that: PR 3b, PR 5 (misc grouping), PR 6 (coverage matrix and parity).
Merge pull request #448 from PyAutoLabs/feature/likelihood-lifecycle-…
Jammy2211 commentedon Oct 24, 2026
PR 3 + PR 4 merged (2026-10-24, /prm)
profiling-summary@3) merged as 81183fd.pyauto-pulse check: lensprofiling-summary@3valid, 445 records (126 current, 319 superseded, 99 archived). That part of the witness is met.Remaining: PR 3b (shim, legacy writers, trend re-key; Mind draft), PR 5 (
scripts/misc/grouping), PR 6 (coverage matrix and parity convention).Jammy2211 commentedon Oct 24, 2026
Auto run (human launch 2026-10-24): PR 3b, PR 5, PR 6 through to open PRs
Under
AUTONOMY.md, an explicit auto launch on a supervised, judge-tier task opens each PR and flags it. Every merge is a human/prm. Heart is YELLOW, with the reason set the human acknowledged at launch: manifest drift and stale PyAutoLens release validation. Any new reason, or RED, stops the run. Each PR is gated on the full pytest suite, every lint--checkstep, an independent review verdict of CLEAN, and Heart. No profiling is run and no measurement artefact is moved or edited.PRs are stacked in this order, each branched from the one before. Merge them in order: 3b → 5 → 6.
build_dashboard.pyandbuild_readme.pytrend series and README tables on the v3 identity, with an explicit old-id → v3-id map so published history continues. Route the ~30 legacy-name writers (versioned summaries, vmap probes,sweep.py/aggregate.py, HPC submits) throughresolve_output_pathstogether with their readers, verified only by path-builder unit tests or dry runs. Deleteresults_layout.py, and add a test that no script writes outside the mirrored tree. Anything that only a real run can verify is listed for the next RAL batch, which needs a human go.scripts/misc/grouping: splitmisc/test/by group (git mvplus import and conftest fixes only), remove themisc/searches/orphan, move one-off scripts toarchive/, and add a README per group.scripts/misc/tooling/coverage_matrix.py: likelihood × class × measurement → current, superseded, missing or stale, read from the v3 catalogue and the §3.3 provisional table. Standard-size cells covered only by a size fallback count as missing.parity.jsonconvention per cell (reference log-likelihood, tolerance, status match, diverged or not_assessed), documented inresults/README.mdand read by the builder intomeasurement: parityrecords.build_catalogue --check --validate-with ../PyAutoPulsemust stay valid at @3.The 3b Mind draft is folded into this task, so it no longer appears as separate backlog.
Jammy2211 commentedon Oct 24, 2026
Auto run finished: PR 3b, PR 5 and PR 6 are open, stacked. Merge in this order with
/prm: #449 → #450 → #451.catalogue/series_id_map.json(144 series and 158 points before, 145 and 159 after; nothing lost). ~40 writer sites and 15 HPC submits now write into the mirrored tree through one placement rule.results_layout.pyis deleted. A guard test stops writes outside the mirrored tree.aggregatereading stale rows) plus 3 non-blocking. All fixed; re-review CLEANscripts/misc/test/split intoscripts/misc/<group>/test/. The move commit is renames only, and the same 1554 test ids are collected before and after. A README per group.misc/searches/was already gone, and nothing was archived.coverage_matrix.py→dashboard/coverage.{json,md}, with a lint check. Of 246 planned cells: 2 current, 6 stale, 238 missing (32 of them size fallbacks, per the size ruling). Theparity.jsonconvention, read intomeasurement: parityv3 records; no witnesses exist yet and none were made up.Heart was YELLOW throughout, within the human-acknowledged reasons. A mid-run RED (local PyAutoLens behind origin) was cleared by a fast-forward the human approved. Profiling items that only a real run can verify are listed in #449. Once #451 merges, phase 0B is complete.
Merge pull request #449 from PyAutoLabs/feature/likelihood-lifecycle-…
Merge pull request #450 from PyAutoLabs/feature/likelihood-lifecycle-…
Merge pull request #451 from PyAutoLabs/feature/likelihood-lifecycle-…
Jammy2211 commentedon Oct 24, 2026
Shipped: phase 0B complete (2026-10-24)
All seven PRs are merged:
results/restructured to mirrorscripts/, plusarchive/profiling-summary@3(126 current, 319 superseded, archive 99)pyauto-pulse checkOK)series_id_map.json), writers into the mirrored tree,results_layout.pydeleted, write guardscripts/misc/tests grouped by folder (same 1554 tests), a README per groupdashboard/coverage.*) and theparity.jsonconventionWitness:
build_catalogue --checkis current at @3, every record carriesclassandstatus, and each key has exactly one current record.pyauto-pulse checkaccepts lens at @3 and its receipt refetched.Carried forward:
current_records_unplanned, and havecatalogue_lifecyclereadprovisional_designations.json.This completes phase 0 of the likelihood-lifecycle epic (0A–0E all merged).