Skip to content

feat: drop the old results layout — v3-keyed trends, writers into the mirrored tree (#442 PR 3b) - #449

Merged
Jammy2211 merged 6 commits into
mainfrom
feature/likelihood-lifecycle-p0-profiling-legacy
Oct 24, 2026
Merged

Jammy2211 merged 6 commits into
mainfrom
feature/likelihood-lifecycle-p0-profiling-legacy

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

PR 3b of #442 (likelihood-lifecycle phase 0B). This finishes what PR 3 (#448) deliberately left: nothing reads or writes the old results/ layout any more.

  • Dashboard trends re-keyed on the v3 identity (<measurement>:<dataset_type>/<variant>:<instrument>:<setup_id>:<host_class>:<model_variant>). The old-id → new-id map is committed in catalogue/series_id_map.json. It is generated mechanically from catalogue/migration_day_zero.json by the new build_series_id_map.py, which has a --check. Of 151 pre-move ids, 107 map to v3 ids. The other 44 have no mirrored cell (archived host-less summaries, or unresolved rows) and keep their old id under legacy:. No history lost: 144 series and 158 points before, 145 and 159 after. Every old id appears under its new id with the same releases. The one extra series is a real A100 point-source breakdown that the old file-name pattern missed.
  • README tables: the runtime table uses v3 cell names (e.g. imaging/rectangular/hst). The breakdown table keeps every row it showed before, 106 → 110: archived rows are labelled "(archived)", and the 4 new rows are 3 light_fixed rows plus the A100 row.
  • Writers re-pointed: about 40 Python writer sites and 15 HPC submits now write into the mirrored tree, and 8 readers changed to match. Sweep rows and comparisons share one placement rule (scripts/misc/likelihood_runtime/sweep_layout.py): sweep.py passes the exact path as --output-path, and aggregate.py reads with the same function. All 308 sweep-matrix rows get distinct paths under their own instrument.
  • results_layout.py deleted, along with every import of it.
  • Guard test test_results_write_destinations.py fails if any script writes a results path outside the mirrored tree. It follows / joins, .joinpath, os.path.join, Path(...) arguments and roots aliased within the same module. A results/<dataset_type>/ path must name a real scripts/<dataset_type>/<variant>.
  • dashboard/summary.json record ids change to the v3 ids; the mapping is in catalogue/series_id_map.json. The dashboard regeneration also rewrites timestamps across ~557 files, as every full re-render does.

Needs a real run (not verifiable without profiling; nothing was run)

  • sweep.py default root: only --dry-run and --skip-existing exercised. --skip-existing no longer counts the stale unresolved rows left under the old runtime folder, so those cells will be re-measured.
  • size_sweep_table, fixed_light_scaling_table and gradient_mode_library_ab find new rows only if the writer's payload facts match the reader's hints.
  • The fixed_light witnesses (levers/l2/l3/s4/s4b) and the point-source *_summary_v* / image_plane_solved writers: checked only by computing the paths.
  • The vmap probe writer and resolve_vmap_batch: consistent by construction, no probe run.

Scripts Changed

  • scripts/misc/tooling/build_dashboard.py: v3 series ids, native mirrored-tree scan, pre-move sources attached through the map
  • scripts/misc/tooling/build_readme.py: native runtime, breakdown, baseline and misc-group scanners; v3-keyed breakdown table that keeps archived rows, labelled
  • scripts/misc/tooling/build_series_id_map.py (new): old → new series id map generator with --check
  • scripts/misc/tooling/results_layout.py: deleted
  • scripts/misc/likelihood_runtime/sweep_layout.py (new): one placement rule for sweep rows and comparisons; sweep.py and aggregate.py use it
  • _profile_cli.py: --output-path, defensive --instrument, legacy landing and locate helpers
  • 12 likelihood_runtime*.py leaves: vmap probe landing directory shared by writer and reader; interferometer and datacube leaves record their transformer; point-source summaries placed
  • fixed_light witnesses, numba scaling/draws, memo_policy, nautilus_batch_capture, separation_rule, oracle_figure, bakeoff, memo_after_summary, misc/latent: re-pointed
  • size_sweep_table, fixed_light_scaling_table, gradient_mode_library_ab, lens/solver/_corpus: readers re-pointed
  • hpc/batch_{cpu,gpu}: 15 submits re-pointed
  • Tests: test_build_dashboard (mirrored fixture, continuity); new test_build_readme, test_results_write_destinations, test_legacy_result_helpers, test_sweep_layout

Test Plan

  • ruff 0.17.2 check and format --check
  • every lint --check step, including build_catalogue --check --validate-with ../PyAutoPulse and build_series_id_map --check
  • pytest scripts/misc/test/: 1548 passed, 6 skipped
  • smoke: the 9 lint.yml scripts plus the 4 changed interferometer/datacube leaves
  • browser setup-page test
  • lychee (CI)

Validation checklist (--auto run — plan was not pre-approved)

  • Effective level: supervised (header: supervised, cap: feature/large → supervised). Explicit human auto launch on 2026-10-24, so the ship checkpoint resolves to opening a PR. Judge tier: merge is a human /prm. Merge order: this PR, then PR 5, then PR 6.
  • Plan: on the issue (feat: likelihood-lifecycle 0B — mirrored results tree + profiling-summary@3 catalogue #442 comment "Auto run … PR 3b, PR 5, PR 6"), written at start, unmodified since
  • Gate: tests 1548 passed / 6 skipped · smoke 13/13 · review: an independent Opus reviewer first returned FINDINGS (1 blocking: sweep writer/reader mismatch for interferometer/datacube; 3 non-blocking: dropped README rows, guard gaps, stray delaunay_nn comparison). All four were fixed in b244831..94dc857, and the re-review returned CLEAN, with the witness basis-cited. One accepted residual: the guard doesn't follow a results root passed as a default argument. · Heart YELLOW within the launch acknowledgement: manifest drift (generated hooks 3, root AGENTS.md 1, shared-standards 1, workspace checkouts 1), PyAutoHands and PyAutoMind open PRs 7 days old, release validation incomplete. A mid-run RED (local PyAutoLens 2 commits behind) was cleared by a fast-forward and a fresh Heart collection, with the human's OK.
  • Human: plan sound in hindsight?
  • Human: diff matches plan (no scope creep)?
  • Human: merge, amend, or reject — then log the outcome

Refs #442

🤖 Generated with Claude Code

https://claude.ai/code/session_01SeDHwsR2dnwCk8Btfhej2L

Jammy2211 and others added 6 commits October 24, 2026 12:48
…y; drop results_layout (#442 PR 3b)

build_dashboard.py reads the mirrored tree natively: a series is
<measurement>:<dataset_type>/<variant>:<instrument>:<setup_id>:<host_class>:<model_variant>,
a cell's comparison.json entries join the setup row they summarise, and
results/lens keeps its versioned grammar. catalogue/series_id_map.json,
generated from catalogue/migration_day_zero.json by build_series_id_map.py,
maps every pre-move series id to its new id (151 ids: 107 to v3 ids, 44 with
no mirrored member kept behind legacy:) and attaches the pre-move points
outside the mirrored tree, so all 144 published series keep their 158
releases; a test checks the continuity and that the map is current.

build_readme.py reads breakdown rows and sweep comparisons from the mirrored
tree (unresolved legacy rows listed as setup not recorded), the misc groups
and their archived history (labelled archived), and matches archived
baselines to the current cell through the map's cell table.

scripts/misc/tooling/results_layout.py and its imports are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeDHwsR2dnwCk8Btfhej2L
…irrored tree; guard test (#442 PR 3b)

Writers that composed old-layout results paths outside resolve_output_paths
now place through place_output_path / resolve_output_paths, or write to the
results/misc/<group>/ landing folder where a file must keep its name:
likelihood_runtime vmap probes (+ resolve_vmap_batch), point-source
*_summary_v* writers, fixed_light witnesses and numba scaling/draws/memo
defaults, nautilus_batch_capture, separation_rule, oracle_figure, bakeoff,
memo_after_summary, misc/latent, likelihood_runtime sweep.py/aggregate.py
(comparisons to the mirrored cell) and 15 HPC submits. Their readers
(aggregate, size_sweep_table, fixed_light_scaling_table,
gradient_mode_library_ab, lens/solver/_corpus) resolve through the new
_profile_cli helpers (legacy_landing_dir, legacy_read_dirs,
locate_legacy_result, ...). Committed tables and 13/14 comparisons
re-render byte-identical.

test_results_write_destinations.py fails any script under scripts/,
_profile_cli.py or hpc/ that names a results path outside the mirrored
tree (heuristic and exclusions in its docstring).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeDHwsR2dnwCk8Btfhej2L
…isons, writer and reader (#442 PR 3b review)

sweep.py now places every row itself with sweep_layout.placed_row (the day-zero
classification of <class>/<model>/<instrument>/<model>_<config>[_sparse].json
fed the instrument, precision and dense-route transformer the matrix fixes)
and passes the exact path as the new --output-path; aggregate.py reads the
same paths (rows a run wrote win over committed rows) instead of rebuilding
locations from instrument/precision hints. All 308 matrix rows get distinct
mirrored paths under their instrument; the 13 committed comparisons rebuild
byte-equal, including interferometer/{delaunay,pixelization}/sma and
datacube/delaunay/sma.

Comparisons are placed from the instrument their rows record, so the
two-level imaging/delaunay_nn cell goes to results/imaging/delaunay_nn/hst/.
Standalone leaf runs name a two-level runtime folder under --instrument and
the interferometric leaves record the transformer they hard-code, so a
config-named row cannot collide across instruments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeDHwsR2dnwCk8Btfhej2L
…ed (#442 PR 3b review)

The breakdown table lists, after the current v3 cells, every published row
with no v3 cell, latest per pre-move group, read where the day-zero move put
it under the name it kept: unresolved rows in results/breakdown/, the current
results/misc/likelihood_breakdown/ (2 datacube/inversion rows) and
archive/*/*/<slug>/breakdown/ (labelled archived). A group whose newest
release the map moved into a mirrored cell is shown by that cell's v3 row.
106 rows before the move-era re-key, 110 now; every old row is present.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeDHwsR2dnwCk8Btfhej2L
…iases and variant names (#442 PR 3b review)

test_results_write_destinations.py flattens each path expression into
segments (/ joins, .joinpath, os.path.join, Path/PurePosixPath args, split
constants, and names bound by a simple same-module NAME = <path>
assignment), so R = ROOT / "results"; R / "runtime" is caught. A
results/<dataset_type>/ path must name a variant scripts/<dataset_type>/ has
(results/imaging/pixelization fails). Self-checks cover each new form; what
it cannot follow (other modules, parameters, non-trivial bindings) is
documented. build_readme's pre-move map prefix becomes a LEGACY_ constant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeDHwsR2dnwCk8Btfhej2L
…ep_layout.py (#442 PR 3b review)

resolve_output_paths reads the instrument with getattr (callers pass a
minimal namespace) and only re-folders legacy results paths; the catalogue's
script navigation lists the new scripts/misc/likelihood_runtime/sweep_layout.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeDHwsR2dnwCk8Btfhej2L
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant