Skip to content

refactor: move results/ into the mirrored tree (#442 PR 2) - #446

Closed
Jammy2211 wants to merge 3 commits into
mainfrom
feature/likelihood-lifecycle-p0-profiling-move
Closed

refactor: move results/ into the mirrored tree (#442 PR 2)#446
Jammy2211 wants to merge 3 commits into
mainfrom
feature/likelihood-lifecycle-p0-profiling-move

Conversation

@Jammy2211

@Jammy2211 Jammy2211 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

PR 2 of 6 for #442 (likelihood-lifecycle 0B): applies the day-zero migration map merged in #444.

DRAFT — hold until #441 merges. #441 edits results/notes/cluster_pointsolver_campaign.md and wiki/campaigns/cluster_pointsolver.md, both of which this PR moves. Once #441 is in:

  1. Branch fresh from main.
  2. Run build_migration_map.py and commit the regenerated map.
  3. Run apply_migration_map.py --apply and commit.
  4. Cherry-pick the docs commit and the path-adapt commit, taking docs(cluster): record the human decision — keep the 1.0″ starting grid (#439) #441's text at the new path, then rerun --rewrite-refs.
  5. Re-render the dashboard and re-pin the campaign hash if the registry changed.

There are no live RAL jobs on a RAL checkout of this repo; the only running job, checked 2026-10-23, belongs to autolens_inference.

Commits, reviewable separately

  1. 6d4ce975 — renames only. 1,384 git mv, every one R100. The 23 unresolved rows stay where they are; they are the map's unresolved array (8 files under results/breakdown/{cluster,datacube}, 15 under results/runtime/{datacube,interferometer,imaging}).
  2. c07b5c7e — docs and applier.
    • scripts/misc/tooling/apply_migration_map.py: --apply checks each file's sha and is safe to re-run; --rewrite-refs is idempotent.
    • results/README.md gets the layout description and an old → new redirect table.
    • A pointer README goes in each of the 14 emptied old folders, plus archive/README.md.
  3. 45b1cfcc — path adaptation, code and config only.
    • Readers and gates: a new results_layout.py lets readers see each moved file under its old path while linking to where it lives now. build_catalogue (which now also reads archive/**), build_dashboard, build_readme, check_wiki, check_results_layout and list_timing_assertions use it.
    • Writers: _profile_cli.resolve_output_paths writes default-folder runs into the mirrored cell, using the generator's own destination(). When no cell can be derived it falls back to results/misc/<group>/ with a printed note.
    • Indexes: provenance_grandfathered.txt, the registry references and evidence paths, and wiki_bindings.json.
    • Docs: links rewritten from the map.
    • Tests: new test_results_mirror_scripts.py (the phase-0 mirror witness) and test_apply_migration_map.py.

Data guard

  • No results/ or archive/ data file changed bytes.
  • Ledgers moved into wiki/campaigns/ show R88–R99 only because their links were rewritten.
  • dashboard/ and wiki/setups/ were regenerated: compared against main after swapping old paths for new, only path strings and the path-derived shard sha256 values differ.
  • Record and setup ids are unchanged, and the output is still profiling-summary@2 (the move to @3 is PR 3).

Writers not adapted yet (PR 3)

These still write the old layout:

  • Anything given an explicit --output-dir: sweep.py → aggregate.py, build_baseline.py, and the HPC submits that set RESULTS_DIR=…/results/breakdown/….
  • Scripts that build their own paths: the point_source_image runtime scripts, cluster/oracle_figure.py, and the fixed_light_numba_*_witness.py scripts.
  • Flat-folder aggregators: size_sweep_table.py, fixed_light_scaling_table.py and nnls_iterations_matrix.py. hazards/aggregate.py too: 3 of its inputs are archived, so a re-run would drop them.

Review asks

  1. Registry hash re-pin. baseline/campaign.json (still a draft) pins the sha of catalogue/registry.json, which changed (new paths plus the archive glob). It has been re-pinned; please confirm.
  2. Temporary reader shim. results_layout.py stays until PR 3. Until then, runs written into the mirrored tree appear on the catalogue and dashboard only weakly.

Checks

  • ruff check and ruff format --check: clean.
  • Every --check step in lint.yml passes: build_readme, build_catalogue --validate-with ../PyAutoPulse, check_wiki, build_dashboard, check_results_layout, check_submits and list_timing_assertions.
  • pytest: 1503 passed, 6 skipped (main: 1489 passed, 6 skipped).
  • Smoke run of 4 scripts: exit 0.

Tier: judge, so this merges through human /prm.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HhogiUPH3y6uuAVz17P89B

Jammy2211 and others added 3 commits October 23, 2026 21:33
@Jammy2211 @claude
refactor: move results/ into the mirrored tree (#442 PR 2, renames only)
6d4ce97 
git mv every entry of catalogue/migration_day_zero.json whose new path
differs from its old one (1,384 files): mirrored cells, history/scans/
experiments, results/lens/<component>, results/misc/<group>, archive/
and results/notes ledgers -> wiki/campaigns/<dataset_type>/<variant>/.
The 23 unresolved rows (setup axis not recorded) stay where they are.
No file content changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HhogiUPH3y6uuAVz17P89B
@Jammy2211 @claude
docs: day-zero move pointers, redirect table and the map applier (#442 …
c07b5c7 
…PR 2)

- scripts/misc/tooling/apply_migration_map.py: the re-runnable applier that
  made the previous commit (--apply: sha-checked git mv of every map entry;
  --rewrite-refs: map-driven reference rewrite, used by the next commit).
- results/README.md: the mirrored layout and the old-prefix -> new-location
  redirect table.
- A pointer README in each emptied results/ top-level folder (runtime/ and
  breakdown/ keep the 23 unresolved rows), and archive/README.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HhogiUPH3y6uuAVz17P89B
@Jammy2211 @claude
refactor: point readers and writers at the mirrored results tree (#442 …
45b1cfc 
…PR 2)

Path adaptation only; no result file changes bytes and the catalogue keeps
profiling-summary@2 (the @3 builder is PR 3).

- results_layout.py: readers see a moved file under its pre-move identity
  (iter_legacy / legacy_glob / current, from the day-zero map) and link it
  where it lives now. build_catalogue (+ archive/**/*.json source),
  build_dashboard, build_readme, check_wiki (ledgers under wiki/campaigns),
  check_results_layout (ledger trees, results/ + archive/),
  list_timing_assertions and the committed-row tests read through it.
  Generated dashboard/ and wiki/setups/ change in path strings only (and the
  shard digests that hash them); setup and record ids are unchanged.
- Writers: _profile_cli.resolve_output_paths places a run whose default folder
  is results/runtime|breakdown in its mirrored cell with the map generator's
  own classification (build_migration_map.destination, facts_from_doc), using
  the result dict as hints at 52 call sites; an explicit --output-dir is
  honoured verbatim. Misc-group writers point at results/misc/<group>/.
- Indexes (map-driven): provenance_grandfathered.txt, registry.json references
  and evidence paths, wiki_bindings.json; baseline/campaign.json re-pins the
  draft inventory hash of the changed registry.
- Docs: apply_migration_map.py --rewrite-refs repoints every literal old path
  and relative markdown link (wiki/, READMEs, hpc/ comments); four links that
  were already broken in the moved ledgers fixed by hand.
- Tests: test_results_mirror_scripts.py (results variants are a subset of
  scripts variants; script-only variants reported), test_apply_migration_map.py,
  nested-ledger check_wiki case, archive provenance case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HhogiUPH3y6uuAVz17P89B
@Jammy2211 Jammy2211 mentioned this pull request Oct 23, 2026
Merged
@Jammy2211

Jammy2211 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #447 — same change rebuilt on main after #441 merged (no force-push of a pushed branch).

@Jammy2211 Jammy2211 closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Jammy2211