Repository navigation
refactor: move job logs and JSON sidecars out of results/notes; provenance block on every result JSON #341
Description
Overview
results/notes/ is the ledger tree, but 16 .out job logs (5,311 lines) and 10 .json sidecars (5,406 lines) sit beside the 46 markdown ledgers and are 38 % of it, and recent measurement PRs are 16k–26k lines of per-repeat JSON. This task writes the artefact policy into results/README.md, moves the 26 files to where they belong with every ledger and wiki link rewritten (history preserved by git mv, nothing deleted), adds a lint-gate check that results/notes/ holds only markdown, and makes every result JSON carry a provenance block (host, loadavg at start and at write, SLURM job id, library revisions, dependency versions). Epic profiling-research-wiki follow-on; Mind prompt draft/refactor/autolens_profiling/notes_logs_and_sidecars_out_of_results_notes.md. Nothing is re-run and no script changes what it measures.
Plan
- Write the artefact policy into
results/README.md: logs underresults/logs/<campaign>/(or cited by job id and left on RAL), JSON sidecars beside the result JSONs they describe, committed result JSON summarised (per-row medians + CI + provenance) rather than per-repeat dumps, the notes-only rule, the two frozen evidence-pack folders as the named exception. git mvthe 10 sidecars toresults/breakdown/imaging/(every one describes a fixed-light trace or numba result there) and the 16 logs toresults/logs/point_source_image/(12) andresults/logs/point_source_source/(4); rewrite the links in the 12 ledgers and 5 wiki pages that cite them.- Add
scripts/misc/tooling/check_results_layout.py --check(stdlib) tolint.yml:results/notes/top level is.mdonly plus the two allowlisted folders, and every result JSON with adeviceblock carriesdevice.provenancewith the required keys unless it is in the grandfather manifest of pre-policy files. - Provenance block in
_profile_cli.py:provenance_dict()attached insidedevice_info_dict(), so the 60 cells that already record the device get it without per-script edits; reuseslikelihood_breakdown/provenance.pyfor the library revisions. - Tests for the check and the block;
check_wiki.py --check,ruff,build_readme.py --check, pytest and lychee stay green.
Detailed implementation plan
Affected Repositories
- autolens_profiling (primary; workspace-type repo)
- PyAutoMind (ledger only:
active.mdregistration, prompt move)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./autolens_profiling | claude/profiling-wiki-phase-2-b8vtjm (restarted from main 5deb704) |
clean |
| ./PyAutoMind | claude/profiling-wiki-phase-2-b8vtjm (= origin/main) | clean |
Suggested branch: claude/profiling-wiki-phase-2-b8vtjm (the session's designated branch, restarted from main after #340 merged).
Work Classification: Workspace (profiling repo; tooling + docs + file moves, no library change).
Worktree root: none — remote container clone.
Implementation Steps
- Moves (
git mv, filenames unchanged):results/notes/{certified_solver_phase_c1_job350768,certified_solver_policy_phase_b_job350588,fixed_light_numba_s4b_source_job343397,fixed_light_numba_s5_source_job343398,fixed_light_numba_s5b_source_job343413,fixed_light_numba_s6_source_jobs,hst_gpu_residue_phase2_historical_job343376,hst_gpu_residue_phase2_job344635,hst_gpu_residue_phase3_job350573,hst_gpu_residue_phase4_job350651}.json→results/breakdown/imaging/.results/notes/point_source_cpu_*.out(12) →results/logs/point_source_image/;results/notes/point_source_source_plane_*.out(4) →results/logs/point_source_source/. Addresults/logs/README.md(what a log is, how it is cited, the RAL-only alternative).- Rewrite links: ledgers
certified_solver_policy_phase_b_2026_09.md,certified_solver_phase_c1_lane_rate_2026_09.md,fixed_lens_light_s4_2026_09.md,fixed_lens_light_numba_memo_2026_09.md,fixed_lens_light_numba_memo_policy_2026_09.md,fixed_lens_light_numba_scaling_2026_09.md,hst_gpu_residue_phase{2_vmap,3_psf,4_logdet}_2026_09.md,point_source_cpu_campaign.md,point_source_source_plane_campaign.md; wiki pagescertified_positive_solver.md,fixed_light_numba_cpu.md,hst_gpu_residue.md,point_source_image_plane_cpu.md,point_source_source_plane.md(whichever cite a moved file).clipper_campaign/andpoint_source_cpu_2026_09_17_reported/stay where they are (frozen evidence packs with their own READMEs).
- Policy: new "Artefact policy" section in
results/README.md(+ alogs/row in the sections table): where logs, sidecars and summaries live; the summarised-JSON rule for new measurements; the provenance contract (field list, where the block sits, start/at-write loadavg semantics); the lint that enforces the notes-only rule and the provenance ratchet. - Provenance block: in
_profile_cli.pyadd_LOADAVG_AT_IMPORTandprovenance_dict()→{captured_at, host, slurm: {job_id, array_job_id, array_task_id}, loadavg_at_import, loadavg_at_write, profiling_revision, library_revisions (from likelihood_breakdown/provenance.source_revisions), library_versions, dependency_versions (jax, jaxlib, numpy, scipy, numba, nufftax, …), provenance_schema: 1};device_info_dict()gains"provenance": provenance_dict(). Every failure inside it degrades toNonefields, never raises. - Check:
scripts/misc/tooling/check_results_layout.py [--check] [--write-grandfather]with the two rules above;results/provenance_grandfathered.txtgenerated once (pre-policy result JSONs, one path per line, header explains the ratchet: entries may only be removed). New step inlint.ymlbesidecheck_wiki.py. - Tests:
scripts/misc/test/test_check_results_layout.py(tmp trees: clean, stray.out, stray dir, missing provenance, grandfathered) andscripts/misc/test/test_provenance_dict.py(keys, SLURM env pickup, no raise without git). - Validate:
check_results_layout.py --check,check_wiki.py --check,build_readme.py --check,ruff check/format --check,pytest scripts/misc/test -q; open the PR; merge stays human (/prm).
Key Files
_profile_cli.py—provenance_dict(),device_info_dict()scripts/misc/likelihood_breakdown/provenance.py— reused forsource_revisionsscripts/misc/tooling/check_results_layout.py— new gate;results/provenance_grandfathered.txt— manifestresults/README.md,results/logs/README.md— policy.github/workflows/lint.yml— new stepresults/notes/*.md,wiki/campaigns/*.md— link rewrites only
Original Prompt
Click to expand starting prompt
Move committed job logs and JSON sidecars out of results/notes/
- Work type: refactor
- Target: @autolens_profiling
- Epic: profiling-research-wiki
- Autonomy: human-required
- Filed: 2026-09-27
Original prompt
From the 2026-09-27 profiling review: results/notes/ holds 74 files / 28,474 lines, of which
16 .out job logs (5,311 lines) and 10 .json sidecars (5,406 lines) are 38 % of the "notes"
tree; recent measurement PRs are 16k–26k lines each, mostly result JSON (e.g. #327 = 24,100 lines
of JSON, #335 = 15.5k). The maintainer agreed this is a problem.
Scope
- Decide and document the artefact policy in
results/README.md:.outlogs go to
results/logs/<campaign>/(or stay on RAL and are cited by job id); JSON sidecars go beside the
result JSONs they describe, not undernotes/; committed result JSON is summarised (per-row
medians + CI + provenance block), not per-repeat dumps. git mvthe existing 26 files with the ledger links rewritten (andcheck_wiki.py, lychee
green). History is preserved by the move; nothing is deleted.- Add a lint-gate check that
results/notes/contains only.md. - Mandatory provenance block for every result JSON: host, loadavg at start/end, SLURM job id,
library revisions, dependency floors (nufftax etc.) — the fields the wiki index and the future
dashboard read.
Out of scope
Re-running anything; changing what any script measures.
Activity
Jammy2211 commentedon Sep 27, 2026
Shipped
mainat282076cf(headab1e4fd, 45 files, +1150/−25) on the human's/prmafterlint.ymlwent green..outlogs →results/logs/{point_source_image,point_source_source}/, 10 JSON sidecars →results/breakdown/imaging/(git mv; ledger and wiki links rewritten).results/notes/is markdown-only again.results/README.md(+results/logs/README.md);check_results_layout.py --checkinlint.ymlwith the 530-file grandfather manifest;_profile_cli.provenance_dict()attached asdevice.provenancebydevice_info_dict().complete/2026/09/profiling-notes-sidecars-out.md.Generated by Claude Code