Affected Repositories

• PyAutoFit (primary)

Branch Survey

Suggested branch: feature/results-library-p0-integrity

Implementation Steps — PR 0a

1. autofit/tools/util.py:76-84 zip_directory: write <out>.zip.tmp → os.replace; ZIP_STORED for .png/.fits/.hdf5, DEFLATED otherwise.
2. non_linear/paths/abstract.py:503-524 restore() (today rmtree → extractall → os.remove(zip), BadZipFile re-raised): extract to a sibling temp dir and swap; remove the zip only when remove_files and it is this search's own; BadZipFile → rename <hash>.zip.corrupt_<utc>, warn, treat as not complete. New read-only open_result().
3. non_linear/paths/directory.py:204-208 is_complete: when output_path/.completed is absent, check <out>.zip for a .completed member + clean testzip(), cached by (path, size, mtime). fit() (abstract_search.py:302) still restores a complete zip because result_via_completed_fit reads extracted files; extraction-free reads are results-library phase 1.
4. non_linear/checkpoint.py (Checkpointer.load :176, NativeFileCheckpointer.load :347) + resume checks at search/nest/nautilus/search.py:535 and search/nest/dynesty/search/abstract.py:303: shared probe (h5py open for Nautilus; catch OSError/ValueError/RuntimeError/KeyError; sampler LinAlgError on resume) → move search_internal/ to search_internal.failed_<utc>/, start fresh, log once.
5. non_linear/timer.py: write under files/ behind has_timer, atomically; read fallback for legacy output.
6. Tests: killed-mid-zip resumes; two processes restore one zip concurrently; truncated-HDF5 and poisoned-dill checkpoints quarantine and start fresh; STORED members verified.

Implementation Steps — PR 0b

1. aggregator/aggregator.py:177 from_directory(..., on_error="skip"|"raise"): BadZipFile at :98/:280 → agg.errors (path, exception); prints at :149/:219/:374/:384/:503 → logging; empty result logs a warning naming directory + filters.
2. aggregator/summary/aggregate_csv/__init__.py: strict=True default (:19); fieldnames (:113) = union of rows; save (:138) via open_atomic, reports blank columns; add_label_column (:90) key-joined.
3. row.py sigma-1/sigma-3/max_lh regression tests on an asymmetric latent fixture; fix inverted post_fit_output docstring.
4. Tests: one corrupt zip → len(agg.errors) == 1; unknown column raises; union header; killed-mid-save leaves previous CSV intact.

Worktree file ownership (concurrent with hpc-campaign phase 1)

This task: tools/util.py, non_linear/paths/, non_linear/checkpoint.py, non_linear/timer.py, aggregator/. Shared touchpoint non_linear/search/abstract_search.py (resume lines only). hpc-campaign phase 1 merges after PR 0a.

Key Files

• autofit/tools/util.py, autofit/non_linear/paths/{abstract,directory}.py, autofit/non_linear/checkpoint.py, autofit/non_linear/timer.py, autofit/aggregator/aggregator.py, autofit/aggregator/summary/aggregate_csv/

Results library phase 0: artefact integrity and honest reads in PyAutoFit (shared with hpc-campaign) — one issue, PR 0a then PR 0b

Type: feature
Target: autofit
Repos:

• PyAutoFit
  Themes:
• results
• aggregator
• hpc
  Difficulty: large
  Autonomy: human-required
  Priority: high
  Status: formalised
  Consequence: judge
  Witness: PR 0a — a killed-mid-zip fixture resumes cleanly; two processes restore one zip concurrently and both read it; truncated-HDF5 and poisoned-checkpoint fixtures start fresh with the old state moved aside (search_internal.failed_<utc>/); .png/.fits/.hdf5 members are ZIP_STORED; PR 0b — a fixture tree with one corrupt zip aggregates with len(agg.errors) == 1; an unknown AggregateCSV column raises; the header is the union of rows; a killed-mid-save leaves the previous CSV intact; row.py sigma-1/sigma-3/max_lh tests pass on an asymmetric latent fixture; autolens_workspace_test smoke green for both PRs
  Review-minutes: 45
  Unattended: never
  Epic: results-library
  Phase: 0
  Shared-with: hpc-campaign
  Parent: draft/feature/autofit/results_library_epic.md
  Filed: 2026-10-28

Phase 0 of the results-library epic (ledger draft/feature/autofit/results_library_epic.md, §Phase 0;
report results_library_epic_report.md P1, P2, P11, P12, P13). Shared with hpc-campaign: its ledger
records this issue as its phase 0 #A and does not re-issue it (ruling F1, 2026-10-28). The fit() failure
record is not here; it is hpc-campaign phase 1. Scoping review:
/mnt/c/Users/Jammy/Workspace/Science/scratch/pyautolens_ii_campaign_results_scoping_2026-10-28/REVIEW.md §D.1.

One PyAutoFit issue, two PRs in order. Ground against PyAutoFit main at or after 8740092dc
(2026.10.26.1); the plan quotes the current lines (the ledger's 710f4b343 references are stale). Build on
open_atomic (tools/util.py), the Checkpointer/internal_state.py split (#1680) and the
"finalise archive then .completed" order (#1683).

PR 0a — artefact integrity

• zip_directory: write <path>.zip.tmp, then os.replace; ZIP_STORED for .png/.fits/.hdf5, text
  DEFLATED. A zip is complete or absent, never truncated.
• restore(): extract to a temp dir and swap; never delete the zip unless the caller owns the result
  (remove_files and this search's identifier); BadZipFile → rename <hash>.zip.corrupt_<utc>, log,
  continue as "not complete". Read-only open_result() for downstream stages.
• is_complete for a zipped result: .completed member + zipfile.testzip(), cached per path; fit()
  consults it before restore().
• Checkpoint validity probe in Checkpointer.load for Nautilus (h5py open; catch OSError, ValueError,
  RuntimeError, KeyError) and dynesty; on failure or a sampler LinAlgError, move search_internal/ to
  search_internal.failed_<utc>/, start fresh, log once.
• Timer files → <output_path>/files/ behind the has_timer capability; atomic write.

PR 0b — honest reads

• Aggregator.from_directory(..., on_error="skip"|"raise"), agg.errors (path, exception); an empty
  aggregator logs a loud warning naming path and filters; print → logging across the aggregator.
• AggregateCSV: strict=True default; union header; atomic save; labels joined by key; save() reports
  blank columns.
• row.py regression tests (sigma-1, sigma-3, max_lh) on an asymmetric latent PDF fixture; fix the
  inverted post_fit_output docstring.

Worktree file ownership (review §D.4, ruling F11)

This lane: autofit/tools/util.py, non_linear/paths/, non_linear/checkpoint.py, aggregator/,
non_linear/timer.py. Shared touchpoint non_linear/search/abstract_search.py (fit() consults
is_complete before restore()) lands in PR 0a; hpc-campaign phase 1 (non_linear/status/) merges after
PR 0a. Ships pending-release. Ledger entry in both epic ledgers on ship.