Skip to content

docs(solver): record phase 2 — PyAutoArray#595 forward polish — cells re-run, euclid validation re-based (PyAutoArray#594) - #357

Merged
Jammy2211 merged 4 commits into
mainfrom
feature/raw-pdip-forward-polish
Sep 30, 2026
Merged

Jammy2211 merged 4 commits into
mainfrom
feature/raw-pdip-forward-polish

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

Phase 2 of epic linear-solver-programme on the profiling side (PyAutoArray#594; library fix PyAutoArray#595 merged 2026-09-30 as 7a89e19a0, pending release). The library now returns the #573 polished iterate as the raw-forward PDIP forward value. This PR records that in the solver programme: every scripts/lens/solver/ cell re-run against merged library main (device.provenance.library_revisions names PyAutoArray 7a89e19a0), the euclid latent cell's validation leg re-based from the pre-fix jit value 3.511093374 to the post-fix 3.320127604 measured on main, the ledger's Phase 2 section, campaign page + index row, and regenerated README tables.

Measured on the 81-system corpus: fixed pdip_raw equals phase 1's pdip_raw_polish on every metric on 81/81 systems — converged 81/81 incl. 48/48 SLaM, forward iterations median 18 / max 24 (+1–7 polish, accepted on all), worst inactive-column flux 3.31e-4 (pre-fix 0.115), worst total-flux error 3.31e-4 (0.115), worst source-flux error 4.96e-2 on euclid only (16.4; ≤ 4.9e-5 on the #571 fixtures), objective gap 8.2e-16, KKT 3.2e-16. Euclid latent +7.47e-5 vs eager (pre-fix +5.76e-2); the euclid pipeline latent test passes on library main. Under the pre-registered rule the fixed pdip_raw passes criterion 1 and still fails 2–4 for the recorded rule weaknesses (euclid source-column proxy far stricter than the latent it stands for; one flat-direction system at 0.219 shared by every accurate candidate; 5/52 KKT ratios at the 3e-16 floor) — the ledger says so rather than re-basing the rule.

Artefacts: the cells stamp al.__version__ = 2026.8.17.1 on a source checkout, so the re-run overwrote the *_v2026.8.17.1.{json,png} artefacts in place; they are "current library main" by contract, and the pre-fix versions live at 3ad68af and in the ledger's phase-1 tables. pdip_raw_tol_1e-2 (direct solve_nnls, no polish) now reproduces the pre-fix pdip_raw; the phase-1 sentence claiming it reproduces pdip_raw is annotated, not deleted.

Upstream PR

Scripts Changed

  • scripts/lens/solver/euclid_latent.py — PDIP_RAW_JIT_EXPECTED re-based to the post-#595 library-main jit value (comment names the PR, merge sha and the replaced pre-fix value); the JSON now records both the released and the library-main jit values
  • scripts/lens/solver/_solvers.py — pdip_raw / pdip_raw_tol_1e-2 descriptions: polished forward value, forward-only iteration count; tol_1e-2 has no polish
  • scripts/lens/solver/README.md, results/README.md (via build_readme.py) — regenerated tables; scripts/misc/tooling/build_readme.py — euclid renderer shows both jit values
  • results/lens/solver/{accuracy,accuracy_posthoc,early_stopping}_summary_{all,slam_fixture_571}_v2026.8.17.1.{json,png}, euclid_latent_by_candidate_v2026.8.17.1.json — re-run against PyAutoArray 7a89e19a0
  • results/notes/linear_solver_accuracy_2026_09.md — Phase 2 section; wiki/campaigns/linear_solver_accuracy.md, wiki/index.md — status, verdict, PRs, phase-2 row + journal entry

Test Plan

  • ruff check . && ruff format --check . (250 files)
  • build_readme.py --check, check_wiki.py --check, check_results_layout.py --check, build_dashboard.py --check, check_submits.py --check
  • pytest scripts/misc/test -q — 984 passed, 5 skipped
  • AUTOLENS_PROFILING_SMOKE=1 — the 7 lint.yml smokes + early_stopping.py + euclid_latent.py, all rc=0
  • euclid_latent.py validation leg VALIDATES on library main (x_ref → eager rel −3.3e-8; pdip_raw → jit rel −2.0e-8)
  • lint.yml green on the PR (incl. lychee, not run locally)

Heart RED override (development only)

  • Authorization: live human in the Claude Code CLI session, 2026-09-30 ("do the workspace phase"; at the ship gate the human pushed feature/raw-pdip-forward-polish directly, ~18:55 BST, in response to the reported RED reasons). Scope: development shipping (push + this PR); merge only via a separate /prm on all-green required checks; no release.
  • Heart readiness at the 18:50 BST gate (verbatim): RED PyAutoGalaxy: CI failure; RED release validation FAILED (stage integrate); YELLOW workspace validation not passing (0 failed, 1 timeout, cloud#36404726969: autolens_test scripts/multi_dataset/rectangular.py); YELLOW manifest drift: public front-door organ tables (generated) — 1 mismatch(es) vs PyAutoMind/repos.yaml. None involve autolens_profiling. This PR does not repair Heart; Heart remains RED for release.
  • Passed branch gates at ad365df: ruff check + format (250 files); build_readme / check_wiki / check_results_layout / build_dashboard / check_submits --check; pytest scripts/misc/test 984 passed / 5 skipped; 9 smokes rc=0; euclid_latent validation leg validates on library main; in-session review (supervised, human-approved plan).

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

Jammy2211 and others added 4 commits September 30, 2026 17:53
…Array#595

The jit path now returns the #573 polished iterate, so the pdip_raw leg
expects the library-main jitted latent 3.320127603567922 (+7.47e-5)
instead of the capture-time released value 3.511093374207152 (+5.76e-2),
which stays in the corpus manifest as the capture record. The JSON and
the README renderer carry both values.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The candidate reproduces solve_nnls_primal_raw_forward, which since
PyAutoArray#595 returns the #573 polish of the raw forward iterate;
iterations still count the forward solve only. The tol_1e-2 row calls
solve_nnls directly with no polish, so it no longer equals pdip_raw.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… (#595)

accuracy (all + slam_fixture_571), posthoc, early_stopping (all +
slam_fixture_571) and euclid_latent re-run on the merged library;
provenance records PyAutoArray 7a89e19a0 in every JSON. pdip_raw now
equals the phase-1 pdip_raw_polish on 81/81 systems. The _v2026.8.17.1
artefacts are overwritten in place (source-checkout version stamp); the
pre-fix ones are at 3ad68af.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ge and README

Ledger Phase 2 section: library mechanism and #573/#594 lineage,
pdip_raw pre/post table, rule verdict per criterion, euclid latent
table, tol_1e-2 correction (phase-1 sentences annotated), artefact
overwrite note, what phase 3 inherits. Campaign page status, phase row,
shipped table and journal; wiki index row; regenerated README tables.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Jammy2211
Jammy2211 merged commit 4652580 into main Sep 30, 2026
1 check passed
@Jammy2211
Jammy2211 deleted the feature/raw-pdip-forward-polish branch September 30, 2026 17:59
Jammy2211 added a commit that referenced this pull request Sep 30, 2026
…rometer-decision-matrix

Resolve wiki/index.md: keep this branch's interferometer row and main's
linear-solver row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant