Skip to content

profiling(point_source): vertex-dedup A/B, RAL CPU + A100 rows, campaign phase 2 (PyAutoArray#568) - #301

Merged
Jammy2211 merged 9 commits into
mainfrom
feature/point-source-cpu-p2
Sep 24, 2026
Merged

Jammy2211 merged 9 commits into
mainfrom
feature/point-source-cpu-p2

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

Profiling evidence for point-source CPU campaign phase 2 (PyAutoArray#568). An interleaved in-process A/B cell compares the JAX PointSolver with the jnp.unique vertex dedup (control) against the library change that drops it (no-dedup). This PR adds the RAL CPU and RAL A100 rows, the phase-2 section of results/notes/point_source_cpu_campaign.md, and a README line.

Median ms/call, control → no-dedup:

Cell RAL CPU (AMD EPYC, 5 calls/round, job 350582) RAL A100 80GB fp64 (20×20 calls, job 350587)
simple, solved 24.00 → 5.38 (4.47×) 1.695 → 0.843 (2.01×)
simple, plain 23.76 → 5.39 (4.40×) 1.684 → 0.822 (2.05×)
vmap-4 31.31 → 11.91 (2.63×) 1.865 → 0.958 (1.95×)
cluster, solved 155.2 → 78.3 (1.98×) 4.206 → 2.457 (1.71×)
cluster, plain 153.9 → 77.7 (1.98×) 3.860 → 2.105 (1.83×)

All correctness gates are bit-identical on CPU and GPU: logL, solved positions, non-zero gradients and vmap. Compile time falls about 20% on CPU and about 40% for the simple cell on GPU. The GPU job imported PyAutoArray 25894d10, the head of the linked library PR.

Caveats:

  • The CPU A/B ran on EPYC, while phase 1 ran on Xeon, so only within-host ratios are quoted.
  • The CPU A/B used 5 calls per round, not 20.
  • The GPU protocol resolves about 4%: hash-identical library and no-dedup programs differ by 3–4%.

Scripts Changed

  • scripts/point_source/likelihood_breakdown/vertex_dedup_ab.py — new interleaved vertex-dedup A/B cell (control vs no-dedup body, correctness gates, HLO sort counts, body provenance captured at import time)
  • hpc/batch_cpu/submit_breakdown_point_source_vertex_dedup_ab_ral_cpu_fp64 — RAL ral-partition CPU submit
  • hpc/batch_gpu/submit_breakdown_point_source_vertex_dedup_ab_a100_fp64 — RAL A100 fp64 submit
  • results/breakdown/point_source/vertex_dedup_ab_{local_cpu,hpc_ral_cpu,hpc_ral_a100}_fp64.{json,png} — laptop witness, RAL CPU (350582) and A100 (350587) rows
  • results/notes/point_source_cpu_2026_09_23_ral_job_350582_vertex_dedup_ab.out, results/notes/point_source_cpu_2026_09_24_ral_job_350587_vertex_dedup_ab_a100.out — job logs
  • results/notes/point_source_cpu_campaign.md — phase-2 section (A/B, GPU regression check, decision)
  • README.md — dashboard line

Upstream PR

PyAutoLabs/PyAutoArray#569 — library-first: merge that PR before this one.

Test Plan

  • scripts/misc/tooling/build_readme.py --check passes
  • scripts/misc/wall/check_submits.py --check passes (9 wall-basis submits, 0 failing)
  • RAL CPU job 350582 and A100 job 350587 COMPLETED, every correctness gate Δ = 0
  • image_plane_solved profiling pin unchanged
  • CI green (profiling lint checks out the PyAutoArray main, so the no-dedup library arm is only exercised once the library PR merges)

Heart RED development override

Shipped under the AUTONOMY.md "Human override for Heart RED (development only)".

  • Authorization (live human, 2026-09-24, this session), faithfully quoted: asked to authorize the development-only override for PyAutoArray#568 after a GPU check, the user replied "GPU check and then I authorize". The GPU check passed (RAL A100 job 350587), so the authorization is in effect.
  • Scope: commit, push and open the pending-release PRs (PyAutoArray, then the linked autolens_profiling PR). No merge, no release. Merge needs its own human /prm with every required check green.
  • Exact current Heart RED reasons (pyauto-heart readiness, verdict RED, score 45):
    • release validation FAILED (stage integrate)
    • workspace validation not passing (4 failed, cloud#35579888156: autolens notebooks/cluster/modeling.ipynb, autolens notebooks/weak/a2744.ipynb, autolens scripts/cluster/modeling.py, +1 more)
    • manifest drift: hub organism blurb (organs present) — 7 mismatch(es) vs PyAutoMind/repos.yaml
  • Passed branch gates: pytest test_autoarray 1626 passed at 25894d10 (re-run at ship); +10 new JAX tests, including an HLO sort-count guard that fails on main (39 sort ops) and passes on the branch. Downstream autolens_workspace_test point-source jax_likelihood ×4 and jax_grad/gradient.py print likelihoods and gradients identical to main. The autolens_profiling image_plane_solved pin is unchanged. A/B correctness gates are bit-identical on CPU (RAL job 350582) and GPU (RAL job 350587). Profiling build_readme.py --check and check_submits.py --check pass.
  • This branch does not fix Heart. Heart stays RED for release purposes.

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

Jammy2211 and others added 9 commits September 23, 2026 17:06
…r (#568)

control (pre-fix jnp.unique body from PyAutoArray 11b93476, injected),
nodedup (post-fix flat table + arange) and library (installed, self-labelled
by HLO sort count) routes; fresh closures + jax.clear_caches() per route;
log L / positions / grad / vmap gates recorded, never asserted.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Clone of the phase-1 point-source submit: 8 CPUs, BLAS pinned to 1, identical
XLA_FLAGS, fresh per-job JAX cache, one full-protocol leg, --time=3:00:00 from a
measured 939 s laptop wall (headroom 2.0).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
control/nodedup median ratio: simple solved 6.00x [5.91, 6.21], plain 6.04x,
vmap-4 3.39x; cluster solved 2.29x [2.22, 2.33], plain 2.30x. All gates
bit-identical; library_matches=control (canonical PyAutoArray 11b93476).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…568)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…p A/B, RAL job 350582 (#568)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMSQRTc7Zthxj1pxRjDVaF
…me (#568)

inspect.getsource at the end of a long run reads the file as it is then;
the laptop witness, edited mid-run, recorded the wrong function texts.
Capture text + a co_code/co_consts hash at import and refuse a text that
is not the named function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(#568)

Same interleaved control/nodedup/library A/B as the CPU submit, fp64,
20 rounds x 20 calls, JAX_PLATFORMS=cuda with a GPU-backend assertion,
and the PyAutoArray feature-branch clone prepended to PYTHONPATH so the
library route is the shipped fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#568)

GPU regression check: nodedup 1.71-2.05x faster than control on every
row, library (branch PyAutoArray 25894d10) matches nodedup, all gates
bit-identical, gradients non-zero.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…587 (#568)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Sep 24, 2026
@Jammy2211
Jammy2211 merged commit 9375790 into main Sep 24, 2026
1 check passed
@Jammy2211
Jammy2211 deleted the feature/point-source-cpu-p2 branch September 24, 2026 08:51
@Jammy2211 Jammy2211 removed the pending-release PR queued for the next release build label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant