Repository navigation
A100 re-measure: JAX Delaunay walk early exit (PyAutoArray#531) - #224
Merged
Merged
Conversation
Same-node same-session A/B on euclid-ral-gpu-2 (jobs 342315-342320), control = private PyAutoArray bcd15cd9 checkout, feature = 6152bdf0 (feature/delaunay-walk-early-exit); the shared /mnt/ral/jnightin/PyAuto install was left untouched while the subhalo/Euclid CPU runs were live on it. Delaunay params->H prefix 45.149 -> 7.228 ms unbatched (-37.9 ms, 6.25x) and 7.555 -> 5.139 ms per call at vmap 16; whole-likelihood single-JIT runtime 93.463 -> 60.355 ms (1.55x). DelaunayNN, which shares the walk but not the concatenation, 178.971 -> 144.789 ms (1.24x). Both EXPECTED_LOG_EVIDENCE_HST pins held exactly (29110.920858 / 29144.581944). Also records the H-row attribution shift in the breakdown script: since #531 the split-point walk rides inside prefix 6, so the step-11 subtraction is the regularization assembly alone and can read ~0/negative. The aggregate to quote across library versions is the params->H prefix (regularization_matrix_prefix_s). Docstring + comment note only, no logic change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B5HT8dp7sWc9qDhZp6moGr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A100 re-measure of the JAX Delaunay walk early exit (PyAutoArray#530 / #531): a
same-node, same-session A/B on
euclid-ral-gpu-2, jobs 342315–342320, all sixinside one 14-minute window on 2026-09-07. The witness asked for a ≥ 18 ms unbatched
drop in the params→H prefix; the measured drop is −37.9 ms, 2.1× the target.
The A/B differs in PyAutoArray and nothing else: control is a private
bcd15cd9checkout (the branch's merge base), feature is
6152bdf0(
feature/delaunay-walk-early-exit). The shared/mnt/ral/jnightin/PyAutoinstall was deliberately not touched — the subhalo-validation and Euclid DR1 CPU runs
(342299, 342301, 342311, 342314) were live on it throughout — so both legs prepend
their own PyAutoArray checkout to
PYTHONPATHand printautoarray.__file__plus thecheckout's
git rev-parse HEADinto the SLURM log. (The shared install sits ate36a5af4, eight commits behind the merge base, which is why the control leg is aprivate checkout rather than the shared tree.)
Delaunay — HST / Hilbert-1500 / MGE-60 /
ConstantSplit, fp64ms per likelihood call;
vmap/16is batch time / 16.vmap16 per callDelaunayNN sibling — shares the walk, not the concatenation
Sibsoncallspix_indexes_delaunay_walk_fromtwice (sibson.py:522,sibson.py:648),so it gets the early exit and the unchunked walk but not the single-call saving.
Pins passed unchanged
EXPECTED_LOG_EVIDENCE_HSTheld exactly on every leg — 29110.920858 for Delaunay(342315/342316, and
pinned_drift: []on both runtime rows) and 29144.581944 forDelaunayNN (342317/342318). The mapping is bit-identical to the eager NumPy reference on
both sides of the change.
The H-row attribution caveat
The H row moved meaning with #531; the params→H prefix did not. The breakdown attributes
step 11 as
t(params → H) − t(params → interpolator outputs). Before #531 a prefixstopping at step 6 never asked for the
ConstantSplitsplit points, so XLAdead-code-eliminated that second walk out of the interpolator prefix and its whole cost
surfaced in the H row's subtraction. The feature concatenates both walks into one locate
call, so the split walk now runs inside prefix 6 — inside "Triangulation +
interpolation" — and the subtraction leaves only the regularization assembly. Hence
H: 13.555 → 0.225 ms, and a small negative would have been equally legitimate.
Quote
regularization_matrix_prefix_s(params→H) across library versions, not eitherrow alone. The NumPy/scipy path is untouched by #531 and still charges the split walk
to H. The one script change in this PR records exactly that, as a docstring paragraph
and a comment above
_setup_prefix_fn— no logic change.Likewise, the +3.878 ms "Blurred mapping matrix (PSF)" row in the four-way split is a
by-difference artifact, not a regression: the same row under
vmap16, where theprefixes are measured independently, is flat (8.399 → 8.343 ms).
Phase 2 recommendation — do not start it
Do not start Phase 2 (static image-plane seed + fan test) on the Delaunay path.
After #531 the Delaunay params→H prefix is 7.23 ms unbatched and 5.14 ms per call at
vmap16 — about 13% of the 39.5 ms batched per-evaluation cost, so Phase 2'sremaining headroom on this cell is a few ms per call for a change that reaches into
Mapper/AdaptImages plumbing. DelaunayNN still spends 144.8 ms unbatched / 24.4 ms per
call in the same prefix, and the cheapest cut there is the one Delaunay already took:
concatenate
sibson.py's two walk calls into one. Filed as the follow-up promptPyAutoMind/draft/feature/autoarray/sibson_single_concatenated_walk.md.Scripts Changed
scripts/imaging/likelihood_breakdown/delaunay.py— module-docstring paragraph and acomment above
_setup_prefix_fnrecording the post-#531 H-row attribution shift anddirecting cross-version comparison to the params→H prefix. Documentation only, no
logic change;
ruff format --check/ruff checkclean.Results Added
results/notes/delaunay_walk_early_exit.md— the full note (provenance table, revisiontable, per-row A/B, the caveat, the Phase 2 recommendation). Source of truth for every
number above.
results/breakdown/imaging/delaunay_hpc_a100_fp64_{control_2026_09_07,walk_early_exit}.{json,png}results/breakdown/imaging/delaunay_nn_hpc_a100_fp64_{control_2026_09_07,walk_early_exit}.{json,png}results/runtime/imaging/delaunay/delaunay_hpc_a100_fp64_{control_2026_09_07,walk_early_exit}.jsonhpc/batch_gpu/submit_{breakdown_imaging_delaunay,breakdown_imaging_delaunay_nn,runtime_imaging_delaunay}_a100_hst_fp64_{control,walk_early_exit}Breakdown flags
--split-setup --vmap-batch 16; runtime cell full timing only.xla_flagsrecorded identically on all six rows(
--xla_disable_hlo_passes=constant_folding --xla_gpu_autotune_level=0), so these arecomparable with the 2026-09-05 rows in
delaunay_nn_breakdown.mdand not with the2026-07-10 rows in
preopt_breakdown_baseline.md. The pre-existingdelaunay_hpc_a100_fp64.json/delaunay_nn_hpc_a100_fp64.jsonrows are left untouched;the control legs here supersede them for comparison against this branch.
Upstream PR
Library-first gate. These results were produced by the feature library
(
6152bdf0) against its own merge base, so they are valid regardless of merge order —this PR records a measurement, it does not import the new API. It carries
pending-releaseand merges after PyAutoArray#531 like any other pending-releaseworkspace PR.
Test Plan
python -m py_compile scripts/imaging/likelihood_breakdown/delaunay.pyruff format --check+ruff checkon the changed script (the repo's lint gate; note the file is not black-clean atmaineither — black is not this repo's formatter)python scripts/misc/wall/check_submits.py --check— 55 submits with a wall-basis contract, 0 failingpython scripts/misc/tooling/build_readme.py --check— 64 versioned artifacts scanned, all READMEs unchangedjson.loadhpc/sync.conf(gitignored) not staged; 18 paths added by nameGenerated by the PyAutoLabs agent workflow.
🤖 Generated with Claude Code
https://claude.ai/code/session_01B5HT8dp7sWc9qDhZp6moGr