Skip to content

results: A100 A/B of the compacted ConstantSplit assembly (PyAutoArray#537, 12x) - #231

Merged
Jammy2211 merged 3 commits into
mainfrom
feature/delaunay-nn-constant-split-assembly
Sep 8, 2026
Merged

Jammy2211 merged 3 commits into
mainfrom
feature/delaunay-nn-constant-split-assembly

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

The autolens_profiling half of the DelaunayNN split-regularization assembly
compaction (PyAutoLabs/PyAutoArray#536): the bench that priced every candidate
assembly, the real HST stencil tables it runs on, the A100 A/B submits, and the
results note that records the verdict.

The bench is what chose the design rather than confirming it. On real HST tables
it priced the alternatives — dense GEMM 10.1 ms (a wash with today's scatter),
BCOO 21 ms, dedup/segment-sum 39 ms — and swept the compact width (12: 0.58 ms,
16: 1.47, 20: 2.78, 24: 4.53, 28: 6.71, 32: 9.31, 33 uncompacted: 10.03), which
is where SPLIT_REG_COMPACT_WIDTH = 12 comes from.

A100 A/B (jobs 342334-342338, same node, same window)

measurement before after speedup
assembly row (ms/call) 10.03 0.84 12.0x
params -> H prefix, vmap 16 (ms/call) 16.423 7.260 2.26x
params -> H prefix, unbatched (ms) 24.341 15.189 1.60x
full likelihood, single JIT (ms) 75.6 66.1 1.14x
full likelihood, vmap 16 (ms/call) 50.04 40.86 1.22x

Pinned likelihood 29144.581944 bit-identical on both legs, pinned_drift: [].
The pre-registered witness (assembly < 3 ms, params -> H < 11 ms) is met.

Scripts Changed

  • scripts/misc/delaunay_nn/assembly_bench.py — new: V0..V6 assembly variants
    (uncompacted scatter, compact scatter + wide-row supplement, dense GEMM, BCOO,
    dedup/segment-sum), compact-width sweep, run against real tables.
  • scripts/misc/delaunay_nn/build_tables.py — new: rebuilds the frozen stencil
    tables the bench consumes.
  • scripts/misc/delaunay_nn/extract_mesh_points.py — new: extracts the HST mesh
    points the tables are built from.
  • scripts/misc/delaunay_nn/README.md — new: how to rebuild and re-run.
  • hpc/batch_gpu/submit_delaunay_nn_assembly_bench_a100 and the four A/B
    breakdown/likelihood submits — new A100 submits (assembly + control legs).
  • results/delaunay_nn/{tables_hst_1500.npz, mesh_points_hst.npy, assembly_bench_a100_fp64.json} — frozen inputs and bench results.
  • results/{breakdown,jit}/...delaunay_nn_hpc_a100_fp64_assembly{,_control}.{json,png}
    — the A/B results.
  • results/notes/delaunay_nn_constant_split_assembly.md — the results note:
    design, the priced alternatives, the width sweep, the A/B verdict, the cap
    audit that sizes the wide-row budget.

Results and notes only apart from the new bench scripts; no existing profiling
script or result is modified.

Upstream PR

PyAutoLabs/PyAutoArray#537

Library-first gate: merge after PyAutoLabs/PyAutoArray#537. The bench's V1
variant mirrors the compacted library path, and the note's verdict describes
behaviour that only exists once the library PR lands.

Test Plan

  • ruff check scripts/misc/delaunay_nn/ — all checks passed.
  • ruff format --check scripts/misc/delaunay_nn/ — 4 files already
    formatted.
  • assembly_bench.py executed on the A100 (job 342338) and on CPU; results
    committed under results/delaunay_nn/.
  • A/B breakdown + likelihood submits ran on the same A100 node and window
    (jobs 342334-342338), pinned likelihood bit-identical.

Links

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw

Jammy2211 and others added 3 commits September 8, 2026 09:01
…les, rebuild scripts (PyAutoArray#536)

`assembly_bench.py` times alternative formulations of the DelaunayNN
`ConstantSplit` regularization-matrix assembly (`reg_split_from` +
`pixel_splitted_regularization_matrix_from`) on the real (4P, K) split stencil
tables from the HST cell, unbatched and at vmap 16, fp64. `V0` is the pre-#536
full padded (4P, K, K) scatter and the reference every variant is checked
against; `V6_library` is the shipped library path with its default compaction,
so the bench measures what production actually runs.

`extract_mesh_points.py` dumps the HST DelaunayNN source-plane mesh points and
`build_tables.py` turns them into the committed
`results/delaunay_nn/{mesh_points_hst.npy,tables_hst_1500.npz}`, so the bench
runs anywhere without a GPU or the imaging dataset.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw
…tion (PyAutoArray#536)

Five submits, mirroring the #532/#533 launch-latency A/B: the breakdown and
runtime cells of imaging/delaunay_nn x hst x fp64 on the control (merge base
180c8a40) and feature (04152bb7) private PyAutoArray checkouts under
/mnt/ral/jnightin/PyAuto_wt/delaunay-nn-constant-split-assembly/, plus a direct
assembly bench that times the assembly function itself on the committed HST
split tables (V0 vs V6_library) with no prefix-attribution caveat.

Same flags (--split-setup --vmap-batch 16), same env, PYAUTO_SIBSON_QUERY_CHUNK
left unset on every leg, shared /mnt/ral/jnightin/PyAuto install untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw
…y#536)

Five jobs (342334-342338) back-to-back on euclid-ral-gpu-2 inside one 8.4-minute
window, control 180c8a40 vs feature 04152bb7 on private PyAutoArray checkouts,
shared /mnt/ral/jnightin/PyAuto install untouched.

params->H prefix 24.34 -> 15.19 ms unbatched (1.60x) and 16.42 -> 7.26 ms per
call at vmap 16 (2.26x). The assembly measured directly on the real split
tables: 10.03 -> 0.84 ms per call at vmap 16 (12.0x), corroborated to 5% by the
breakdown's own H step row (10.05 -> 0.80). Whole likelihood at vmap 16
50.04 -> 40.86 ms per call (1.22x). The 29144.581944 pin is bit-identical on
both legs, pinned_drift [] on both. Both halves of the witness are met.

The note records honestly that the --split-setup differenced "H, ConstantSplit
assembly" cell goes negative (-1.60 ms) on the feature leg once the interval is
smaller than the compile-boundary materialisation slack, and re-costs Phase B
against the new numbers: the assembly is now 11% of the params->H prefix and
Sibson 89%, reversing #533's ordering.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Sep 8, 2026
@Jammy2211
Jammy2211 merged commit c7c07ae into main Sep 8, 2026
1 check passed
@Jammy2211
Jammy2211 deleted the feature/delaunay-nn-constant-split-assembly branch September 8, 2026 13:54
@Jammy2211 Jammy2211 removed the pending-release PR queued for the next release build label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant