Phase I synthesis docs + fusion-sweep 256-row re-run + board hygiene (#4)

* Core vertical slice: docs/abi.md contract, native/lgj-abi, Java facade

Ships the fully verified core of the Panama x ndarray::simd x Valhalla
vertical slice (Phases A-E of the mission plan):

- docs/abi.md: the normative Rust<->Java ABI contract, written before
  either side was implemented so both could be checked against one
  frozen doc instead of each other.
- Five new ndarray::simd primitives (eq_u32_to_mask, gt_i32_to_mask,
  mask_and/mask_or(_assign), masked_sum_i32), added under ndarray's own
  W1a consumer contract.
- native/lgj-abi: the Rust ABI crate. Generation-checked handle
  registry, generic SoA fixture, bulk kernels routed exclusively
  through ndarray::simd, 14-symbol extern "C" surface. 72/72 tests
  green, clippy/fmt clean, and the registry's core safety check was
  disable-verified (short-circuited, confirmed exactly the two
  guarding tests go red, restored).
- java/: the Panama membrane (internal/ffm, never exposed publicly)
  and the public semantic facade (NativePattern/View/Predicate/
  Pattern/Mask). 132/132 checks green across 8 suites, including a
  reflection-enforced ApiSurfaceTest that mechanically proves zero FFM
  types ever reach a public signature, and a LazinessTest that
  empirically proves the thesis: building a chain costs zero
  crossings, evaluating it costs exactly one, independent of row
  count up to 1,000,000.
- .claude/: a 6-agent ensemble, 6 knowledge docs, and a full board
  (LATEST_STATE/STATUS_BOARD/AGENT_LOG/EPIPHANIES/TECH_DEBT/ISSUES/
  PR_ARC_INVENTORY/INTEGRATION_PLANS/CODEX_REVIEW_CHECKLIST), all
  scoped to this repo's actual seams.

A mechanical audit (D-LGJ-AUDIT) found and fixed the one real rule
violation before this commit: kernels.rs::simd_popcount was calling
the internal ndarray::hpc::bitwise path instead of the sanctioned
ndarray::simd re-export.

Deliberately NOT included: the Valhalla lab (valhalla-lab/) and the
Vector API benchmark harness (bench/) — still in flight, tracked as
open STATUS_BOARD.md rows, to land in a follow-up PR once reviewed
with the same rigor as this slice.

Generated by [Claude Code](https://claude.ai/code)

* Valhalla lab: three-truths method, causal isolation, 3 real reproducers

Completes D-LGJ-F. One experiment source (src/shared/), compiled twice
against real JDKs -- stable JDK 26 GA (record) and the official JEP 401
early-access binary (value record) -- via a self-verifying run.sh that
mechanically diffs the two Vocab.java files modulo the 'value' keyword
before trusting the A/B is honest.

Experiments: IdentityExperiment (semantic truth -- is identity actually
unobservable), FootprintExperiment (real per-object/array/field bytes
via allocation-delta + JOL where available), FfmAddressingExperiment
(is the wrapper free where it touches native memory), ThesisExperiment
(the mandatory headline: 65,536 rows as one native lane vs hydrated
Java objects, on both platforms).

Causal isolation via three additional run.sh passes: escape analysis
off, and UseArrayFlattening/UseFieldFlattening toggled independently --
isolates which flag actually drives the measured difference rather than
inferring it.

Three real Valhalla limitations reproduced and filed under
reproducers/, none of which changed the production API:
- R1: @NullRestricted field on an identity class is a VerifyError
  (javac's fault -- no source form expresses the required strict-field
  init order relative to super())
- R2: array flattening has a hard 8-byte payload cliff, confirmed via
  -XX:+PrintFlatArrayLayout. LaneId/Ordinal/MaskId (<=8B) flatten;
  RowRange/Row (16B) do not. This turns "Valhalla helps descriptors,
  not entities" from a hand-wave into a measured VM cutoff -- and
  RowRange landing on the wrong side is flagged as the one place the
  expectation was too optimistic.
- R3: the densest null-restricted array form is jdk.internal-only and
  generics erase flattening entirely; Foo! null-restricted type syntax
  confirmed not to parse, matching the earlier archaeology finding.

One real defect found and fixed before landing: IdentityExperiment and
the stable Platform called Class::isValue() directly on four vocabulary
types with a comment incorrectly claiming it was "final API on JDK 26"
-- it does not exist there at all, confirmed by a real javac failure.
Fixed by routing every query through Platform.isValueClass(Class<?>),
answered honestly per platform.

Generated by [Claude Code](https://claude.ai/code)

* Vector API bench: real JMH, cross-checked; the crossing does not always win

Completes D-LGJ-G, the mission's mandated "where does execution belong"
comparison -- measured, not assumed to favor the Rust crossing.

Real JMH 1.37 (fork+warmup+compiler-blackholes confirmed in the log, not
a hand-rolled loop -- that lives in valhalla-lab and is labelled as
such there). Four components, cost kept strictly separate per the
mission brief: A_DowncallOverhead (bare crossing, no work), B_SegmentAccess
(raw native-memory read throughput), C_ExecutionBoundary (native fused
plan vs Java Vector API vs Java scalar, swept 64 to 4,194,304 rows),
E_FusionAndPlanning (fused vs unfused vs the scalar reference kernel vs
plan-construction-only, swept 1-8 predicates). 50/50 rows, 0 failures.
Data.crossCheck() runs in @setup and throws if the three kernels
disagree on count or sum, so a faster-but-wrong Vector kernel could not
have won the comparison undetected.

The headline complicates the thesis honestly: for a single predicate
over one native lane, the Java Vector API -- reading the SAME native
MemorySegment zero-copy via IntVector.fromMemorySegment, no byte[], no
bounce buffer -- beats the native crossing at EVERY row count tested,
56.4x at small sizes down to 1.3-1.4x at 4M rows. A second crossover is
also real: native beats a plain Java scalar loop only past roughly
4,096-16,384 rows. Component E shows why this doesn't overturn the
project's premise: SIMD-vs-scalar is the largest lever measured
anywhere in this suite (10.8x-31.1x, growing with predicate count),
and fused/unfused land within this harness's own ~10% noise floor of
each other at 65,536 rows -- the fused plan's real value is the
structural one-crossing guarantee (already proven by LazinessTest),
not a large measured time saving at this scale. Verdict: the crossing
is worth paying for composed, multi-predicate work, not for reading
one predicate off one lane, where Java on the same memory is simply
faster.

RESULTS.md was hand-written from the raw CSV, then independently
cross-checked against summarise.sh -- a script the same PR ships that
mechanically regenerates every table from results/jmh-results.csv, so
a re-run's numbers can never silently drift from a hand-transcribed
table. Both productions agreed to 3 decimal places on every cell
checked.

Generated by [Claude Code](https://claude.ai/code)

* Phase I synthesis docs + fusion-sweep 256-row re-run + board hygiene

Four synthesis docs close D-LGJ-I: architecture.md (the four layers and
what each is provably responsible for), panama.md (manifest-over-header,
belt-and-braces ownership, restricted-method discipline), valhalla-lab.md
(three-truths synthesis, the 8-byte flattening cliff, zero API adoption),
execution-boundary.md (the measured crossover picture + three structural
hot-path facts, each checked rather than assumed).

The fusion sweep was re-run with a 256-row arm after the first pass's
'fusion does nothing' finding proved true only at 65,536 rows: at 256
rows x 8 predicates unfused/fused reaches 2.99x. RESULTS.md is rewritten
from jmh-results-merged.csv; TABLES.md is mechanically generated from
the same file so the two cannot drift.

MultiLaneColumn (ndarray::simd_soa) evaluated for the fixture kernels
and declined on two concrete API mismatches (64-byte-multiple constraint,
no u32 lane); earmarked for the future 512-byte row-store slice where it
fits by construction. Operator layout reference recorded on the board.

PR_ARC_INVENTORY backfilled for merged PRs 1-3; the lapse is owned in
the file itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Author: AdaWorldAPI

.claude/board/EPIPHANIES.md

@@ -4,6 +4,36 @@
 > `**Status:**`/`**Confidence:**` line. A correction gets its own new,
 > dated entry that references the one it corrects — the storno rule.
 
+## 2026-08-17 — E-LGJ-SIMD-SOA-IS-FOR-THE-ROW-STORE-NOT-THE-FLAT-LANES-1
+
+**Status:** DECISION (declined refactor, with the trigger for revisiting named).
+**Confidence:** High — decided by reading `ndarray/src/simd_soa.rs`'s full API, not by taste.
+
+Operator suggestion: "if you use SoA, calling simd_soa.rs would make sense" — should
+`native/lgj-abi/src/kernels.rs` route through `ndarray::simd_soa::MultiLaneColumn` (the canonical
+`Arc<[u8]>` SoA carrier) instead of raw `&[u32]`/`&[i32]` slices? **Answer: not for today's
+flat-lane fixture; yes for the future 512-byte row-store slice.** Two concrete API mismatches,
+not a style call:
+
+1. **No tail handling.** `MultiLaneColumn::new()` hard-requires `len % 64 == 0`; every `iter_*`
+   yields only full 64-byte chunks via `as_chunks::<64>()` — no remainder arm. The
+   `simd_int_ops` primitives this project consumes do the opposite by design: full 16-lane
+   groups + a scalar tail for arbitrary caller-chosen `n_rows`. Wrapping the fixture's lanes in
+   `MultiLaneColumn` would force 64-byte padding on every allocation, bought for nothing.
+2. **No `u32` lane.** `MultiLaneColumn` ships u8x64/f32x16/f64x8/u64x8/i32x16/i64x8 iterators —
+   no u32. The fixture's `ids`/`classes` are `u32` (`eq_u32_to_mask`).
+
+So `kernels.rs` already calls the correct layer: the `ndarray::simd_int_ops` primitives own their
+chunking internally. `MultiLaneColumn` sits *above* that layer, for uniform pre-padded columns.
+
+**Where it DOES fit — the operator-stated layout reference (recorded verbatim so it survives):**
+"the 64k x 512 bytes SoA layout is enforced everywhere in lance-graph (32 Lanes each 4 bytes
+classview+12 bytes). For Java the layout might differ — just for reference." A 512-byte,
+64-byte-aligned row store (32 × 16-byte V3 facets) is padded/aligned *by construction* — no tail
+problem — and each row is a natural `iter_u8x64` chunk-of-chunks. When the real
+`NodeRow`/facet slice replaces the generic fixture (`docs/abi.md` §10, `docs/architecture.md`
+"where a real graph slice would attach"), `MultiLaneColumn` is the type to reach for. Not before.
+
 ## 2026-08-17 — E-LGJ-VECTOR-API-BEATS-THE-CROSSING-1
 
 **Status:** FINDING. **Confidence:** High (real JMH 1.37, `Data.crossCheck()` guards every fork,

.claude/board/LATEST_STATE.md

@@ -1,3 +1,27 @@
+## 2026-08-17 (later) — Phase I docs written, fusion re-run merged, simd_soa question answered (PR #4)
+
+- **All four synthesis docs shipped** (`docs/architecture.md`,
+  `docs/panama.md`, `docs/valhalla-lab.md`, `docs/execution-boundary.md`)
+  — D-LGJ-I DONE. Each cites the proving artifact instead of restating it.
+- **Fusion sweep re-run with a 256-row arm** (`./run.sh E_`): the first
+  pass's "fusion does nothing" (true at 65,536 rows, where kernel time
+  dominates) is false at small rows — unfused/fused grows 0.95× → 2.99×
+  at 256 rows × 8 predicates, because per-crossing overhead dominates
+  there. `RESULTS.md` rewritten from `jmh-results-merged.csv` (A/B/C from
+  the full sweep + E from the re-run), `TABLES.md` mechanically generated
+  from the same file. Valhalla lab result files refreshed by a same-box
+  re-run; findings unchanged.
+- **`MultiLaneColumn` question answered** (operator: "if you use SoA,
+  calling simd_soa.rs would make sense"): declined for the flat-lane
+  fixture (64-byte-multiple constraint + no u32 lane — two concrete API
+  mismatches), earmarked for the 512-byte row-store slice where it fits
+  by construction. Operator layout reference recorded: 64K × 512 B rows,
+  32 lanes × (4 B classid + 12 B), enforced everywhere in lance-graph;
+  Java-side layout may differ. See
+  `E-LGJ-SIMD-SOA-IS-FOR-THE-ROW-STORE-NOT-THE-FLAT-LANES-1`.
+- **PR_ARC_INVENTORY backfilled** for merged PRs #1-#3 (hygiene lapse
+  owned in the file itself).
+
 ## 2026-08-17 — D-LGJ-AUDIT complete, core vertical slice VERIFIED GREEN, PR #1 opened
 
 ### Current Contract Inventory — the vertical slice is real and green

.claude/board/PR_ARC_INVENTORY.md

@@ -3,14 +3,56 @@
 # updatable in place — corrections append as new dated lines; reversals
 # get their own PR entry)
 
-_No PR has been opened against this repository yet — session 1, 2026-08-17._
+> **Hygiene lapse, owned (2026-08-17):** PRs #1-#3 merged without their
+> entries landing in the same commit — the exact retroactive-hygiene
+> anti-pattern the imported board rules name. Backfilled below in one
+> pass rather than left stale; PR #4 onward gets its entry at merge time.
 
-The first entry in this file will be written when the first PR against
-`lance-graph-java` merges (expected: the vertical slice on
-`claude/lance-graph-java-panama-valhalla-sus9w8` → `main`, once
-`ISS-LGJ-FANOUT-UNREVIEWED` closes). Until then, ground truth for
-in-progress work lives on `LATEST_STATE.md` (current contract inventory),
-`STATUS_BOARD.md` (per-D-id status), and `AGENT_LOG.md` (what actually
-happened) — this file stays empty rather than backfilled with a
-pre-registration entry that would misrepresent something as merged before
-it is.
+## PR #3 — Vector API bench: real JMH, cross-checked (merged 2026-08-17, squash)
+
+- **Added:** `bench/` — real JMH 1.37 suite (Components A/B/C/E:
+  downcall overhead, segment access, execution boundary sweep 64→4.2M
+  rows, fusion/planning), `Data.crossCheck()` gating every fork,
+  `summarise.sh` mechanical table generator, `RESULTS.md`, raw
+  run logs + CSV.
+- **Locked:** the headline finding — Java Vector API zero-copy on the
+  native segment beats the native crossing at every row count tested
+  (56.4× → 1.33×); native beats Java *scalar* only past ~4K-16K rows.
+  Recorded as `E-LGJ-VECTOR-API-BEATS-THE-CROSSING-1`.
+- **Deferred:** fusion sweep ran at 65,536 rows only (repaid post-merge
+  by the E_ re-run with a 256-row arm — see PR #4).
+- **Docs:** `bench/README.md`, board updates.
+- **Confidence:** High — 50/50 rows, 0 failures, two independent
+  computations of the same CSV agree.
+
+## PR #2 — Valhalla lab: three-truths, causal isolation, 3 reproducers (merged 2026-08-17, squash)
+
+- **Added:** `valhalla-lab/` — shared/stable/valhalla trees, self-verifying
+  `run.sh` (vocab-diff honesty gate + flattening-flag causal isolation),
+  `docs/three-truths.md`, reproducers R1/R2/R3 with observed outputs.
+- **Locked:** the 8-byte array-flattening cliff (R2, VM-confirmed);
+  native-one-crossing beats hydration ~38-57× on BOTH JDKs; production
+  API adopts zero Valhalla-only mechanisms — migration stays
+  `record` → `value record`, one word per type.
+- **Deferred:** nothing; the lab is complete for this vocabulary.
+- **Docs:** lab README + three-truths; board updates.
+- **Confidence:** High — one real defect (`Class::isValue()` not on
+  JDK 26) found by compile failure and fixed before landing.
+
+## PR #1 — Core vertical slice: ABI contract, native crate, Java facade (merged 2026-08-17, squash)
+
+- **Added:** `docs/abi.md` (normative, 14 symbols / 4 repr(C) types /
+  13 status codes / generation-checked handles); `native/lgj-abi`
+  (72/72, clippy/fmt clean, 14/14 exported symbols via `nm -D`);
+  `java/` facade + FFM membrane (132/132, reflection-enforced zero-FFM
+  public surface); 5 new `ndarray::simd` primitives under the W1a
+  contract (41/41); the `.claude/` ensemble + board.
+- **Locked:** disable-verified generation check (exactly 2 tests red
+  when broken, 70 green); the manifest cross-check rejects a real wrong
+  `.so`; laziness measured (0 crossings to build, exactly 1 to
+  evaluate); target-cpu=x86-64-v4 divergence recorded.
+- **Deferred:** real graph types (`NodeRow`/`WideFieldMask`) — generic
+  fixture first, by design (`docs/abi.md` §10).
+- **Docs:** `docs/abi.md`, knowledge docs, board.
+- **Confidence:** High — one real audit violation (`ndarray::hpc`
+  import) found and fixed pre-merge; recorded in EPIPHANIES.

.claude/board/STATUS_BOARD.md

@@ -16,7 +16,7 @@ list.
 | D-LGJ-F | Valhalla lab — three-truths method on the small semantic value vocabulary | **DONE 2026-08-17** — `valhalla-lab/`: 4 experiments + a self-verifying `run.sh` (mechanically diffs the two `Vocab.java`s modulo the `value` keyword before trusting the A/B) + 3 causal-isolation runs (escape-analysis off; `UseArrayFlattening`/`UseFieldFlattening` toggled independently). 3 real Valhalla limitations reproduced and filed under `reproducers/` (R1: `@NullRestricted` field on an identity class is a `VerifyError`, javac's fault — no source form expresses required strict-field order; **R2: array flattening has a hard 8-byte payload cliff, VM-confirmed via `-XX:+PrintFlatArrayLayout`** — `LaneId`/`Ordinal`/`MaskId` (≤8B) flatten, `RowRange`/`Row` (16B) do not, so "Valhalla helps descriptors not entities" is a measured VM cutoff, not a hand-wave, and `RowRange` landing on the wrong side is flagged as the one place the expectation was too optimistic; R3: the densest null-restricted array form is `jdk.internal`-only and generics erase flattening entirely — `Foo!` type syntax confirmed NOT to parse, matching the archaeology finding). 1 real defect found + fixed before landing (see `EPIPHANIES.md`). None of the three limitations changed the production API — the migration path stays exactly `record` → `value record` | I |
 | D-LGJ-G | Java Vector API comparative bench vs Panama→`ndarray::simd` | **DONE 2026-08-17** — real JMH 1.37 (fork+warmup+blackholes confirmed in the log), 50/50 rows, 0 failures, `Data.crossCheck()` guards every fork. **Headline (Component C, single predicate, zero-copy `IntVector.fromMemorySegment`): the Java Vector API beats the native crossing at EVERY row count tested, 64 to 4,194,304** — 56.4x at small sizes down to 1.3-1.4x at the largest. Native beats a plain Java scalar loop only past ~4,096-16,384 rows. Component E: SIMD-vs-scalar is the biggest lever measured (10.8x-31.1x); fused vs unfused are within noise of each other at 65,536 rows (crossing-count guarantee matters more than measured time here, since Component A puts one downcall at ~22ns). Independently cross-checked: hand-written `RESULTS.md` numbers verified byte-for-byte against `summarise.sh`'s mechanically-generated tables from the same CSV | I |
 | D-LGJ-H | Falsification: handle lifecycle (adversarial), SIMD/scalar parity, Java/native parity | **DONE 2026-08-17, all scopes closed** — Rust+Java core (D-LGJ-C disable-verification, D-LGJ-E `FusionParityTest`/`LifetimeTest`); Valhalla lab (`run.sh`'s vocab-honesty self-check + causal-isolation runs); bench (`Data.crossCheck()` on every fork, `summarise.sh` cross-check) | I |
-| D-LGJ-I | Docs: `architecture.md`, `panama.md`, `valhalla-lab.md`, `execution-boundary.md` | **Unblocked** — F and G both landed; next action | — |
+| D-LGJ-I | Docs: `architecture.md`, `panama.md`, `valhalla-lab.md`, `execution-boundary.md` | **DONE 2026-08-17** — all four written as synthesis (each cites the artifact that proves its claim rather than restating it); `execution-boundary.md` additionally records the three structural hot-path facts (zero-copy precision incl. the lance-graph `SoaEnvelope` inheritance, no-thread-pool/caller-threads-are-the-parallelism, `array_windows`/`array_chunks` precisely traced as un-invoked at any input size). Ships in PR #4 with the fusion-sweep 256-row re-run merged into `RESULTS.md`/`TABLES.md` | — |
 | D-LGJ-AUDIT | Mechanical post-fan-out audit: `grep` for `ndarray::hpc` imports, any `.h`/`cbindgen`/`jextract` artifact, any FFM type leaking into public Java API | **DONE 2026-08-17** — 1 real violation found (`kernels.rs::simd_popcount` used the internal `ndarray::hpc::bitwise` path), fixed in place; everything else confirmed to be the one sanctioned exception or explanatory prose | closed D-LGJ-C/D/E for the core |
 
 ### Reading this table