[michalharakal]

Problem

In a JVM decode profile of BitNet b1.58 2B4T (#1357), 61 % of all allocation came from FloatVector.fromArray0Template ← JvmVectorKernels.binaryFloat ← DefaultCpuOpsJvm.vectorFloatBinary: 9.6 GB in a 45 s window. That's the Vector API's Java fallback, not intrinsified code. There were two causes:

1. The op was passed as a lambda. binaryFloat/unaryFloat and vectorFloatBinary/vectorFloatUnary took the FloatVector op as Function2/Function1. One shared helper serves add/sub/mul/div/relu/…, so the call is megamorphic. The vectors escape into the generic invoke, and C2 can't box-eliminate them.
2. The species was an instance field. DefaultCpuOpsJvm.floatSpecies was an instance val. C2 constant-folds static final fields only, and the Vector API intrinsics need a constant species. relu alone allocated 3× its output because of this.

Change

• binaryFloat, unaryFloat, vectorFloatBinary and vectorFloatUnary are now inline, so each op gets its own monomorphic loop with the lambda body in place.
• DefaultCpuOpsJvm.floatSpecies reads JvmVectorKernels.floatSpecies (an object property, so a static final field) through an inline getter.
• The scalar-broadcast and last-dim (bias) broadcast loops reuse the same inline helpers: binaryFloat gains outOffset, plus new binaryFloatScalarLeft/Right. That removes about 60 lines of duplicated loop code.

All of it is internal API; apiCheck passes.

Evidence

• :skainet-backends:skainet-backend-cpu:jvmTest: 490/490 pass.
• Host microbenchmark, bytes allocated per op on 1×2560 FP32 after warm-up (output alone is 10 256 B):
  • relu: 31 016 → 10 536 B/op, 6.9 → 1.5 µs/op.
  • same-shape add/sub/mul/div: output-only in both builds when warmed in isolation.
  • Scalar and bias broadcast still showed fallback boxes in the isolated, lambda-driven probe on x86 under tiered compilation; with -XX:-TieredCompilation scalar-mul is output-only. I didn't chase that further, because what matters is the real decode below.
• Real decode on the board (Arduino Ventuno Q, 4× Cortex-A78C, Temurin 21, JFR settings=profile, 45 s decode window): FloatVector.fromArray0Template disappears from the allocation-by-site list (it was 61 %). Greedy 128-token output is byte-identical (md5 52f57d1d…).
• Decode speed is unchanged within noise (2.28 → 2.32 tok/s). On the JVM, the fallback mostly cost allocation, not wall time, since GC pauses are 4–10 ms. The win is allocation pressure, which matters more for Kotlin/Native and Android heaps.

Not in this PR: the next allocation sites the profile now surfaces

With the fallback gone, decode allocation is dominated by generic element-wise copies that spread get(*indices), which copies an IntArray and boxes a Float per element:

• RMSNormalization.eagerFusedRmsNorm → MmapFloatTensorData.copyToFloatArray() (TensorData default, per forward, on mmap'd norm gains): ~10 GB / 45 s. A bulk FloatBuffer override removes it; I verified that locally.
• DefaultCpuOpsBase.split (used by RoPE.applyRoPESplitHalfFull): ~7.8 GB / 45 s.

I'm happy to send those as follow-ups.

Refs #1357

🤖 Generated with Claude Code