Skip to content

feat(wasm): convolutions as im2col + GEMM in the SIMD128 sidecar - #1428

Merged
michalharakal merged 6 commits into
SKaiNET-developers:developfrom
michalharakal:feature/wasm-conv-im2col
Oct 20, 2026
Merged

michalharakal merged 6 commits into
SKaiNET-developers:developfrom
michalharakal:feature/wasm-conv-im2col

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Summary

Seventh step of the JVM ↔ wasm parity series (after #1363, #1364, #1368, #1369, #1426, #1427), and the last compute gap on the list: convolutions. On the JVM conv1d / conv2d / conv3d run as im2col + the Vector API (JvmVectorKernels); in a browser they were the common per-element loop — the only FP32 op a forward pass spends time in that was still scalar Kotlin on wasm after the fused-op step.

skainet_f32_conv (f32_conv.c, shared native tree, every native build) — one kernel for all three ranks over three spatial dims (a lower rank passes the identity geometry for the leading ones). Per batch and group it gathers the receptive fields patch-major into a caller-provided scratch (col[p, :] is patch p's window, zero in the padding; whole rows copied with memcpy when the width dilation is 1) and multiplies them against the weight rows with the existing skainet_fp32_matmul_wt — every output is a contiguous dot product of a weight row and a patch row, so the SIMD128 / NEON body that kernel already has serves the convolution, accumulated in ascending k. Gather and GEMM thread over patches through the shared row pool; a bias of [c_out] is added in the kernel. Groups (depthwise included), stride, padding and dilation are all geometry. The 22-int geometry block travels as one pointer (SKAINET_CONV_GEO_INTS).

WasmCpuOps.conv1d/conv2d/conv3d — dense FP32 input, weight and optional bias ([c_out] or [1, c_out, 1, …]) go to the sidecar: the weight is a parameter and stays resident after its first call (denseWeightHandle), a heap input is staged per call and a resident one read in place, the output is resident under the usual budget — so conv → relu → … never leaves linear memory. Anything the common validation would reject stays on the common path, which raises its message. Sidecar rebuilt (SidecarArtifacts.kt); exports grow by one.

Matrix, bench, docs — the Kernel packs table gains Float32 conv1d / conv2d / conv3d (im2col + GEMM): panama-vector on the JVM (the one row whose JVM cell is the ops layer itself, not a provider — noted in the test), wasm-simd128 on JS/WASM. engine-conv2d (a 3 × 3 same-padding layer, 64 → 64 channels over 56 × 56, with bias) joins jvm-cpu-publish and wasm-bench with identical shapes. eager-execution.adoc states what still runs as common Kotlin on wasm (memory movement: reshape / permute / concat / gather / unfold / pooling, and the integer and narrow-float element paths); the browser-lane page's status line and the CHANGELOG carry the step.

Verification

  • :skainet-backends:skainet-backend-wasm-simd:wasmJsNodeTest (13/13) — new convolutions run as im2col and GEMM in the sidecar and match the scalar lane: conv2d with and without bias and with the 4-D bias form, a 2 × 3 kernel with stride, dilation and asymmetric padding, depthwise (groups = c_in), two groups with stride, 1 × 1; conv1d with stride, dilation and groups; conv3d with stride and padding; a conv → relu chain and a resident input, every output resident and the input never downloaded; the weight resident after its first call and not re-uploaded by the second. All against DefaultCpuOps on the same inputs.
  • :skainet-backends:skainet-backend-native-cpu:jvmTest — KernelSupportMatrixTest (the CMake build compiles the new file into libskainet_kernels), NativeFusedKernelsTest; generateKernelMatrix; the host gcc -Wall -Wextra compile of f32_conv.c is clean.
  • scripts/pr-gate.sh --quick, full gate.

Measured

engine-conv2d, batch 1, 64 → 64 channels, 56 × 56, 3 × 3, pad 1, bias (231 MFLOP per call); 12-core laptop.

Tier GFLOP/s
wasm scalar (before) 0.04
wasm simd128-st 6.76
wasm simd128-mt (4 workers) 22.0
JVM panama (im2col + Vector API, DefaultCpuOpsJvm) 10.7
JVM ffm (same path — no native conv binding on the JVM) 11.6

The sidecar convolution lands at roughly half the sidecar's dense GEMM rate on the same tier (14.1 st / 43.6 mt for the 1024³ GEMM): the gather is memory-bound and the dot kernel is the one-row-at-a-time x · Wᵀ body, not the 4 × 8 register-blocked GEMM. The JVM path is single-threaded and keeps its Vector API dot per output; the four-worker sidecar now outruns it. Binding skainet_f32_conv on the JVM through FFM (an Fp32ConvKernel face beside the fused-op faces) would close that in the other direction; left out of this PR as agreed for the series — noted as a follow-up.

…l on the JVM

Wasm: the sidecar exports skainet_ternary_f32_gemv, skainet_ternary_lmhead_stage1 and
skainet_bitnet_gemv_tq2_0. The portable branch of skainet_ternary_f32.c (MSVC, Emscripten) holds
the vendored decode table as a compile-time constant with a SIMD128 body and threads over output
rows through the shared row pool; bitnet_gemv.c gains a SIMD128 block body and the same row
threading on every tier. WasmTernaryKernelPack (installed by WasmSimd.install) registers the three
native faces under the dispatch keys the JVM FFM and Android JNI ternary packs serve.

JVM: Fp32MatmulWtKernel / KernelProvider.matmulFp32Wt() and the JVM-only Fp32MemSegMatmulWtKernel /
MemSegKernelProvider.matmulFp32WtMemSeg() expose skainet_fp32_matmul_wt; NativeFp32MatmulWtKernel
binds it with a heap and a segment face. DefaultCpuOpsJvm.matmulWeightTransposed multiplies a dense
[out, in] FP32 weight in place over a resident off-heap weight instead of transposing and caching a
copy. WasmKernelProvider exposes the same accessor.

Kernel support matrix: a fourth table, "Kernel packs (view dispatch)". Bench: engine-fp32-linear
on both harnesses. Docs: eager-execution ternary rows and x·Wᵀ row; the wasm ternary gap is closed.
…ion and reduction kernels

PlatformStorage.offHeapAllocator (Kotlin/Wasm) binds MemoryDomain.HOST_OFFHEAP to a backend's
memory. WasmSimd.install() binds it to the SIMD128 sidecar's linear memory: WasmLinearStorage
(Storage.OffHeap: pointer, bulk copies, slices, close frees) and WasmLinearTensorData
(FloatArrayTensorData whose heap mirror is downloaded once on first access), so a ForwardScope
frees sidecar activations at reset() and StorageCapabilities reports off-heap support in a
browser. Ambient outputs are freed through a FinalizationRegistry; WasmSimd.residentBudgetBytes
bounds what stays resident.

WasmCpuOps keeps matmul / matmulWeightTransposed outputs resident and runs add/subtract/multiply/
divide, relu/sigmoid/silu/gelu/erf/tanh and sum/mean in the sidecar on resident operands
(transcendentals on heap operands too); two heap operands keep the common path. toResident()
uploads an input once.

New shared C kernels in f32_ops.c (skainet_f32_binary/unary/reduce_all/reduce_axis; SIMD128 and
NEON bodies, row-pool chunks), compiled into the native library and the sidecar. wasm-bench gains
the laya.chain case and resident-operand variants of the add and sum scenarios.
…sm sidecar and the JVM FFM provider

New shared C kernels in f32_fused.c — skainet_f32_softmax_rows, skainet_f32_layer_norm,
skainet_f32_rms_norm and skainet_f32_sdpa (grouped-query heads, causal, sliding window with
rightContext, per-batch key lengths, dense additive mask with trailing-aligned broadcast) —
with SIMD128 and NEON bodies and row-pool threading over query rows. SPI faces
Fp32SoftmaxKernel / Fp32NormKernel / Fp32AttentionKernel behind KernelProvider.softmaxFp32() /
normFp32() / attentionFp32() (supports("softmax" | "layer_norm" | "rms_norm" | "sdpa",
["Float32"])), served by NativeKernelProvider through FFM and by WasmKernelProvider.

DefaultCpuOpsJvm routes softmax/logSoftmax over the last dimension and scaledDotProductAttention
to the native kernels when the FFM provider is loaded; WasmCpuOps runs them in the sidecar with
resident outputs. The common SDPA's validation moves into protected helpers both overrides share.
FusedNormOps (lang-core) is an optional interface beside TensorOps that LayerNormalization
consults for FP32 input; only the sidecar and the JVM FFM op sets implement it, so recording and
differentiating op sets keep the decomposition.

Also fixes a latent loader failure: GCC with -ffast-math vectorised the expf/tanhf/erf loops of
f32_ops.c into libmvec calls the shared library is not linked against, which terminated the
process at the first JVM call into such a kernel; the transcendentals now go through opaque
scalar wrappers and libskainet_kernels.so has no libmvec reference.

Bench: engine-softmax, engine-layer-norm, engine-sdpa on both harnesses. Kernel matrix: three
fused rows. Docs, CHANGELOG, apiDump.
…worker-side blocking reads, sharded SafeTensors from Blobs, PlannerProfile.BROWSER, browser MemoryProbe

JsBlobRandomAccessSource (Kotlin/Wasm) addresses a Blob with Long positions and size
(Blob.slice takes 53-bit numbers; the Int path truncated a shard over 2 GB), moves bytes
into Kotlin through 4 MB stages in wasm linear memory instead of one interop call per byte
(219 ms -> 36 ms per 8 MB), and serves blocking reads beyond the preloaded window from the
Blob itself inside a Web Worker through FileReaderSync (supportsSyncReads, openSync,
hasFileReaderSync()); on the main thread such a read throws with the explanation. The
Kotlin/JS variant gets Long addressing and a copy-free ByteArray view.

ShardedSafeTensorsParametersLoader(indexJson, shardSource) and
StreamingShardedSafeTensorsReader.openFromIndexContent / openFromParsedIndex take the
index as text and a suspending provider that opens each shard by file name, so a sharded
checkpoint loads from Blobs or fetched bytes where there is no file system; withPolicy for
the new form. Both forms now open every shard once and close what was opened on failure.

PlannerProfile.BROWSER (Budget.RESERVE_BROWSER, BROWSER_AVAILABLE_BYTES): 256 MB reserved,
dense weights quantized to Q8_0 at load, KV auto-quantized past 80 %, nothing mapped;
forDevice() returns it on JS and wasmJs (expect/actual isWebHost); skainet-plan --profile browser.

MemoryProbe on Kotlin/Wasm sums the JS heap in use (performance.memory), the module's linear
memory and the installed OffHeapAllocator's heldBytes() (new default-null member; the SIMD128
sidecar reports resident weights + activations); Kotlin/JS reports the JS heap. Faults stay null.

Docs: io-readers gains the browser section and a per-target table naming each remaining
difference with its reason; planner and memory-model pages updated. Tests: Blob source under
Karma incl. the worker path (FileReaderSync polyfilled by a sync XHR on a Blob URL), sharded
checkpoint from Blobs end to end, common reader provider tests, JVM path-vs-provider parity,
browser profile rules. M2AcceptanceTest no longer couples "RSS known" to "faults known".
conv1d/conv2d/conv3d on dense FP32 were the last FP32 ops a forward pass
spends time in that still ran as the common per-element loop on wasmJs
(im2col + the Vector API on the JVM). New shared C kernel
skainet_f32_conv (f32_conv.c, every native build): one kernel for all
three ranks over three spatial dims; per batch and group the receptive
fields are gathered patch-major into caller scratch, threaded over
patches, and multiplied against the weight rows by the existing
skainet_fp32_matmul_wt SIMD128/NEON dot kernel; groups (depthwise
included), stride, padding, dilation and a [c_out] bias are geometry.

WasmCpuOps overrides the three convolutions: the weight resident after
its first call, a heap input staged, a resident one read in place, the
output resident, so conv -> relu chains never leave linear memory;
shapes the common validation rejects stay on the common path. Sidecar
rebuilt (one new export).

Kernel support matrix: a "Float32 conv1d / conv2d / conv3d (im2col +
GEMM)" row in the Kernel packs table (panama-vector on the JVM,
wasm-simd128 on JS/WASM). engine-conv2d joins jvm-cpu-publish and
wasm-bench with identical shapes. Docs: eager-execution gap bullet
rewritten, browser-lane status, benchmarks table, CHANGELOG.

Tests: WasmSimdParityTest gains the conv parity test (eleven cases incl.
depthwise, grouped, dilated, asymmetric padding, the 4-D bias form and
conv3d, a resident chain, weight resident once) against DefaultCpuOps.
Measured (1x64x56x56, 64 filters 3x3): wasm scalar 0.04 GFLOPS ->
simd128 st 6.8 / mt 22.0; JVM Vector API path 10.7.
@michalharakal
michalharakal merged commit eacf552 into SKaiNET-developers:develop Oct 20, 2026
23 checks passed
@michalharakal
michalharakal deleted the feature/wasm-conv-im2col branch October 20, 2026 14:44
michalharakal added a commit that referenced this pull request Oct 21, 2026
CHANGELOG [0.62.0] - 2026-10-21 with the parity headline over #1363, #1364,
#1368, #1369, #1426, #1427, #1428 and #1430, plus entries for the operator
coverage matrix (#1370), the BitNet decode allocation and threading fixes
(#1359, #1360, #1361, #1362), the ClassificationSupport extraction (#1429)
and the toolchain bumps (Kotlin 2.4.21, kotest 6.2.6, Gradle 9.8.1).
VERSION_NAME, CITATION, Antora version, README bom coordinate, What's New
and contributors, Android tutorial coordinates; regenerated operator and
kernel-support matrices; SKEEP-007 rollout step 3 marked shipped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants