Repository navigation
feat(wasm): convolutions as im2col + GEMM in the SIMD128 sidecar - #1428
Merged
michalharakal merged 6 commits intoOct 20, 2026
Merged
michalharakal merged 6 commits into
michalharakal merged 6 commits into
Conversation
…l on the JVM Wasm: the sidecar exports skainet_ternary_f32_gemv, skainet_ternary_lmhead_stage1 and skainet_bitnet_gemv_tq2_0. The portable branch of skainet_ternary_f32.c (MSVC, Emscripten) holds the vendored decode table as a compile-time constant with a SIMD128 body and threads over output rows through the shared row pool; bitnet_gemv.c gains a SIMD128 block body and the same row threading on every tier. WasmTernaryKernelPack (installed by WasmSimd.install) registers the three native faces under the dispatch keys the JVM FFM and Android JNI ternary packs serve. JVM: Fp32MatmulWtKernel / KernelProvider.matmulFp32Wt() and the JVM-only Fp32MemSegMatmulWtKernel / MemSegKernelProvider.matmulFp32WtMemSeg() expose skainet_fp32_matmul_wt; NativeFp32MatmulWtKernel binds it with a heap and a segment face. DefaultCpuOpsJvm.matmulWeightTransposed multiplies a dense [out, in] FP32 weight in place over a resident off-heap weight instead of transposing and caching a copy. WasmKernelProvider exposes the same accessor. Kernel support matrix: a fourth table, "Kernel packs (view dispatch)". Bench: engine-fp32-linear on both harnesses. Docs: eager-execution ternary rows and x·Wᵀ row; the wasm ternary gap is closed.
…ion and reduction kernels PlatformStorage.offHeapAllocator (Kotlin/Wasm) binds MemoryDomain.HOST_OFFHEAP to a backend's memory. WasmSimd.install() binds it to the SIMD128 sidecar's linear memory: WasmLinearStorage (Storage.OffHeap: pointer, bulk copies, slices, close frees) and WasmLinearTensorData (FloatArrayTensorData whose heap mirror is downloaded once on first access), so a ForwardScope frees sidecar activations at reset() and StorageCapabilities reports off-heap support in a browser. Ambient outputs are freed through a FinalizationRegistry; WasmSimd.residentBudgetBytes bounds what stays resident. WasmCpuOps keeps matmul / matmulWeightTransposed outputs resident and runs add/subtract/multiply/ divide, relu/sigmoid/silu/gelu/erf/tanh and sum/mean in the sidecar on resident operands (transcendentals on heap operands too); two heap operands keep the common path. toResident() uploads an input once. New shared C kernels in f32_ops.c (skainet_f32_binary/unary/reduce_all/reduce_axis; SIMD128 and NEON bodies, row-pool chunks), compiled into the native library and the sidecar. wasm-bench gains the laya.chain case and resident-operand variants of the add and sum scenarios.
…on A/B; chain line reports the budget
…sm sidecar and the JVM FFM provider
New shared C kernels in f32_fused.c — skainet_f32_softmax_rows, skainet_f32_layer_norm,
skainet_f32_rms_norm and skainet_f32_sdpa (grouped-query heads, causal, sliding window with
rightContext, per-batch key lengths, dense additive mask with trailing-aligned broadcast) —
with SIMD128 and NEON bodies and row-pool threading over query rows. SPI faces
Fp32SoftmaxKernel / Fp32NormKernel / Fp32AttentionKernel behind KernelProvider.softmaxFp32() /
normFp32() / attentionFp32() (supports("softmax" | "layer_norm" | "rms_norm" | "sdpa",
["Float32"])), served by NativeKernelProvider through FFM and by WasmKernelProvider.
DefaultCpuOpsJvm routes softmax/logSoftmax over the last dimension and scaledDotProductAttention
to the native kernels when the FFM provider is loaded; WasmCpuOps runs them in the sidecar with
resident outputs. The common SDPA's validation moves into protected helpers both overrides share.
FusedNormOps (lang-core) is an optional interface beside TensorOps that LayerNormalization
consults for FP32 input; only the sidecar and the JVM FFM op sets implement it, so recording and
differentiating op sets keep the decomposition.
Also fixes a latent loader failure: GCC with -ffast-math vectorised the expf/tanhf/erf loops of
f32_ops.c into libmvec calls the shared library is not linked against, which terminated the
process at the first JVM call into such a kernel; the transcendentals now go through opaque
scalar wrappers and libskainet_kernels.so has no libmvec reference.
Bench: engine-softmax, engine-layer-norm, engine-sdpa on both harnesses. Kernel matrix: three
fused rows. Docs, CHANGELOG, apiDump.
…worker-side blocking reads, sharded SafeTensors from Blobs, PlannerProfile.BROWSER, browser MemoryProbe JsBlobRandomAccessSource (Kotlin/Wasm) addresses a Blob with Long positions and size (Blob.slice takes 53-bit numbers; the Int path truncated a shard over 2 GB), moves bytes into Kotlin through 4 MB stages in wasm linear memory instead of one interop call per byte (219 ms -> 36 ms per 8 MB), and serves blocking reads beyond the preloaded window from the Blob itself inside a Web Worker through FileReaderSync (supportsSyncReads, openSync, hasFileReaderSync()); on the main thread such a read throws with the explanation. The Kotlin/JS variant gets Long addressing and a copy-free ByteArray view. ShardedSafeTensorsParametersLoader(indexJson, shardSource) and StreamingShardedSafeTensorsReader.openFromIndexContent / openFromParsedIndex take the index as text and a suspending provider that opens each shard by file name, so a sharded checkpoint loads from Blobs or fetched bytes where there is no file system; withPolicy for the new form. Both forms now open every shard once and close what was opened on failure. PlannerProfile.BROWSER (Budget.RESERVE_BROWSER, BROWSER_AVAILABLE_BYTES): 256 MB reserved, dense weights quantized to Q8_0 at load, KV auto-quantized past 80 %, nothing mapped; forDevice() returns it on JS and wasmJs (expect/actual isWebHost); skainet-plan --profile browser. MemoryProbe on Kotlin/Wasm sums the JS heap in use (performance.memory), the module's linear memory and the installed OffHeapAllocator's heldBytes() (new default-null member; the SIMD128 sidecar reports resident weights + activations); Kotlin/JS reports the JS heap. Faults stay null. Docs: io-readers gains the browser section and a per-target table naming each remaining difference with its reason; planner and memory-model pages updated. Tests: Blob source under Karma incl. the worker path (FileReaderSync polyfilled by a sync XHR on a Blob URL), sharded checkpoint from Blobs end to end, common reader provider tests, JVM path-vs-provider parity, browser profile rules. M2AcceptanceTest no longer couples "RSS known" to "faults known".
conv1d/conv2d/conv3d on dense FP32 were the last FP32 ops a forward pass spends time in that still ran as the common per-element loop on wasmJs (im2col + the Vector API on the JVM). New shared C kernel skainet_f32_conv (f32_conv.c, every native build): one kernel for all three ranks over three spatial dims; per batch and group the receptive fields are gathered patch-major into caller scratch, threaded over patches, and multiplied against the weight rows by the existing skainet_fp32_matmul_wt SIMD128/NEON dot kernel; groups (depthwise included), stride, padding, dilation and a [c_out] bias are geometry. WasmCpuOps overrides the three convolutions: the weight resident after its first call, a heap input staged, a resident one read in place, the output resident, so conv -> relu chains never leave linear memory; shapes the common validation rejects stay on the common path. Sidecar rebuilt (one new export). Kernel support matrix: a "Float32 conv1d / conv2d / conv3d (im2col + GEMM)" row in the Kernel packs table (panama-vector on the JVM, wasm-simd128 on JS/WASM). engine-conv2d joins jvm-cpu-publish and wasm-bench with identical shapes. Docs: eager-execution gap bullet rewritten, browser-lane status, benchmarks table, CHANGELOG. Tests: WasmSimdParityTest gains the conv parity test (eleven cases incl. depthwise, grouped, dilated, asymmetric padding, the 4-D bias form and conv3d, a resident chain, weight resident once) against DefaultCpuOps. Measured (1x64x56x56, 64 filters 3x3): wasm scalar 0.04 GFLOPS -> simd128 st 6.8 / mt 22.0; JVM Vector API path 10.7.
aharakal
approved these changes
Oct 20, 2026
michalharakal
added a commit
that referenced
this pull request
Oct 21, 2026
CHANGELOG [0.62.0] - 2026-10-21 with the parity headline over #1363, #1364, #1368, #1369, #1426, #1427, #1428 and #1430, plus entries for the operator coverage matrix (#1370), the BitNet decode allocation and threading fixes (#1359, #1360, #1361, #1362), the ClassificationSupport extraction (#1429) and the toolchain bumps (Kotlin 2.4.21, kotest 6.2.6, Gradle 9.8.1). VERSION_NAME, CITATION, Antora version, README bom coordinate, What's New and contributors, Android tutorial coordinates; regenerated operator and kernel-support matrices; SKEEP-007 rollout step 3 marked shipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Seventh step of the JVM ↔ wasm parity series (after #1363, #1364, #1368, #1369, #1426, #1427), and the last compute gap on the list: convolutions. On the JVM
conv1d/conv2d/conv3drun as im2col + the Vector API (JvmVectorKernels); in a browser they were the common per-element loop — the only FP32 op a forward pass spends time in that was still scalar Kotlin on wasm after the fused-op step.skainet_f32_conv(f32_conv.c, shared native tree, every native build) — one kernel for all three ranks over three spatial dims (a lower rank passes the identity geometry for the leading ones). Per batch and group it gathers the receptive fields patch-major into a caller-provided scratch (col[p, :]is patch p's window, zero in the padding; whole rows copied withmemcpywhen the width dilation is 1) and multiplies them against the weight rows with the existingskainet_fp32_matmul_wt— every output is a contiguous dot product of a weight row and a patch row, so the SIMD128 / NEON body that kernel already has serves the convolution, accumulated in ascendingk. Gather and GEMM thread over patches through the shared row pool; a bias of[c_out]is added in the kernel. Groups (depthwise included), stride, padding and dilation are all geometry. The 22-int geometry block travels as one pointer (SKAINET_CONV_GEO_INTS).WasmCpuOps.conv1d/conv2d/conv3d— dense FP32 input, weight and optional bias ([c_out]or[1, c_out, 1, …]) go to the sidecar: the weight is a parameter and stays resident after its first call (denseWeightHandle), a heap input is staged per call and a resident one read in place, the output is resident under the usual budget — soconv → relu → …never leaves linear memory. Anything the common validation would reject stays on the common path, which raises its message. Sidecar rebuilt (SidecarArtifacts.kt); exports grow by one.Matrix, bench, docs — the Kernel packs table gains
Float32 conv1d / conv2d / conv3d (im2col + GEMM):panama-vectoron the JVM (the one row whose JVM cell is the ops layer itself, not a provider — noted in the test),wasm-simd128on JS/WASM.engine-conv2d(a 3 × 3 same-padding layer, 64 → 64 channels over 56 × 56, with bias) joinsjvm-cpu-publishandwasm-benchwith identical shapes.eager-execution.adocstates what still runs as common Kotlin on wasm (memory movement: reshape / permute / concat / gather / unfold / pooling, and the integer and narrow-float element paths); the browser-lane page's status line and the CHANGELOG carry the step.Verification
:skainet-backends:skainet-backend-wasm-simd:wasmJsNodeTest(13/13) — newconvolutions run as im2col and GEMM in the sidecar and match the scalar lane: conv2d with and without bias and with the 4-D bias form, a 2 × 3 kernel with stride, dilation and asymmetric padding, depthwise (groups = c_in), two groups with stride, 1 × 1; conv1d with stride, dilation and groups; conv3d with stride and padding; aconv → reluchain and a resident input, every output resident and the input never downloaded; the weight resident after its first call and not re-uploaded by the second. All againstDefaultCpuOpson the same inputs.:skainet-backends:skainet-backend-native-cpu:jvmTest—KernelSupportMatrixTest(the CMake build compiles the new file intolibskainet_kernels),NativeFusedKernelsTest;generateKernelMatrix; the hostgcc -Wall -Wextracompile off32_conv.cis clean.scripts/pr-gate.sh --quick, full gate.Measured
engine-conv2d, batch 1, 64 → 64 channels, 56 × 56, 3 × 3, pad 1, bias (231 MFLOP per call); 12-core laptop.panama(im2col + Vector API,DefaultCpuOpsJvm)ffm(same path — no native conv binding on the JVM)The sidecar convolution lands at roughly half the sidecar's dense GEMM rate on the same tier (14.1 st / 43.6 mt for the 1024³ GEMM): the gather is memory-bound and the dot kernel is the one-row-at-a-time
x · Wᵀbody, not the 4 × 8 register-blocked GEMM. The JVM path is single-threaded and keeps its Vector API dot per output; the four-worker sidecar now outruns it. Bindingskainet_f32_convon the JVM through FFM (anFp32ConvKernelface beside the fused-op faces) would close that in the other direction; left out of this PR as agreed for the series — noted as a follow-up.