feat(kernels): ternary kernels on the wasm sidecar; direct x·Wᵀ kernel on the JVM

CHANGELOG.md

@@ -11,6 +11,10 @@
 - **Explicit SIMD128 bodies for every sidecar matmul kernel**: Q4_K, Q5_K, Q6_K, Q5_0 and Q5_1 join Q8_0 and Q4_0 with hand-written `__wasm_simd128__` paths (the K-quant int8 dots on `i32x4.dot_i16x8_s`, the Q5 fifth-bit plane via a byte swizzle); the dense FP32 GEMM gets a 4×8 register-blocked, k-blocked micro-kernel; BF16 and FP16 widen eight weights per step. BF16 and FP16 also gain NEON bodies (none existed on any tier). Results stay bit-identical in accumulation order to the portable bodies.
 - **Threaded dense kernels**: `skainet_fp32_matmul`, `skainet_bf16_matmul` and `skainet_fp16_matmul` now split the output over the shared row-thread pool like the packed kernels do (the GEMM over the longer side of `C`, the 16-bit kernels over columns) — the mt sidecar's pthreads in a browser, the native pool on the JVM.
 - **`WasmCpuOps.matmul`**: a dense 2-D `A · B` on heap FP32 operands runs the sidecar GEMM; the common `[k, n]` loop no longer serves that shape on wasm (it was the only dense path that never reached the sidecar).
+- **Ternary kernels on the wasm tier**: the sidecar now exports `skainet_ternary_f32_gemv`, `skainet_ternary_lmhead_stage1` and `skainet_bitnet_gemv_tq2_0`, and `WasmTernaryKernelPack` (installed by `WasmSimd.install()`) registers them under the same `KernelDispatch` keys the JVM FFM and Android JNI ternary packs serve — exact FP32 × `BitNet-b1.58`, the fused multi-plane `BitNet-planes` lm_head and int8 × `TQ2_0`. A BitNet weight in a browser previously went to the decoding reference kernel. The portable branch of `skainet_ternary_f32.c` (MSVC and Emscripten) now holds the vendored kernel's 256-entry decode table as a compile-time constant with a SIMD128 body and threads over output rows through the shared row pool; `bitnet_gemv` gains a SIMD128 block body and the same row threading on every tier.
+- **Direct `x · Wᵀ` on the JVM**: `Fp32MatmulWtKernel` / `KernelProvider.matmulFp32Wt()` (and the JVM-only `Fp32MemSegMatmulWtKernel` / `MemSegKernelProvider.matmulFp32WtMemSeg()`) expose `skainet_fp32_matmul_wt`, which the wasm sidecar has served since 0.61.0 and the JVM never bound. `NativeKernelProvider` carries it; `DefaultCpuOpsJvm.matmulWeightTransposed` multiplies a dense `[out, in]` FP32 weight in place through it — a segment-backed weight where it lies, a heap weight after one resident off-heap copy — instead of transposing the weight once into `[k, n]` and caching the copy. The wasm provider exposes the same accessor (`WasmFp32MatmulWtKernel`), and `KernelProvider.supports("matmul_wt", …)` reports it.
+- **Kernel support matrix, fourth table** *Kernel packs (view dispatch)*: the ternary formats and the direct `x · Wᵀ` per platform, which the `matmul` accessors the first table introspects do not cover.
+- **`engine-fp32-linear`** in `jvm-cpu-publish` and `wasm-bench`: a `Linear` layer's `x · Wᵀ` (16 tokens, 768 → 3072) through `ctx.ops.matmulWeightTransposed`; on the JVM `--provider panama` measures the cached-transpose path and `--provider ffm` the direct kernel.
 - **`wasm-bench` runs the `engine-*` scenarios** of the JVM publish harness (`engine-fp32-gemm`, `engine-kernel-matmul`, `engine-bf16-matmul`, `engine-q8-matmul`, `engine-q4-gemm`, `engine-ternary-f32-gemv`, `engine-elementwise-add`, `engine-reductions-sum/mean`) with identical shapes and metrics on every tier it measures, so the JVM and wasm numbers sit in one table.
 
 ## [0.61.0] - 2026-10-16

build-logic/convention/src/main/kotlin/GenerateKernelMatrixTask.kt

@@ -128,6 +128,33 @@ abstract class GenerateKernelMatrixTask : DefaultTask() {
                 appendLine("|===")
                 appendLine("")
             }
+            if (module.packs.isNotEmpty()) {
+                appendLine("== Kernel packs (view dispatch)")
+                appendLine("")
+                appendLine(
+                    "Kernels that are not reached through the `matmul` accessors above: the ternary formats " +
+                        "are served by `ViewKernelPack`s keyed in `KernelDispatch` (`BitNet-b1.58` with exact " +
+                        "FP32 activations, the fused multi-plane `BitNet-planes` lm_head, `TQ2_0` with int8 " +
+                        "activations), and the dense `x · Wᵀ` of a `Linear` layer by the `matmulFp32Wt` " +
+                        "accessor, which multiplies a row-major `[out, in]` weight in place instead of " +
+                        "transposing and caching a copy. The scalar reference kernels serve every cell that " +
+                        "is empty here (`—`), at reference speed; the common row-dot serves `x · Wᵀ` there.",
+                )
+                appendLine("")
+                appendLine("[cols=\"$colSpec\", options=\"header\"]")
+                appendLine("|===")
+                append("| Kernel ")
+                module.platforms.forEach { append("| $it ") }
+                appendLine("")
+                appendLine("")
+                module.packs.forEach { fmt ->
+                    append("| `${fmt.name}` ")
+                    module.platforms.forEach { p -> append("| ${fmt.byPlatform[p] ?: "—"} ") }
+                    appendLine("")
+                }
+                appendLine("|===")
+                appendLine("")
+            }
             appendLine(
                 "See also the eager backends & kernels mindmap " +
                     "(xref:explanation/eager-execution.adoc[]) for the narrative overview and gaps.",

build-logic/convention/src/main/kotlin/models/KernelSupportModels.kt

@@ -23,6 +23,12 @@ data class KernelSupportModule(
      */
     val mapped: List<KernelFormatSupport> = emptyList(),
     val resident: List<KernelFormatSupport> = emptyList(),
+    /**
+     * Kernel packs (view dispatch): the ternary formats and the direct dense `x · Wᵀ`, served by
+     * `ViewKernelPack`s / the `matmulFp32Wt` accessor rather than the `matmul` accessors. Absent in
+     * older JSON.
+     */
+    val packs: List<KernelFormatSupport> = emptyList(),
 )
 
 @Serializable

docs/modules/ROOT/pages/contributing/benchmarks.adoc

@@ -33,6 +33,7 @@ microbench tests under `skainet-backends/skainet-backend-native-cpu`:
 |===
 | Scenario | What it exercises | Unit | Direction
 | engine-fp32-gemm        | `ctx.ops.matmul` end-to-end FP32 square matmul              | GFLOPS        | higher is better
+| engine-fp32-linear      | `ctx.ops.matmulWeightTransposed` — a `Linear` layer's `x · Wᵀ` with the weight stored `[out, in]` (panama: cached transpose + GEMM; ffm: native direct kernel, no transposed copy) | GFLOPS | higher is better
 | engine-q4-gemm          | `PanamaVectorQ4KMatmulKernel` F32 × Q4_K matmul at LLM shapes | GOP/s        | higher is better
 | engine-kernel-matmul    | Direct `Fp32MatmulKernel` (scalar vs panama-vector)         | GFLOPS        | higher is better
 | engine-bf16-matmul      | `Bf16MatmulKernel` F32 × BF16-packed matmul (scalar vs panama-vector) | GFLOPS | higher is better

docs/modules/ROOT/pages/explanation/eager-execution.adoc

@@ -29,7 +29,8 @@ mindmap
       Native FFM
         JVM only — C kernels via CMake
         Q4_K MemSeg zero-copy
-        Q6_K missing
+        direct x·Wᵀ, no transposed copy
+        ternary LUT gemv and planes lm_head
       Native JNI
         Android only — same C kernels via JNI
         two .so tiers, cpuinfo-gated armv8-a vs armv8.2+dotprod
@@ -47,7 +48,7 @@ mindmap
       Android — scalar, Native-JNI
       Native linux — scalar, Native-cinterop
       Native apple — scalar, Accelerate, Native-cinterop
-      JS and WASM — scalar; packed weights relaid once into kernel feed order (SKEEP-007)
+      JS and WASM — scalar; wasmJs plus the SIMD128 sidecar pack: every matmul format, ternary, direct x·Wᵀ (SKEEP-007)
 ----
 
 == Kernel × provider (matmul, FP32 activations)
@@ -56,6 +57,7 @@ mindmap
 |===
 | Weight format | Scalar (all targets) | Panama (JVM) | Native FFM (JVM) | Native JNI (Android) | Native cinterop (K/N) | Wasm SIMD128 (wasmJs)
 | FP32 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅
+| FP32 `x · Wᵀ` direct (weight `[out, in]`, no transposed copy) | row-dot | ❌ (cached transpose) | ✅ | ❌ | ❌ | ✅
 | BF16 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅
 | FP16 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅
 | Q8_0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
@@ -65,9 +67,9 @@ mindmap
 | Q5_K | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
 | Q5_1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
 | Q5_0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
-| TQ2_0 / BitNet b1.58 (int8 activations) | ✅ | — | — | ✅ NEON | — | —
-| BitNet b1.58 (exact FP32 activations, vendored NeoGPU kernel) | ✅ | ✅ FFM | — | ✅ JNI + fused lm_head | ✅ K/N cinterop | ❌ (reference)
-| BITNET_PLANES (multi-plane ternary packing) | ✅ | via kernel pack | — | via kernel pack | via kernel pack | ❌ (reference)
+| TQ2_0 / BitNet b1.58 (int8 activations) | ✅ | — | — | ✅ NEON | — | ✅ SIMD128
+| BitNet b1.58 (exact FP32 activations, LUT gemv) | ✅ | — | ✅ FFM (vendored NeoGPU) | ✅ JNI | ✅ K/N cinterop | ✅ SIMD128 LUT
+| BITNET_PLANES (multi-plane ternary packing, fused lm_head) | ✅ | — | ✅ FFM (vendored NeoGPU) | ✅ JNI | ✅ K/N cinterop | ✅ SIMD128 LUT
 | Q2_K / Q3_K / Q8_K / IQ4 | ❌ (dequant to FP32 only) | ❌ | ❌ | ❌ | ❌ | ❌
 |===
 
@@ -78,7 +80,15 @@ columns are mutually exclusive per platform rather than stacked. The Wasm SIMD12
 sidecar pack (`skainet-backend-wasm-simd`), present only after `WasmSimd.install()`; every matmul
 kernel in it has an explicit SIMD128 body (the K-quant int8 dots on `i32x4.dot_i16x8_s`, a 4×8
 register-blocked dense GEMM, eight-wide BF16/FP16 widening), and the dense kernels thread over the
-sidecar's pthread pool like the packed ones when the page is cross-origin isolated.
+sidecar's pthread pool like the packed ones when the page is cross-origin isolated. The ternary
+kernels in the sidecar are the portable mirror of the vendored NeoGPU kernel — the same 256-entry
+decode table as a compile-time constant, read four codes at a time in SIMD128 — registered under the
+same dispatch keys as the FFM and JNI ternary packs (`WasmTernaryKernelPack`).
+
+The dense `x · Wᵀ` row is the product a `Linear` layer computes. The JVM and the wasm tier multiply
+the `[out, in]` weight in place through `skainet_fp32_matmul_wt` (`KernelProvider.matmulFp32Wt`;
+the JVM makes a heap weight resident off-heap once, the sidecar keeps it in its linear memory) and
+never materialise the transposed copy the Panama path caches per weight.
 
 Packed weights reach a kernel through the registry by declaring their
 xref:explanation/packed-weight-layout.adoc[block order]; a canonical weight is relayouted once,
@@ -101,8 +111,6 @@ Packed-quant matmul works on every target: those formats have a `commonMain` sca
 
 == Gaps
 
-* *Wasm ternary / BitNet* — the ternary gemv and planes lm-head kernels are not in the sidecar
-build yet, so BitNet models in a browser run the decoding reference kernel.
 * *Wasm beyond matmul* — elementwise, activations, reductions, normalisation and attention are the
 common scalar Kotlin paths on wasmJs; Kotlin/Wasm has no SIMD types and no shared-memory threads, so
 these follow the matmuls into the sidecar (SKEEP-007 rollout step 3).

docs/modules/ROOT/pages/explanation/kernel-selection.adoc

@@ -91,7 +91,7 @@ platform support is absent (a missing native library, no vector unit).
 
 | Kotlin/Native, wasm, JS
 | manual — e.g. `installNativeKernels()`; on wasmJs `WasmSimd.install()` (SKEEP-007)
-| manual — call the pack's `install()` yourself; `WasmSimd.install()` registers the browser pack and its provider
+| manual — call the pack's `install()` yourself; `WasmSimd.install()` registers the browser pack, its ternary pack and its provider
 |===
 
 Without a row-major pack, a canonical packed weight is relaid **once per weight** into kernel feed

docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc

@@ -58,4 +58,18 @@ The browser has no files to map, so the wasm tier keeps weights *resident* in th
 | `Q5_0` | — | — | — | — | wasm-resident 
 |===
 
+== Kernel packs (view dispatch)
+
+Kernels that are not reached through the `matmul` accessors above: the ternary formats are served by `ViewKernelPack`s keyed in `KernelDispatch` (`BitNet-b1.58` with exact FP32 activations, the fused multi-plane `BitNet-planes` lm_head, `TQ2_0` with int8 activations), and the dense `x · Wᵀ` of a `Linear` layer by the `matmulFp32Wt` accessor, which multiplies a row-major `[out, in]` weight in place instead of transposing and caching a copy. The scalar reference kernels serve every cell that is empty here (`—`), at reference speed; the common row-dot serves `x · Wᵀ` there.
+
+[cols="1,1,1,1,1,1", options="header"]
+|===
+| Kernel | JVM | Android | Native·linux | Native·apple | JS/WASM 
+
+| `BitNet-b1.58 (FP32 activations)` | ffm-ternary | native-jni | native-cinterop | native-cinterop | wasm-simd128 
+| `BitNet-planes` | ffm-ternary | native-jni | native-cinterop | native-cinterop | wasm-simd128 
+| `TQ2_0 (int8 activations)` | — | native-jni | — | — | wasm-simd128 
+| `Float32 x·Wᵀ direct` | native-ffm | — | — | — | wasm-simd128 
+|===
+
 See also the eager backends & kernels mindmap (xref:explanation/eager-execution.adoc[]) for the narrative overview and gaps.

skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/Fp32LinearScenario.kt

@@ -0,0 +1,79 @@
+package sk.ainet.bench.publish.scenarios
+
+import sk.ainet.backend.api.kernel.KernelRegistry
+import sk.ainet.bench.publish.runner.Scenario
+import sk.ainet.context.DirectCpuExecutionContext
+import sk.ainet.exec.kernel.NativeKernelProvider
+import sk.ainet.exec.kernel.PanamaVectorKernelProvider
+import sk.ainet.exec.kernel.ScalarKernelProvider
+import sk.ainet.lang.tensor.Shape
+import sk.ainet.lang.tensor.Tensor
+import sk.ainet.lang.types.FP32
+
+/**
+ * A `Linear` layer's product, `x · Wᵀ` with the weight stored `[out, in]`, through the public
+ * `ctx.ops.matmulWeightTransposed` path — what every projection of a transformer pays, as opposed to
+ * the `[k, n]` GEMM of `engine-fp32-gemm`. Shape: a 16-token prefill against a 768 → 3072 FFN
+ * projection. Primary metric: GFLOP/s over `2·m·n·k`.
+ *
+ * The provider selects the path: `panama` (the default) transposes the weight once into `[k, n]`
+ * and runs the Vector API GEMM; `ffm` runs the native direct kernel (`skainet_fp32_matmul_wt`) over a
+ * weight made resident off-heap, no transposed copy; `scalar` is the common row-dot. Only the chosen
+ * provider is registered, so the ops layer cannot fall through to another.
+ */
+internal class Fp32LinearScenario(
+    smoke: Boolean,
+    private val providerName: String,
+) : Scenario {
+    override val id: String = "engine-fp32-linear"
+    override val suite: String = "skainet-engine"
+    override val primaryMetric: String = "gflops"
+    override val unit: String = "gflops"
+    override val higherIsBetter: Boolean = true
+    override val kernelProvider: String = providerName
+
+    private val m: Int = 16
+    private val k: Int = 768
+    private val n: Int = if (smoke) 768 else 3072
+    override val parameters: Map<String, String> = mapOf(
+        "m" to m.toString(),
+        "k" to k.toString(),
+        "n" to n.toString(),
+        "kernel" to providerName,
+    )
+
+    private lateinit var ctx: DirectCpuExecutionContext
+    private lateinit var x: Tensor<FP32, Float>
+    private lateinit var w: Tensor<FP32, Float>
+
+    override fun setup() {
+        KernelRegistry.clearForTesting()
+        when (providerName.lowercase()) {
+            "panama", "panama-vector" -> KernelRegistry.register(PanamaVectorKernelProvider)
+            "ffm", "native", "native-ffm" -> {
+                check(NativeKernelProvider.isAvailable()) { "bundled libskainet_kernels missing — the ffm provider cannot run" }
+                KernelRegistry.register(NativeKernelProvider)
+            }
+            "scalar" -> KernelRegistry.register(ScalarKernelProvider)
+            else -> error("unknown kernel provider: $providerName (use 'panama', 'ffm' or 'scalar')")
+        }
+        ctx = DirectCpuExecutionContext()
+        x = ctx.fromFloatArray(Shape(m, k), FP32::class, FloatArray(m * k) { ((it % 251) - 125).toFloat() / 127f })
+        w = ctx.fromFloatArray(Shape(n, k), FP32::class, FloatArray(n * k) { ((it * 13 % 257) - 128).toFloat() / 127f })
+        // First call: the transpose cache or the resident copy is built here, not in a measured run.
+        ctx.ops.matmulWeightTransposed(x, w)
+    }
+
+    override fun runOnce(): Double {
+        val start = System.nanoTime()
+        val result = ctx.ops.matmulWeightTransposed(x, w)
+        val elapsedNs = System.nanoTime() - start
+        @Suppress("UNUSED_VARIABLE") val sink = result.hashCode()
+        val flops = 2.0 * m.toDouble() * n.toDouble() * k.toDouble()
+        return (flops / (elapsedNs / 1_000_000_000.0)) / 1e9
+    }
+
+    override fun teardown() {
+        KernelRegistry.clearForTesting()
+    }
+}

skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/ScenarioRegistry.kt

@@ -6,6 +6,7 @@ public object ScenarioRegistry {
 
     public fun ids(): List<String> = listOf(
         "engine-fp32-gemm",
+        "engine-fp32-linear",
         "engine-q4-gemm",
         "engine-kernel-matmul",
         "engine-bf16-matmul",
@@ -18,6 +19,7 @@ public object ScenarioRegistry {
 
     public fun byId(id: String, provider: String, smoke: Boolean): Scenario? = when (id) {
         "engine-fp32-gemm" -> Fp32GemmScenario(smoke = smoke, providerName = provider)
+        "engine-fp32-linear" -> Fp32LinearScenario(smoke = smoke, providerName = provider)
         "engine-q4-gemm" -> Q4GemmScenario(smoke = smoke, providerName = provider)
         "engine-kernel-matmul" -> KernelMatmulScenario(smoke = smoke, providerName = provider)
         "engine-bf16-matmul" -> Bf16MatmulScenario(smoke = smoke, providerName = provider)

skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Scenarios.kt

@@ -48,6 +48,15 @@ internal fun engineScenarios(): List<WasmScenario> = listOf(
         val flops = 2.0 * n * n * n;
         { flops / seconds { ctx.ops.matmul(a, b) } / 1e9 }
     },
+    WasmScenario("engine-fp32-linear", "gflops", mapOf("m" to "16", "k" to "768", "n" to "3072"), reps = 10) {
+        val m = 16; val k = 768; val n = 3072
+        val ctx = DirectCpuExecutionContext()
+        val x = ctx.fromFloatArray<FP32, Float>(Shape(m, k), FP32::class, FloatArray(m * k) { ((it % 251) - 125).toFloat() / 127f })
+        val w = ctx.fromFloatArray<FP32, Float>(Shape(n, k), FP32::class, FloatArray(n * k) { ((it * 13 % 257) - 128).toFloat() / 127f })
+        ctx.ops.matmulWeightTransposed(x, w) // the resident upload happens here, not in a measured run
+        val flops = 2.0 * m * n * k;
+        { flops / seconds { ctx.ops.matmulWeightTransposed(x, w) } / 1e9 }
+    },
     WasmScenario("engine-kernel-matmul", "gflops", mapOf("size" to "1024"), reps = 3) {
         val n = 1024
         DirectCpuExecutionContext()

skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Fp32MatmulWtKernel.kt

@@ -0,0 +1,28 @@
+package sk.ainet.backend.api.kernel
+
+/**
+ * Dense `x · Wᵀ` for a row-major `[out, in]` FP32 weight — the shape a `Linear` layer stores its
+ * weight in, multiplied in place:
+ *
+ *   `out[i, o] = Σ_p x[i, p] · w[o, p]`
+ *
+ * Every output is one contiguous dot product over the weight's own row, so no transposed copy of
+ * the weight is ever made. This is the SPI face of `skainet_fp32_matmul_wt` (SKEEP-007), which the
+ * wasm sidecar has served since 0.61.0 and the JVM FFM provider binds as well: a tier whose
+ * [Fp32MatmulKernel] wants the weight input-major (`[k, n]`) otherwise pays a scattered transpose
+ * copy per weight and caches it — about 0.5 GB for a 22-layer encoder.
+ *
+ * Operands are contiguous: `x` is `m` rows of `k`, `w` is `n` rows of `k`, `out` is `m` rows of
+ * `n`, each from its offset. Implementations must not mutate `x` or `w`, must fully overwrite the
+ * `m × n` block of `out`, and may assume the arrays do not alias. `k == 0` zeroes the block;
+ * `m == 0` or `n == 0` is a no-op. Each output accumulates its products in ascending `p`, so a
+ * conforming kernel is bit-identical to transpose-then-GEMM with the same accumulation order.
+ */
+public interface Fp32MatmulWtKernel {
+    public fun matmulWt(
+        x: FloatArray, xOffset: Int,
+        w: FloatArray, wOffset: Int,
+        out: FloatArray, outOffset: Int,
+        m: Int, n: Int, k: Int,
+    )
+}