feat(laya): expert heads, prompted routing and a mixture adapter

CHANGELOG.md

@@ -11,6 +11,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- **Expert heads and routing for Laya** (`sk.ainet.models.laya`): a Laya decision head can be loaded on its own and run on the base model's encoder output. `LayaExpertLoader.load(metaJson, weights, base, ctx)` binds an `expert.safetensors` (the `type_emb.*` / `head.*` / `scorer.*` tensors of a head copied from the base checkpoint and fine-tuned for a domain) with its `expert.json` (name, domains, calibration temperatures) into a `LayaExpert`; `LayaWeightPreparation.prepareHead` and `LayaDecisionModel.buildHead` are the steps underneath, and a head prepared on its own reproduces the base model bit for bit. `LayaDecisionModel.logitsFrom` / `logitsBatchFrom` and `LayaAgent.encodeFor` / `logitsFrom` run any `LayaDecisionHead` on an encoder output computed once, so several heads answer the same sequences for one encoder pass each. `LayaRouterSpec` is the prompted router (the base head answers a choice over kinds of text with descriptions; kind → expert; threshold; `general` fallback), `LayaMixtureAgent.decide(state, questions, expert, compare)` routes and answers with the chosen head (forced head reported as overridden; `compare` adds the base head's answers on the same encoder output; encoder and head timings), and `LayaMixtureDecisionAdapter` puts a mixture behind the typed decision contract with `onRouted` reporting the head that answered. Verified against the published evaluation of layaMOE (gated `LayaMoeEvalTest`, fixtures under `laya-moe/`, Apache-2.0).
+- `LayaAgent.decide` / `decideAll` / `logitsFor` overloads that take a head and a decoder; `LayaAgent.decodeAll`; `LayaDecisionModel.encodeBatch(sequences, padId)` returning a `LayaEncodedBatch`; `LayaDecisionModel.defaultHead`. The existing signatures are unchanged.
+
+### Changed
+
+- `LayaEncoderConfig.requireSupported()` accepts `position_embedding_type: "absolute"` beside `"sans_pos"`: ModernBERT positions are rotary whatever the field says, and transformers 5 exports (`convaiinnovations/laya-typed-decisions`) write `absolute`. The English checkpoint loads and its `sequences.json` section reproduces (gated on `LAYA_TYPED_DECISIONS_PATH`).
+
 ## [0.62.2] — 2026-10-18
 
 Against engine **0.61.0**, a transformers-only fix: the Android runtime's streaming Moonshine libraries

README.md

@@ -98,7 +98,7 @@ Honest status — see the project-status note at the top of this README.
 | **BERT** | Sentence embeddings on the DSL path (`bertNetwork()` + `BertEncoderRuntime`, eager or traced/fused) — verified against sentence-transformers on MongoDB/mdbr-leaf. One-call `BertEmbeddingModel.fromHuggingFace(...)` with built-in Hub download; MEAN or CLS pooling and retrieval prefixes cover LEAF, BGE and E5-style models. No text generation, no tool calling. |
 | **T5 / GTR** | Encoder-decoder runtime (hand-coded, batch 1, no KV cache) + `GtrEmbedder`, powering the **vec2text** embedding-inversion pipeline, with greedy and beam-search decoding — verified with a real-weights gtr-base round-trip test. |
 | **Voxtral** | TTS / voice; architecture code only — no runtime facade or CLI yet. |
-| **Laya** | Typed decisions, not text: `LayaDecisionAdapter` answers `choice` / `score` / `boolean probability` questions about a state through the shared `sk.ainet.decisions` API, abstaining under the caller's policy. mmBERT-base encoder + decision head on the FP32 CPU path, verified against the reference fixtures of the multilingual checkpoint (sequences, tokenizer corpus, predictions within 5e-5). JVM only for now; see `docs/…/reference/typed-decisions.adoc`. |
+| **Laya** | Typed decisions, not text: `LayaDecisionAdapter` answers `choice` / `score` / `boolean probability` questions about a state through the shared `sk.ainet.decisions` API, abstaining under the caller's policy. ModernBERT-family encoder (mmBERT-base `multilingual`, ModernBERT-large `typed-decisions`) + decision head, FP32 or Q8_0 at load, common loader on every target; verified against the reference fixtures (sequences, tokenizer corpus, predictions within 5e-5). Expert heads fine-tuned on the frozen encoder load on their own (`LayaExpertLoader`) and a prompted router picks one per request (`LayaMixtureAgent`, `LayaMixtureDecisionAdapter`), verified against the published layaMOE evaluation. See `docs/…/reference/typed-decisions.adoc`. |
 
 ### Near term
 

docs/modules/ROOT/pages/reference/typed-decisions.adoc

@@ -168,11 +168,13 @@ scope by accident.
 
 == Laya (`sk.ainet.models.laya`)
 
-`skainet-transformers-inference-laya` wraps the multilingual Laya checkpoint
-(https://huggingface.co/convaiinnovations/laya[convaiinnovations/laya], Apache 2.0):
-an mmBERT-base encoder (22 layers, hidden 768, GeGLU, RoPE on every layer, a local attention
-window on two of every three layers) followed by a type embedding, a two-layer decision head
-and a scorer. The port follows the reference `build_sequence` exactly, including CPython
+`skainet-transformers-inference-laya` wraps the Laya checkpoints (Apache 2.0): the multilingual one
+(https://huggingface.co/convaiinnovations/laya[convaiinnovations/laya], an mmBERT-base encoder, 22
+layers, hidden 768) and the English one
+(https://huggingface.co/convaiinnovations/laya-typed-decisions[convaiinnovations/laya-typed-decisions],
+ModernBERT-large, 28 layers, hidden 1024) — a ModernBERT-family encoder (GeGLU, RoPE on every layer,
+a local attention window on two of every three layers) followed by a type embedding, a two-layer
+decision head and a scorer. Every size comes from the checkpoint's own `encoder/config.json`. The port follows the reference `build_sequence` exactly, including CPython
 `json.dumps` float formatting for structured states (`PythonJson`), the 48-token option cap and
 the state-type-dependent truncation direction.
 
@@ -192,6 +194,44 @@ sequences, the 40-text tokenizer corpus (CJK, RTL, combining marks, emoji, liter
 and all six `predict.json` cases within 5e-5. The model-gated tests run when
 `LAYA_MULTILINGUAL_PATH` points at the checkpoint directory.
 
+== Expert heads and routing
+
+A Laya checkpoint is an encoder plus a small decision head (type embedding, two transformer
+blocks, scorer), and the head is what a publisher fine-tunes for a domain: the encoder stays
+frozen, the head is copied, trained on domain data and saved on its own as `expert.safetensors`
+(the `type_emb.*`, `head.*`, `scorer.*` tensors, `act_head.*` ignored) with an `expert.json`
+(name, domains, its own calibration temperatures). `sk.ainet.models.laya` runs any number of such
+heads on one encoder:
+
+* `LayaExpertLoader.load(metaJson, weights, base, ctx)` binds an expert head for a loaded base
+  `LayaAgent` (`weightFormFor = LayaWeightForms.q8_0For` packs its projections like the base's);
+  `LayaExpert` holds the `LayaDecisionHead` and a `LayaDecoder` with the expert's temperatures over
+  the base agent's budgets. `LayaWeightPreparation.prepareHead` / `LayaDecisionModel.buildHead`
+  are the steps underneath; a head prepared on its own reproduces the base model bit for bit.
+* `LayaDecisionModel.logitsFrom(encoded, sequence, type, head)` runs a head on an encoder output:
+  the encoder runs once per question (the sequence contains the question) and every head that
+  should answer it reads the same hidden states. `LayaAgent.encodeFor` / `logitsFrom` do the same
+  over the padded batch plan.
+* `LayaRouterSpec(question, options, experts, threshold)` is the prompted router: the base head
+  answers a choice over fine-grained *kinds of text* (each with a description) about the state,
+  every kind maps to an expert, and the expert answers when it is loaded and the top probability
+  reaches the threshold — otherwise `general`, the base head. Routing is sequence-level, one head
+  per request; this is not a sparse mixture of experts.
+* `LayaMixtureAgent(base, experts, router).decide(state, questions, expert = null, compare = false)`
+  routes (one extra encoder pass for the router question) and answers with the chosen head;
+  `expert` forces a head (the routing is still reported as overridden), `compare` adds the base
+  head's answers on the same encoder output. The result carries the routing (probabilities over
+  the kinds, the pick, the threshold), the answers, and encoder / head milliseconds.
+* `LayaMixtureDecisionAdapter(mixture, modelId, onRouted)` is the `DecisionAdapter` of a mixture:
+  every request is routed once and answered by the chosen head with its calibration; the contract
+  has no field for the head that answered, so `onRouted` reports it.
+
+The gated `LayaMoeEvalTest` replays the published evaluation of
+https://github.com/vishalmysore/layaMOE[layaMOE] (Apache-2.0; `safety` and `customer_ops` heads on
+`convaiinnovations/laya-typed-decisions`, 108 hand-labelled cases) and requires the same routing
+and the same answers as the PyTorch FP32 run; it needs `LAYA_TYPED_DECISIONS_PATH` and
+`LAYA_MOE_EXPERTS_PATH`.
+
 == Quantized lane
 
 `LayaModelLoader.load(files, weights, ctx, weightFormFor = LayaWeightForms.q8_0For)` loads the