feat(kvarn): KVarN variance-normalized KV cache by asher · Pull Request #116 · asher/gmlx

asher · 2026-09-05T04:42:31Z

No description provided.

…varn)

…ollback gate)

…tarts, APC gate, B=1 fast-path rebuild)

…de stamp, entry salt)

…rts, rollback trim guard, bypass gate lifted)

…inning it resident (bulk prefill leak)

…eme (both were silently benching fp16 KV)

…wen3.5/3.6 ship 256-dim heads)

… now converts; fused verify mask mapping on the owned arm

…alidation

…ate and owned-tree decline

… at 0 when disabled)

…ion, CLI run/chat) with kv-bits x max-kv-size crash fix

…s-eviction)

…ag, blockless inline-state records, offset gate, heavy-skel gate)

…eam kvarn in place (shared rebuild predicate, live per-batch layout signatures, warm-clobber guards)

…in to re-cut, boundary's only carrier; byte budget governs)

…salt on hash+metadata, kvarn tag re-derivation + clone on disk restore, sidecar salt both sides)

…inal boundary skeleton is the only restart-restorable record; chain bounded by W)

…wa, plamo2-shaped three-kind; tag-driven schedule builder; live kvarn engagement family) + budget-knob docs

…stamp wrapper+lm, convert ChunkedKVCache, sidecar trim check

…ructure replay, moves)

…structure replay)

… wrappers

…k; restored caches regrow on write

…ay fused, rotating compaction shrinks and evals once

…indow quantized_kv_start, strict KV_TAIL_TOKENS, salted estimate probe, per-model notes

…rve, rotating kvarn is not liftable, verify width counts the bonus row, spec-path banner logs

…p-only entry salt, scaffolding dedup, pinned installer sweeps

…t request; stress runner takes --scheme/--bits

…e and exit codes documented, kld harness counts chunked layers, stale docstrings and library prints tidied

…FA kernels; GMLX_KVARN_FA=0 keeps the vector kernel

…ix-unit kernels

…t cache

…s, fidelity tables), cli and server docs trimmed to their flags, KVarN attributed to Huawei and beellama.cpp with an MIT notice

…top-1

… 32k

… against them

…ragraph

…etry composed with kvarn slabs and fixed rows)

…s when the GPU caps the threadgroup

… cache allocates, single and batched

…log entry condensed

…nse generator; the runtime model cache is a registry, not a dict, so the salt was never applied

asher added 30 commits August 8, 2026 15:48

feat(kv): kvarn KV-cache scheme phase 1 (run/chat --kv-quant-scheme k…

83dbcad

…varn)

feat(kv): kvarn on the B=1 MTP paths (run/chat/serve spec cache, R1 r…

422bded

…ollback gate)

feat(kv): kvarn on the serve batch path (BatchKVarNKVCache, per-row s…

212a6c8

…tarts, APC gate, B=1 fast-path rebuild)

feat(kv): kvarn APC exact tier (extract/merge, kq_kvarn disk kind, mo…

3deb0cf

…de stamp, entry salt)

feat(kv): qwen3.5 kvarn dispatch arm (owned-route claim, explicit sta…

d0a9d36

…rts, rollback trim guard, bypass gate lifted)

fix(kv): kvarn horizon adopted a view of the per-call rotated slab, p…

4bca38d

…inning it resident (bulk prefill leak)

kld harness: mixed-width kvarnk6v5-style arms for the width matrix

7635368

fix(bench): --bench/--bench-depths honor --kv-bits and --kv-quant-sch…

90c506b

…eme (both were silently benching fp16 KV)

docs: kvarn lever in performance.md, head_dim-128 coverage wording (q…

f21a5f1

…wen3.5/3.6 ship 256-dim heads)

feat(kv): kvarn head_dim 256 (two-slice records) — qwen3.5/3.6 family…

08d56b4

… now converts; fused verify mask mapping on the owned arm

docs(kv): qualify kvarn KLD claim per leg and arch from Qwen3.6-27B v…

cb956c2

…alidation

feat(kv): kvarn head_dim 512 (gemma-4 global layers) with mixed-dim g…

bebfdf4

…ate and owned-tree decline

docs: kvarn head_dim-512 coverage (gemma-4 global layers)

c102eb8

kvarn: rotating-cache base seams (evicted/visible watermark, identity…

011d2ae

… at 0 when disabled)

feat(kv): rotating --max-kv-size kvarn window (entry-compacting evict…

bc415d9

…ion, CLI run/chat) with kv-bits x max-kv-size crash fix

docs: kvarn rotating --max-kv-size coverage (window floor, undo-acros…

a7d8185

…s-eviction)

feat(apc): kvarn layer kind in the ckpt tier (concrete-class layout t…

ec0fafb

…ag, blockless inline-state records, offset gate, heavy-skel gate)

feat(apc): ckpt-active hybrids convert B=1 serve caches to single-str…

e786c16

…eam kvarn in place (shared rebuild predicate, live per-batch layout signatures, warm-clobber guards)

fix(apc): exempt blockless kvarn records from strip-on-extend (no cha…

15f2813

…in to re-cut, boundary's only carrier; byte budget governs)

fix(apc): kvarn disk restart repair (skeleton write mirrors the wire …

24746c7

…salt on hash+metadata, kvarn tag re-derivation + clone on disk restore, sidecar salt both sides)

fix(apc): persist rot window chains with kvarn-layout skeletons (term…

4c8b31b

…inal boundary skeleton is the only restart-restorable record; chain bounded by W)

test(apc): kvarn routing/schedule/engagement rows (kvarn-gdn, kvarn-s…

fb95389

…wa, plamo2-shaped three-kind; tag-driven schedule builder; live kvarn engagement family) + budget-knob docs

kvarn serve: gate stamp and wire salt on an actual-conversion probe, …

f87af51

…stamp wrapper+lm, convert ChunkedKVCache, sidecar trim check

docs: kvarn eligibility head_dim rule + ckpt-tier reuse under kvarn

9314944

merge main @0.4.3 into kvarn (pre-restructure catch-up)

a9875d8

chore(refactor): move modules and tests into subsystem packages (rest…

87f375a

…ructure replay, moves)

chore(refactor): rewrite imports and paths for subsystem packages (re…

e5ea8d0

…structure replay)

fix(cache): merge exact-tier budget trim with kvarn entry salt in APC…

cbaef86

… wrappers

fix(cache): repair imports of relocated kvarn modules

7f29531

merge restructured main into kvarn

13a62ee

asher added 30 commits September 4, 2026 08:59

fix(kv): kvarn clones and disk records carry content, not growth slac…

ac810ed

…k; restored caches regrow on write

fix(kv): kvarn fused-decode cap per (head_dim, width), narrow legs st…

588d880

…ay fused, rotating compaction shrinks and evals once

fix(serve): kvarn split widths from KV_KEY_BITS/KV_VALUE_BITS, load-w…

957bce8

…indow quantized_kv_start, strict KV_TAIL_TOKENS, salted estimate probe, per-model notes

fix(spec): per-model MTP KV notes, CLI declines window stacks like se…

4e13203

…rve, rotating kvarn is not liftable, verify width counts the bonus row, spec-path banner logs

test(kvarn): ops gate and APC-arms fixture in the root conftest, stam…

3ba9eb2

…p-only entry salt, scaffolding dedup, pinned installer sweeps

test(e2e): runners gate on the served kv_quant verdict after the firs…

e2a913a

…t request; stress runner takes --scheme/--bits

docs(kvarn): cache-share figures from the cost model, width precedenc…

31bd4d3

…e and exit codes documented, kld harness counts chunked layers, stale docstrings and library prints tidied

perf(kvarn): verify width 2-8 attends the records on the matrix-unit …

41e1787

…FA kernels; GMLX_KVARN_FA=0 keeps the vector kernel

docs(performance): kvarn verify rounds cost a decode step on the matr…

159bcd7

…ix-unit kernels

chore(scripts): kld harness gains a turboN arm on mlx-vlm's TurboQuan…

db2a6ed

…t cache

docs(kv): one KV quantization section (schemes, coverage, when it pay…

c831ba3

…s, fidelity tables), cli and server docs trimmed to their flags, KVarN attributed to Huawei and beellama.cpp with an MIT notice

docs(kv): fidelity tables in plain decimals

375f323

docs(kv): split-width row sorts by its mean width

ed4560b

docs(kv): Qwen3.8-27B 32k fidelity rows

fded89a

docs(kv): fidelity tables carry the decode p99 beside the median and …

d9a7896

…top-1

docs(kv): how to read median, p99 and top-1 when they disagree

06d3c01

docs(kv): TurboQuant rows on the 9B, affine fill-in on Qwen3.6-27B at…

94240dc

… 32k

docs(kv): where TurboQuant lands between affine and kvarn

31c617f

docs(kv): one 27B subject in the fidelity tables

3a02782

docs(kv): TurboQuant and k6 v5 rows on Qwen3.8-27B, claims re-checked…

481d479

… against them

docs(kv): TurboQuant leaves the fidelity tables, summarized in one pa…

5950b7e

…ragraph

Merge branch 'main' into kvarn (capacity pricing per cache-stack geom…

32ffe1d

…etry composed with kvarn slabs and fixed rows)

fix(kvarn): fused decode splits the GQA fold and probes the FA kernel…

ec543e2

…s when the GPU caps the threadgroup

test(capacity): kvarn boot pricing checked against the bytes a filled…

6de5bda

… cache allocates, single and batched

dep: bump mlx-kquant floor 0.4.6

a54a3a5

chore(deps): CI pin and floor comment follow mlx-kquant 0.4.6, change…

dbe824b

…log entry condensed

fix(serve): kvarn APC wire salt reads the stamped model off the respo…

b453578

…nse generator; the runtime model cache is a registry, not a dict, so the salt was never applied

fix(serve): estimate ckpt probe signs with the stamped kvarn layout

98b89f9

fix(spec): injected packed B=1 rows lift to fp16 batch caches

d6e7902

docs(changelog): injected packed-row lift fix

673c0d0

Navigation Menu

feat(kvarn): KVarN variance-normalized KV cache - #116

feat(kvarn): KVarN variance-normalized KV cache#116
asher wants to merge 116 commits into
mainfrom
kvarn

asher commented Sep 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

asher commented Sep 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant