Repository navigation
[Feature]: GGUF dequantization for the IQ1/IQ2/IQ3 families and MXFP4 #1408
Copy link
Copy link
Open
0 / 50 of 5 issues completedOpen
0 / 50 of 5 issues completed
Copy link
Labels
darcFeature driven through the DARC workflow (Document / Assess / Research / Code)Feature driven through the DARC workflow (Document / Assess / Research / Code)enhancementNew feature or requestNew feature or requestfile-format:ggufGGUF file formatGGUF file formatquantizationtrackingParent/tracking issue with sub-issuesParent/tracking issue with sub-issues
Description
🧠 D: DOCUMENT — Problem & Opportunity
DequantOps (skainet-io/skainet-io-gguf/src/commonMain/kotlin/sk/ainet/io/gguf/dequant/DequantOps.kt) rejects GGUF tensors of type IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ1_S, IQ1_M and MXFP4, so many current llama.cpp quantizations cannot be loaded. IQ4_NL and IQ4_XS are implemented and are the pattern to follow.
Summary: add bit-exact dequantization (vs llama.cpp ggml-quants.c) for the remaining types, one family per lane.
🔍 A: ASSESS — Feasibility & Impact
- Feasibility: medium. IQ2/IQ3/IQ1 need llama.cpp's codebook grids (
iq2xxs_grid,iq3s_grid, …) ported as constant tables; MXFP4 is a simple e2m1 + shared-exponent format. - Risk: bit-exactness — verify against llama.cpp output, not against the paper.
📚 R: RESEARCH — What Must Be Understood First?
- Grid tables + block layouts extracted from llama.cpp (Lane 1)
🛠️ C: CODE — Implementation Plan
- One lane per type family
Acceptance Criteria
- Each type dequantizes bit-identically to llama.cpp on a reference tensor
Lane breakdown
- [Lane 1 · numerics] GGUF IQ dequant — llama.cpp reference blocks and grid tables #1409 — [Lane 1 · numerics] llama.cpp reference blocks and grid tables (
size:s, good first issue) - [Lane 2 · kotlin-core] GGUF IQ dequant — MXFP4 dequantization #1410 — [Lane 2 · kotlin-core] MXFP4 dequantization (
size:s) - [Lane 2 · kotlin-core] GGUF IQ dequant — IQ2_XXS / IQ2_XS / IQ2_S dequantization #1411 — [Lane 2 · kotlin-core] IQ2_XXS / IQ2_XS / IQ2_S dequantization (
size:m) - [Lane 2 · kotlin-core] GGUF IQ dequant — IQ3_XXS / IQ3_S dequantization #1412 — [Lane 2 · kotlin-core] IQ3_XXS / IQ3_S dequantization (
size:m) - [Lane 2 · kotlin-core] GGUF IQ dequant — IQ1_S / IQ1_M dequantization #1413 — [Lane 2 · kotlin-core] IQ1_S / IQ1_M dequantization (
size:m)
Lanes 0, 3, 5 skipped.
New here? Read Getting started as a contributor and the Issue taxonomy. Lanes marked good first issue assume no prior SKaiNET codebase knowledge. The gap markers and the coverage matrix were introduced in #1370.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
darcFeature driven through the DARC workflow (Document / Assess / Research / Code)Feature driven through the DARC workflow (Document / Assess / Research / Code)enhancementNew feature or requestNew feature or requestfile-format:ggufGGUF file formatGGUF file formatquantizationtrackingParent/tracking issue with sub-issuesParent/tracking issue with sub-issues
Activity