Skip to content

[Feature]: GGUF dequantization for the IQ1/IQ2/IQ3 families and MXFP4 #1408

Open
0 of 5 issues completed
Open

Description

@michalharakal

🧠 D: DOCUMENT — Problem & Opportunity

DequantOps (skainet-io/skainet-io-gguf/src/commonMain/kotlin/sk/ainet/io/gguf/dequant/DequantOps.kt) rejects GGUF tensors of type IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ1_S, IQ1_M and MXFP4, so many current llama.cpp quantizations cannot be loaded. IQ4_NL and IQ4_XS are implemented and are the pattern to follow.

Summary: add bit-exact dequantization (vs llama.cpp ggml-quants.c) for the remaining types, one family per lane.


🔍 A: ASSESS — Feasibility & Impact

  • Feasibility: medium. IQ2/IQ3/IQ1 need llama.cpp's codebook grids (iq2xxs_grid, iq3s_grid, …) ported as constant tables; MXFP4 is a simple e2m1 + shared-exponent format.
  • Risk: bit-exactness — verify against llama.cpp output, not against the paper.

📚 R: RESEARCH — What Must Be Understood First?

  • Grid tables + block layouts extracted from llama.cpp (Lane 1)

🛠️ C: CODE — Implementation Plan

  • One lane per type family

Acceptance Criteria

  • Each type dequantizes bit-identically to llama.cpp on a reference tensor

Lane breakdown

Lanes 0, 3, 5 skipped.

New here? Read Getting started as a contributor and the Issue taxonomy. Lanes marked good first issue assume no prior SKaiNET codebase knowledge. The gap markers and the coverage matrix were introduced in #1370.

Metadata

Metadata

Assignees

No one assigned

    Labels

    darcFeature driven through the DARC workflow (Document / Assess / Research / Code)enhancementNew feature or requestfile-format:ggufGGUF file formatquantizationtrackingParent/tracking issue with sub-issues

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions