Skip to content

Mark missing ops in source and show them in the coverage matrix - #1370

Merged
michalharakal merged 3 commits into
developfrom
feature/op-coverage-gaps
Oct 20, 2026
Merged

Mark missing ops in source and show them in the coverage matrix#1370
michalharakal merged 3 commits into
developfrom
feature/op-coverage-gaps

Conversation

@michalharakal

@michalharakal michalharakal commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Why

Real gaps such as TransposedConv2d.forward (NotImplementedError), GroupNormalization.forward (four TODO()s, so groupNorm {} crashes), the SDPA backward (null gradients) and missing StableHLO converters only showed up when code threw. The operator coverage matrix showed — in 65 of 66 backend rows. It had no view of layers, StableHLO lowering or autograd, and nothing in CI kept it current.

What

One marker for a gap. @NotImplemented now takes reason and issue in addition to its backend list. In the docs KSP processor it wins over @InProgress and over a detected backend override. The processor now also reads method-level markers on @Backend classes.

Real backend data. The processor now also runs in skainet-backend-cpu, and generateDocs merges its statusByBackend into the matrix. The cpu column is now 66/67 ✅ instead of all —.

Wider matrix:

  • StableHLO column, from a jvmTest dump of the extended converter registry. 50/67 ✅; the ❌ rows are conv3d, convTranspose1d/2d, elu, leakyRelu, logSoftmax, powScalar, log2, log10, pad2d, unfold, lt, ge and tril, plus three non-op helpers.
  • Backward column, from the tape's dispatch table plus a new stubbedBackwardOpNames set. SDPA shows ❌ stub.
  • NN layers table: every concrete nn Module, with TransposedConv2d and GroupNormalization ❌.
  • ❌ cells link the issue from the marker. Notes now render; previously they were always empty because the generator read message while the processor writes content.

Gaps marked in source:

  • TensorOps.convTranspose2d added as a placeholder with a throwing default (additive API, see the .api dumps).
  • TransposedConv2d.forward now calls convTranspose2d.
  • TransposedConv2d and GroupNormalization are marked @NotImplemented.
  • Removed 35 stale @InProgress("cpu", issue = "task-ops.md#…") markers from implemented DefaultCpuOps methods, and the fake @InProgress("Metal", issue = "GH-1234") in VoidTensorOps.

Guard rails:

  • ./gradlew checkUnmarkedStubs, a new stub-markers leg in build.yml. It fails on NotImplementedError / TODO( in production code that is neither inside a marked declaration nor listed in config/stub-allowlist.txt. Each allowlist entry is path|count|issue, so adding a stub to an allowlisted file also fails.
  • docs.yml regenerates the matrix and fails if the committed pages are stale (the date stamp is ignored). Its path filters now include backend-cpu, compile-hlo and compile-dag.
  • documentation.yml uploaded from a stale path; that is fixed.

Contributor docs: issue-taxonomy.adoc § From a marked gap to an issue; darc-workflow.adoc (C-phase criterion); operator-design.adoc.

Tracking issues

DARC parents with lane sub-issues; 19 lanes are good first issue:

Parent Lanes
#1371 TransposedConv2d / convTranspose2d 7
#1379 GroupNormalization forward 4
#1384 SDPA backward 4
#1389 StableHLO converter coverage 10
#1400 ONNX importer placeholder ops + streaming reader 7
#1408 GGUF IQ1/IQ2/IQ3 + MXFP4 dequant 5
#1414 Remaining non-op stubs 11

The second commit writes these numbers into the @NotImplemented markers and config/stub-allowlist.txt, so every ❌ cell links its issue.

Verification

  • :skainet-lang:skainet-lang-ksp-processor:jvmTest: 14 tests, including a new case for marker precedence, * expansion over backends and layer discovery.
  • jvmTest for skainet-lang-core (1046), skainet-backend-cpu (490), compile-dag (115, 3 skipped as before) and compile-hlo (258). All green.
  • :skainet-backends:skainet-backend-cpu:compileKotlinLinuxX64 and compileKotlinJs (KSP added there).
  • apiDump: additive only (convTranspose2d).
  • checkUnmarkedStubs passes. Adding a bare TODO() to lang-core commonMain makes it fail.
  • generateDocs run twice with --rerun-tasks: git diff -I 'Generated from' is clean.

🤖 Generated with Claude Code

@michalharakal @claude
Mark missing ops in source and surface them in the coverage matrix
5f3792b 
Known gaps (TransposedConv2d, GroupNormalization, SDPA backward, missing
StableHLO converters) only showed up when code threw at runtime; the ops
status matrix had "—" in almost every backend cell and no layer, lowering
or autograd view.

- @NotImplemented gains reason/issue; the docs KSP processor gives it
  precedence over @inprogress and over a detected backend override, and
  reads method-level markers on @backend classes.
- The processor now also runs in skainet-backend-cpu; generateDocs merges
  its statusByBackend, so the cpu column reflects real overrides.
- Matrix gains StableHLO and Backward columns (from jvmTest dumps of the
  converter registry and the backward dispatch table) and an NN-layers
  table; ❌ cells link the tracking issue.
- TensorOps.convTranspose2d added as a marked placeholder;
  TransposedConv2d and GroupNormalization marked @NotImplemented.
- Removed stale @inprogress markers on implemented DefaultCpuOps methods
  and the fake Metal markers in VoidTensorOps.
- checkUnmarkedStubs fails on NotImplementedError / TODO( in production
  code that is neither marked nor in config/stub-allowlist.txt (new CI
  leg stub-markers); docs.yml fails on a stale committed matrix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 19, 2026
Open
Open
Open
Open
Open
Open
Open
@github-actions

github-actions Bot commented Oct 19, 2026

Copy link
Copy Markdown

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

  • Operator documentation: docs/modules/operators/_generated_/
  • JSON schema output: operators.json

Artifacts:

  • Download the documentation-preview-1370 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.

@michalharakal @claude
Link coverage gaps to their DARC tracking issues
d42af95 
Fill issue = "#…" on the @NotImplemented markers (convTranspose2d and
TransposedConv2d #1371, GroupNormalization #1379), note the SDPA stub
backward issue (#1384) and the stub-allowlist entries, and regenerate
the coverage matrix so every ❌ links its issue.

Scope checkUnmarkedStubs to per-project src/ trees so it no longer
overlaps the generated docs output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 19, 2026

Copy link
Copy Markdown

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

  • Operator documentation: docs/modules/operators/_generated_/
  • JSON schema output: operators.json

Artifacts:

  • Download the documentation-preview-1370 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.

@michalharakal @claude
Run coverage dumps before generateDocs in the docs workflow
bd841e6 
generateDocs had no ordering against the HloCoverageDumpTest /
BackwardCoverageDumpTest runs, so on a clean CI checkout it rendered the
matrix before the JSON existed and dropped the StableHLO and Backward
columns, failing the drift check. Order it after both jvmTest tasks and
declare the dump directory as a test output so up-to-date and cached
test runs still provide the files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 19, 2026

Copy link
Copy Markdown

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

  • Operator documentation: docs/modules/operators/_generated_/
  • JSON schema output: operators.json

Artifacts:

  • Download the documentation-preview-1370 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.

@michalharakal
michalharakal merged commit fd0f6da into develop Oct 20, 2026
22 checks passed
@michalharakal
michalharakal deleted the feature/op-coverage-gaps branch October 20, 2026 15:25
This was referenced Oct 21, 2026
Merged
Merged
michalharakal added a commit that referenced this pull request Oct 21, 2026
@michalharakal
release: prepare 0.62.0 -- JVM <-> Kotlin/Wasm parity for compute and IO
65599e8 
CHANGELOG [0.62.0] - 2026-10-21 with the parity headline over #1363, #1364,
#1368, #1369, #1426, #1427, #1428 and #1430, plus entries for the operator
coverage matrix (#1370), the BitNet decode allocation and threading fixes
(#1359, #1360, #1361, #1362), the ClassificationSupport extraction (#1429)
and the toolchain bumps (Kotlin 2.4.21, kotest 6.2.6, Gradle 9.8.1).
VERSION_NAME, CITATION, Antora version, README bom coordinate, What's New
and contributors, Android tutorial coordinates; regenerated operator and
kernel-support matrices; SKEEP-007 rollout step 3 marked shipped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aharakal aharakal aharakal approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@michalharakal @aharakal