Skip to content

[Lane 2 · kotlin-core] SDPA backward — implement scaledDotProductAttentionBackward #1386

Open

Description

@michalharakal

Sub-issue of #1384 ([Feature]: scaledDotProductAttention backward — gradients currently stop at attention).

Lane: 2 · Kotlin core
Skill needed: Kotlin; the autodiff tape in DefaultExecutionTape.kt.
Size: m (1–2 days)
Blocked by: #1385

What to do

  1. Implement scaledDotProductAttentionBackward from the Lane-1 note using existing TensorOps (matmul, softmax, multiply, sum…), recomputing P from Q, K and the saved attributes.
  2. Remove scaledDotProductAttention from stubbedBackwardOpNames.
  3. Run ./gradlew :skainet-compile:skainet-compile-dag:jvmTest, regenerate the matrix (./gradlew :skainet-compile:skainet-compile-hlo:jvmTest --tests '*HloCoverageDumpTest' :skainet-compile:skainet-compile-dag:jvmTest --tests '*BackwardCoverageDumpTest' generateDocs) and commit it.

Acceptance

  • Backward column shows ✅
  • Result reported back on the parent issue

Notes

If something here turns out to be wrong or much bigger than the size label, comment on this issue or open a specific bug instead of blocking the parent.

Coverage matrix: docs/modules/ROOT/pages/reference/ops-status-matrix.adoc · How lanes work: Issue taxonomy

Metadata

Metadata

Assignees

No one assigned

    Labels

    codingImplementation task (DARC: C)size:m1-2 daysskill:kotlin-coreKotlin implementation in commonMainsub-issueSub-issue of a tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions