You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sub-issue of #1384 ([Feature]: scaledDotProductAttention backward — gradients currently stop at attention).
Lane: 2 · Kotlin core Skill needed: Kotlin; the autodiff tape in DefaultExecutionTape.kt. Size: m (1–2 days) Blocked by:#1385
What to do
Implement scaledDotProductAttentionBackward from the Lane-1 note using existing TensorOps (matmul, softmax, multiply, sum…), recomputing P from Q, K and the saved attributes.
Remove scaledDotProductAttention from stubbedBackwardOpNames.
Run ./gradlew :skainet-compile:skainet-compile-dag:jvmTest, regenerate the matrix (./gradlew :skainet-compile:skainet-compile-hlo:jvmTest --tests '*HloCoverageDumpTest' :skainet-compile:skainet-compile-dag:jvmTest --tests '*BackwardCoverageDumpTest' generateDocs) and commit it.
Acceptance
Backward column shows ✅
Result reported back on the parent issue
Notes
If something here turns out to be wrong or much bigger than the size label, comment on this issue or open a specific bug instead of blocking the parent.
Activity