feat(moore): support paged flash attention decode - #962
Open
voltjia wants to merge 3 commits into
Open
Conversation
voltjia
force-pushed
the
feat/moore-flash-attn-kvcache
branch
from
September 2, 2026 10:14
1b4e3b2 to
6f2a152
Compare
voltjia
force-pushed
the
feat/moore-flash-attn-varlen
branch
from
September 2, 2026 13:34
cd69650 to
70e5008
Compare
voltjia
force-pushed
the
feat/moore-flash-attn-kvcache
branch
from
September 2, 2026 13:34
6f2a152 to
c2d7605
Compare
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FlashAttnWithKvcache.Current base: InfiniOps #819 at
70e50081.Current head:
c2d76051ca21774f0b515dfd8afe59537e200ac2.Scope
This provider implements the decode contract used by InfiniLM. It intentionally rejects KV append, rotary embedding, cache remapping/left padding, local windows, softcap, LSE output, and scalar cache lengths. Unsupported forms fail closed before dispatch.
Stacked Review
This PR contains three decode-only commits on top of #819:
ae2ce9c5- native Moore paged decode providerc6de2f45- paged decode smoke coveragec2d76051- fail closed for scalar cache lengthsgit range-diffmaps the three hardware-tested pre-rebase commits to these current commits one-for-one. After #819 merges, this PR can be retargeted tomasterwithout changing its review diff.Validation
Operator coverage
-1, ALiBi and scale.4 passed, 8 deselected.InfiniLM integration
Validated with InfiniLM #506 and the current Moore runtime/communication prerequisites:
host_segments > 0; no command fell back to wholly eager execution.CI
Clang-format, Ruff, and documentation checks pass. Because this is a stacked PR whose base is #819 rather than
master, the repository backend matrix is not triggered yet; it will run after retargeting.Notes for Reviewers
FlashAttnWithKvcacheAPI directly.-std=c++17to Iluvatar CUDA flags #69 for Moore BF16 collectives; that dependency is independent of this operator PR.