Skip to content

Support sliding-window layers on the smart_mask cache ring - #21929

Open
YIWENX14 wants to merge 2 commits into
pytorch:mainfrom
YIWENX14:export-D116524534
Open

Support sliding-window layers on the smart_mask cache ring#21929
YIWENX14 wants to merge 2 commits into
pytorch:mainfrom
YIWENX14:export-D116524534

Conversation

@YIWENX14

Copy link
Copy Markdown
Contributor

Summary: Sliding-window (local) attention layers previously worked only with the shift_pointer KV cache layout. This makes them work with smart_mask as well, so chunked prefill matches single-chunk prefill under either layout.

Differential Revision: D116524534

Summary:

Chunked prefill of a sliding-window (local) attention layer let every query row see more keys than the window allows, so multi-chunk prefill did not match single-chunk prefill. This makes the cache region of the attention mask per-query-row in `StaticAttentionIOManager`.

Reviewed By: billmguo

Differential Revision: D116080453
Summary: Sliding-window (local) attention layers previously worked only with the `shift_pointer` KV cache layout. This makes them work with `smart_mask` as well, so chunked prefill matches single-chunk prefill under either layout.

Differential Revision: D116524534
@pytorch-bot

pytorch-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21929

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Unclassified Failure

As of commit b381b90 with merge base ed2d9ec (image):

NEW FAILURES - The following jobs have failed:

  • Cadence Build & Test / hifi-build / hifi4 (gh)
    ##[error]Refusing to check out fork pull request code from a 'pull_request_target' workflow. This workflow runs with the base repository's GITHUB_TOKEN, secrets, default-branch cache scope, and runner access. Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set 'allow-unsafe-pr-checkout: true' on the actions/checkout step.
  • Cadence Build & Test / vision-build / vision (gh)
    ##[error]Refusing to check out fork pull request code from a 'pull_request_target' workflow. This workflow runs with the base repository's GITHUB_TOKEN, secrets, default-branch cache scope, and runner access. Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set 'allow-unsafe-pr-checkout: true' on the actions/checkout step.

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

  • Labeler / label (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
    ##[error]HttpError: API rate limit exceeded for installation. If you reach out to GitHub Support for help, please include the request ID 7400:4C50A:3B769:C8995:6A84EBF0 and timestamp 2026-08-18 23:34:08 UTC. For more on scraping GitHub and how it may affect your rights, please review our Terms of Service (https://docs.github.com/en/site-policy/github-terms/github-terms-of-service)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 18, 2026
@meta-codesync

meta-codesync Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

@YIWENX14 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D116524534.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant