Repository navigation
Conversation
[Medium impact] The PR appears safe to merge, with a non-blocking gap in MXFP8 CUDA graph test coverage. Findings
SummaryAdds expert-parallel communication overlapped with grouped GEMMs for the PyTorch MoE operation sequence.
No new actionable issues were established. Diagram%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Input tokens and routing] --> B[Dispatch overlaps FC1 and SwiGLU]
B --> C[FC2 overlaps combine]
C --> D[Output tokens]
D --> E[Backward dispatch overlaps FC2 gradient]
E --> F[FC1 gradient overlaps backward combine]
F --> G[Input and routing gradients]
E --> H[FC2 weight gradient]
F --> I[FC1 weight gradient]
Reviews (2) · Last reviewed commit: "[Common][PyTorch] Add MoE EP communicati..." · Reviewed by Greptile |
| config = self._overlap_config() | ||
| ops = self._moe_ops(config, self._make_buffer_from_config(config)) | ||
| ref_ops = self._moe_ops(config, self._make_buffer_from_config(config)) |
There was a problem hiding this comment.
test_moe_overlap_cuda_graph builds both models with _overlap_config() and never enables MXFP8 autocast. It therefore misses capture and replay of the new shared dispatch window, requantization, and MXFP8 weight gradients. The tests could pass while that training path breaks.
Run this test with both BF16 and MXFP8BlockScaling, using MXFP8 transport for the latter. Compare replayed outputs and gradients after changing the inputs and routing.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…) with in-call weight gradients for BF16 and MXFP8 Signed-off-by: Phuong Nguyen <phuonguyen@nvidia.com>
82d9201 to
7c92f4a
Compare
Description
Please include a brief summary of the changes, relevant motivation and context.
Fixes # (issue)
Type of change
Changes
Please list the changes introduced in this PR:
Checklist: