Repository navigation
Conversation
6 of 8 tasks
pggPL
force-pushed
the
layernorm_compile
branch
from
September 3, 2026 12:42
90f05e3 to
9422845
Compare
pggPL
changed the base branch from
layernorm_compile_refactor
to
layernorm_compile_prep
September 3, 2026 12:42
6 of 8 tasks
pggPL
force-pushed
the
layernorm_compile
branch
from
September 3, 2026 12:49
9422845 to
4cfffaf
Compare
pggPL
force-pushed
the
layernorm_compile
branch
from
September 17, 2026 11:00
4cfffaf to
dc5fe4c
Compare
pggPL
force-pushed
the
layernorm_compile
branch
from
September 17, 2026 16:56
01275cd to
e09c276
Compare
Register LayerNorm module forward and backward custom ops with matching fake tensor metadata, shared eligibility checks, and compiled weight caches. Extend the existing compile tests to cover the LayerNorm modules. Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
pggPL
force-pushed
the
layernorm_compile
branch
from
October 29, 2026 12:07
70e4b7d to
83641e8
Compare
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Keep the merged optimize_for_gemm implementation and adapt the LayerNormLinear compile path to the current check_gemm_dims signature. Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Restore Linear and fused-attention backward to main. Make normalization gradient setup optional and remove the unused SHAPE field kind and its test. Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add full-graph
torch.compilesupport forLayerNormLinearandLayerNormMLP, including backward, dynamic batch dimensions and normalization outputs.Targets
main. This branch includes the NVFP4 packed-buffer, TensorSpec and Bias correctness fixes from NVIDIA/TransformerEngine#3628.Changes
Describe forward outputs, saved tensors, aliases and backward gradients for both LayerNorm modules; retain eager fallbacks for unsupported configurations.
Route gradients from both normalization outputs through an optional backward hook. Share quantizer validation between the LayerNorm modules.
Validate both MLP GEMMs, including gated activations and sequence-parallel dimensions.
Support compiled FP8 weight caching in both LayerNorm modules, following Linear: replace the cache on the first microbatch and reuse it on subsequent microbatches, including both MLP weights. GPU-controlled cache updates and fused wgrad accumulation retain eager fallbacks.
Share the cache regression across Linear, LayerNormLinear and LayerNormMLP, checking independent eager outputs/gradients, cache refresh/reuse, optimizer updates, immediate/deferred backward and CUDA Graph replay.
Describe LayerNorm/RMSNorm statistics and activation outputs through
dynamo.fake, using the quantizer layout contract merged in NVIDIA/TransformerEngine#3817. Preserve gathered LayerNormMLP output shapes.Parameterize module compile tests across recipes, shapes and modes, with LayerNorm-specific outputs, gradients and eager fallbacks. Retain the non-matrix Linear regressions and expanded cold CUDA Graph recipe coverage inherited from [PyTorch] Fix NVFP4 metadata in compiled Linear operations NVIDIA/TransformerEngine#3628.
Cover metadata reconstruction and CustomRecipe MXFP8 2D cases in the compiled normalization scale-layout matrix.
Validation
2026-10-23 RMSNorm follow-up: the new metadata regression detects two failures on the old helper. The corrected selection passes 113 tests, with 83 hardware skips on Ada. The final helper also passes all 32 metadata cases after the lint-only condition refactor. Native MXFP8 2D compiled forward/backward cases are added for supported hardware but remain unverified on Ada. Existing native libraries reused.
2026-10-23, workstation RTX 5880 Ada, PyTorch
2.15.0a0+875d815502.nvinternal.main; existing native libraries reused, no fresh native build:dd02f3f7: fake-versus-real outputs, LayerNorm variants and inherited Linear cache selection: 85 passed, 15 skipped.fused_mla_q_uproj.py:19C0411 import-order diagnostic.Historical validation before this merge/cache extension:
B100/SM100, PyTorch
2.15.0a0+875d815502.nvinternal.main, CUDA 13.4; existing October 7 native libraries reused, no fresh native build.attention/fused_mla_q_uproj.pyin the GPU container; no unrelated source cleanup is included. Pylint on all changed production files passed.No complete compile-suite rerun or distributed TP/SP/Userbuffers validation. The gathered-output test emulates the collective and does not establish multi-GPU correctness.