Fix SM89 FP8 blockwise scale indexing under threadblock swizzling - #3394
Open
Noperi0r wants to merge 1 commit into
Open
Fix SM89 FP8 blockwise scale indexing under threadblock swizzling#3394Noperi0r wants to merge 1 commit into
Noperi0r wants to merge 1 commit into
Conversation
Author
|
Hi @hwu36, could you please help route this PR to the appropriate reviewer when you have a chance? It fixes a correctness issue in the SM89 FP8 CUTLASS 2.x |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
MmaMultistageBlockwisederives theScaleAandScaleBrow indices from the raw CUDA block coordinates,blockIdx.xandblockIdx.y.With
GemmIdentityThreadblockSwizzle<N>andN > 1, the physical launch coordinates do not necessarily match the logical GEMM tile coordinates returned by the swizzle and used by the operand iterators and epilogue. The blockwise mainloop can therefore select scale rows that do not correspond to the M and N tiles being processed. This produces incorrect output and can read beyond exact-size scale tensors.Fix
Pass the post-swizzle logical M and N tile coordinates from
GemmUniversalBlockwisetoMmaMultistageBlockwise, and use those coordinates when calculating the blockwise scale row indices instead of readingblockIdxdirectly.Add an SM89 FP8 blockwise regression test covering:
GemmIdentityThreadblockSwizzle<1>as a positive control.GemmIdentityThreadblockSwizzle<2>as the swizzled case that exposes the incorrect indexing.The regression uses a
256x256x128problem with a64x128x128threadblock shape, creating multiple logical tiles along both M and N. A and B are filled with ones, while each 128-row M scale block and 128-column N scale block uses a distinct power-of-two value, so the expected result can be computed directly for every output element.The scale tensors are allocated at their exact required size, and the full output matrix is compared exactly. This makes an incorrect scale-row selection observable and prevents it from being hidden by unused padding.
Validation
Tested on an NVIDIA GeForce RTX 4060 Laptop GPU (Ada, SM89) with CUDA 12.8 under WSL2 Ubuntu 24.04.
Identity<2>case produced 49,152 output mismatches.Identity<2>case reported 0 errors.Identity<4>, multiple non-split K scale blocks, and the public ColumnMajor adapter.git diff --checkpassed.