Skip to content

fix(nvfp4): derive the colwise scale window count from the block size - #2

Closed
NolanHo wants to merge 1 commit into
mainfrom
fix/colwise-scale-window-count
Closed

NolanHo wants to merge 1 commit into
mainfrom
fix/colwise-scale-window-count

Conversation

@NolanHo

@NolanHo NolanHo commented Nov 18, 2026

Copy link
Copy Markdown

tid_Y_colwise is computed with a bare % 4:

const int tid_Y_colwise = (thread_lane / 2 + warp) % 4;

That 4 is BUFF_IN_DIM_Y / SCALE_DIM — the number of SCALE_DIM-tall windows in sIn — written out
as a literal. It is correct today only because BUFF_IN_DIM_Y / SCALE_DIM == 4 at SCALE_DIM = 16.

What goes wrong if the block size moves

At a 32-element block the window count is 2, so tid_Y_colwise still reaches 3 and the derived
offsets walk past the buffers:

thread_offset_Y_colwise = tid_Y_colwise * SCALE_DIM  ->  {0, 32, 64, 96}   (sIn has 64 rows)
out_tr_thread_offset_X  = thread_offset_Y_colwise/2  ->  {0, 16, 32, 48}   (BUFF_OUT_TR_DIM_X == 32)

The 8-byte st_shared_b64 at column offset 32 or 48 crosses the row and runs off the end of
sOut_tr. sSFrowwise is allocated immediately after that buffer, so the overflow lands in the
rowwise scale array: measured, rows 0/1 and 4/5 of every scale column are overwritten while rows 2/3
stay intact — matching the two overflow addresses exactly. compute-sanitizer --tool memcheck reports
no error, because the store is in-bounds of the dynamic shared allocation as a whole.

The change

Derive the count instead of writing it out, and pin the divisibility it depends on:

static_assert(BUFF_IN_DIM_Y % SCALE_DIM == 0, "SCALE_DIM must divide the input buffer height");
constexpr int WINDOWS_PER_BUFF_IN = BUFF_IN_DIM_Y / SCALE_DIM;
const int tid_Y_colwise = (thread_lane / 2 + warp) % WINDOWS_PER_BUFF_IN;

At SCALE_DIM = 16 the derived value is 4, so behaviour and numerics are unchanged — this only
removes the hidden coupling. No functional dependency on the block size is introduced here; the
literal is simply replaced by what it always meant.

`tid_Y_colwise` was computed with a bare `% 4`, which is only correct because
`BUFF_IN_DIM_Y / SCALE_DIM` happens to be 4 at the current `SCALE_DIM = 16`.
It is an unexplained literal standing in for a derived quantity, and any change
to the block size makes the index run past the input buffer: on a 32-element
block the count is 2, so `tid_Y_colwise` reaches 3, `thread_offset_Y_colwise`
reaches 96 against a 64-row `sIn`, and the derived `out_tr_thread_offset_X`
reaches 48 against a `BUFF_OUT_TR_DIM_X` of 32 — the 8-byte shared store then
crosses the row and writes past the end of `sOut_tr`.

Behaviour is unchanged at `SCALE_DIM = 16` (the derived value is 4), so this is
a correctness-of-expression change with no numerical effect. The `static_assert`
pins the divisibility the derivation depends on.
@NolanHo

NolanHo commented Nov 18, 2026

Copy link
Copy Markdown
Author

Closing this: on its own it is a no-op cleanup with no behavioural benefit, since SCALE_DIM is a fixed constexpr and the literal is equivalent to the derived value in every current configuration. It belongs as the first commit of the MXFP4 block-size PR, where the literal actually becomes an out-of-bounds write and the change has a reason a reviewer can check.

@NolanHo NolanHo closed this Nov 18, 2026
@NolanHo
NolanHo deleted the fix/colwise-scale-window-count branch November 18, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant