Matmul tutorial code still works when K % BLOCK_SIZE_K != 0?

[zw2326]

In the Matrix Multiplication tutorial, there is a comment and an assertion that claim if K % BLOCK_SIZE_K != 0, there would be out-of-bound memory access (which makes sense).

But if I change the input matrices to shape (512, 513) and (513, 512) (i.e. K=513), with BLOCK_SIZE_K=32, the code still works and result is close to cublas result. What's going on?

[jonathlela]

Have you tried with the latest version of triton ? By reproducing the tutorial and changing the shapes as above, I have:

triton_output=tensor([[nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan],
        ...,
        [nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan]], device='cuda:0',
       dtype=torch.float16)
torch_output=tensor([[-24.5938,  10.2188,  -5.4023,  ...,  -9.2500,   4.5273,  -4.9922],
        [ -9.6953, -17.7344, -22.6719,  ..., -73.6250, -58.8125,  -6.9531],
        [ 33.5625,   3.2031,  39.4062,  ..., -14.7734,   7.2344,  -7.7031],
        ...,
        [ 24.7188,  -1.1152, -27.5781,  ..., -14.5000,  18.1250,  16.4844],
        [-19.1094,  -9.7891,   8.1094,  ...,  21.5156,  -0.6982, -18.0469],
        [  7.1797,  33.6875, -13.9609,  ...,  -8.8828, -19.1719,  18.7812]],
       device='cuda:0', dtype=torch.float16)
❌ Triton and Torch differ

as expected.