[WIP] [StreamK] #782
[WIP] [StreamK] #782
Conversation
|
|
||
| if STREAMK_TILES == 0: | ||
| return | ||
|
|
There was a problem hiding this comment.
in fact, I was trying to use STREAK_TILES to avoid this "if" with line 113 and 115, otherwise, we could use full tiles and total tiles for this if ?
There was a problem hiding this comment.
In theory, when STREAM_TILES==0, start_iter == last_iter and we should not need the while loop. However, they are not constant, so compiler cannot optimize them out. I found this if a very nice way to early return :)
| else: | ||
| print("Unknown target") | ||
| exit(0) | ||
|
|
There was a problem hiding this comment.
how do we deal with MI308 80/64 CUs ?
There was a problem hiding this comment.
idk. There must be a way (hopefully a torch API) to query the number of cus from the GPU. Let me find it.
| kpack = 1 | ||
| else: | ||
| print("Unknown target") | ||
| exit(0) |
| loop_k -= 1 | ||
|
|
||
| tl.assume(loop_k > 1) | ||
|
|
| acc_dtype = tl.float32 if C.type.element_ty != tl.int8 else tl.int32 | ||
|
|
||
| for tile_id in range(pid, total_full_tiles, NUM_SMS): | ||
| for tile_id in tl.range(pid, total_full_tiles, NUM_SMS, flatten=True): |
There was a problem hiding this comment.
have you observed any perf improvement by enable this loop fusion ?
There was a problem hiding this comment.
no, this is just an experiment and it doesn't compile. I filed a ticket for it https://github.com/ROCm/triton-internal/issues/784
No description provided.