Fix performance degradation of HIP dot #207
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The results did not match between cuda-stream and hip-stream on the same NVIDIA GPU card (NVIDIA A100 40GB PCIe) when large arraysize is specified. cuda-stream uses the number of SMs to decide dot_num_blocks, which looks more sensible than to use arraysize to determine the parameter. It is used as kernel grid size and iteration count for reduction in the host code. Link: UoB-HPC@9954b7d Signed-off-by: Daisuke Matsuda <[email protected]>
Signed-off-by: Daisuke Matsuda <[email protected]>
Signed-off-by: Daisuke Matsuda <[email protected]>
|
@thomasgibson - can you comment on this as I think you wrote the current HIP version? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The workload of dot calculation is not consistent among the different implementations. The larger the arraysize, the longer it takes for the HIP version to complete.
The HIP version currently uses arraysize to determine 'dot_num_blocks', which is used as kernel grid size and iteration count for reduction in the host code. The CUDA counterpart uses the number of SM (based on GPU specs) to determine 'dot_num_blocks'. The result should be more reliable with the CUDA one because of higher occupancy and more reasonable overhead of reduction on the host.