[UR][CUDA][HIP] Unify queue handling between adapters #17641
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The CUDA and HIP adapters are both using a nearly identical complicated queue that handles creating an out-of-order UR queue from in-order CUDA/HIP streams. This patch extracts all of the queue logic into a separate templated class that can be used by both adapters. Beyond removing a lot of duplicated code, it also makes it a lot easier to maintain. There was a few functional differences between the queues in both adapters, but mostly due to fixes done in the CUDA adapter that were not ported to the HIP adapter. There might be more but I found at least one race condition (intel#15100) and one performance issue (intel#6333) that weren't fixed in the HIP adapter. This patch uses the CUDA version of the queue as a base for the generic queue, and will thus fix for HIP the race condition and performance issue mentioned above. This code is quite complex, so this patch also aimed to minimize any other changes beyond the structural changes needed to share the code. However it did do the following changes in the two adapters: CUDA: * Rename `ur_stream_guard_` to `ur_stream_guard` * Rename `getNextEventID` to `getNextEventId` * Remove duplicate `get_device` getter, use `getDevice` instead HIP: * Fix queue finish so it doesn't fail when no streams need to be synchronized
Capturing the result is no longer needed
npmiller
force-pushed
the
cuda-hip-common
branch
from
cb3bdac to
8b178a0
Compare
aarongreig
reviewed
Mar 26, 2025
| LastSyncComputeStreams{0}, LastSyncTransferStreams{0}, Flags(Flags), | ||
| URFlags(URFlags), Priority(Priority), HasOwnership{BackendOwns} { | ||
| urContextRetain(Context); | ||
| urDeviceRetain(Device); |
Contributor
There was a problem hiding this comment.
DeviceRetain/Release can be removed, it's a no-op unless we expect Device to be a subdevice (I have a ticket open to rename and better document those entry points)
Device retain release is a no-op
aarongreig
changed the title
aarongreig
approved these changes
Mar 26, 2025
keyradical
reviewed
Mar 27, 2025
keyradical
approved these changes
Mar 27, 2025
jchlanda
approved these changes
Apr 2, 2025
Contributor
Author
|
@intel/llvm-gatekeepers I believe this is ready to merge
And this patch only affects CUDA and HIP, so missing PVC and Arc testing shouldn't be an issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The CUDA and HIP adapters are both using a nearly identical complicated queue that handles creating an out-of-order UR queue from in-order CUDA/HIP streams.
This patch extracts all of the queue logic into a separate templated class that can be used by both adapters. Beyond removing a lot of duplicated code, it also makes it a lot easier to maintain.
There was a few functional differences between the queues in both adapters, but mostly due to fixes done in the CUDA adapter that were not ported to the HIP adapter. There might be more but I found at least one race condition (#15100) and one performance issue (#6333) that weren't fixed in the HIP adapter.
This patch uses the CUDA version of the queue as a base for the generic queue, and will thus fix for HIP the race condition and performance issue mentioned above.
This code is quite complex, so this patch also aimed to minimize any other changes beyond the structural changes needed to share the code. However it did do the following changes in the two adapters:
stream_queue.hpp:urDeviceRetain/Release: essentially a no-opCUDA:
ur_stream_guard_tour_stream_guardgetNextEventIDtogetNextEventIdget_devicegetter, usegetDeviceinsteadHIP: