Experimental rocSHMEM support #356
Conversation
alextmagro
requested review from
ipanfilo,
wangye805 and
wenchenvincent
as code owners
| "Clean up and finalize the NVSHMEM communication backend and free associated resources", | ||
| py::call_guard<py::gil_scoped_release>()); | ||
| #else | ||
| // rocshmem functions |
There was a problem hiding this comment.
I think we don't need these function names at python level. We can just use the nvshmem wrappers at python level.
| case WaitKind::ROCSHMEM_WAIT: | ||
| // rocshmem__ulonglong_wait_until_on_stream(sig_addr, | ||
| // ROCSHMEM_CMP_EQ, | ||
| // wait_value, |
There was a problem hiding this comment.
Let's add a warning here saying that rocshmem_wait is not supported on ROCm.
|
@alextmagro Could you please address the comments? |
alextmagro
force-pushed
the
rocshmem_enablement
branch
4 times, most recently
from
c958b2b to
53bc808
Compare
I have made the changes requested and rebased. A few issues with submodules that were added accidentally but things should be clean now -- will run level 3. |
wenchenvincent
left a comment
wenchenvincent
left a comment
There was a problem hiding this comment.
LGTM. Was the CI failure due to networking issue?
alextmagro
force-pushed
the
rocshmem_enablement
branch
from
53bc808 to
588aebd
Compare
alextmagro
force-pushed
the
rocshmem_enablement
branch
from
e98d964 to
e82a4b1
Compare
Requires support for rocSHMEM within pytorch for full functionality -- Able to bootstrap with PMI codechanges but not safe or recommended.
rocSHMEM also lacks stream specific functionality, and signaling. Given that Pytorch communications tend to be stream focused, this is a feature we need from rocSHMEM team.