Skip to content

[AUTOGENERATED] [rocm7.1_internal_testing] [ROCm] Use fine-grain fence in reduction #2562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[AUTOGENERATED] [rocm7.1_internal_testing] [ROCm] Use fine-grain fence in reduction #2562

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions aten/src/ATen/native/cuda/Reduce.cuh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -797,14 +797,23 @@ struct ReduceOp {
if (should_store) {
index_t offset = config.staging_memory_offset(blockIdx.y);
reduce_buffer[offset] = value;
#ifdef USE_ROCM
__builtin_amdgcn_fence(__ATOMIC_RELEASE, "agent"); // make sure writes are globally visible
#endif
}

#ifndef USE_ROCM
__threadfence(); // make sure writes are globally visible
#endif
__syncthreads(); // if multiple warps in this block wrote to staging, make sure they're all done
bool is_last_block_done = mark_block_finished();

if (is_last_block_done) {
#ifdef USE_ROCM
__builtin_amdgcn_fence(__ATOMIC_ACQUIRE, "agent"); // complete the acquire pattern after release
#else
__threadfence(); // complete the acquire pattern after atomic
#endif
for (auto &v : value) {
v = ident;
}
Expand Down