Skip to content

[#2517] fix(client): IllegalReferenceCountException about ShuffleBlockInfo #2638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zuston merged 3 commits into from
Oct 20, 2025
Merged

[#2517] fix(client): IllegalReferenceCountException about ShuffleBlockInfo #2638

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4267520
[#2517] fix(client): IllegalReferenceCountException about ShuffleBloc…
cchung100m Oct 9, 2025
d56afd3
[#2517] mvn spotless:apply
cchung100m Oct 9, 2025
93eba64
[#2517] refactor: fast fail will be better than slient sucess
cchung100m Oct 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions ...nal-client/src/main/java/org/apache/uniffle/client/impl/grpc/ShuffleServerGrpcClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,12 @@ public RssSendShuffleDataResponse sendShuffleData(RssSendShuffleDataRequest requ
List<ShuffleBlock> shuffleBlocks = Lists.newArrayList();
int partitionRequireSize = 0;
for (ShuffleBlockInfo sbi : ptb.getValue()) {
if (sbi.getData().refCnt() == 0) {
Copy link
Member

@zuston zuston Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not safe, I think the fast fail will be better than slient sucess.
But for this case, I haven't found the root cause about this.

Copy link
Collaborator Author

@cchung100m cchung100m Oct 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zuston

Thanks for the reply. I updated the part you mentioned.

Since we throw the proper RssException here, we might leverage the retry machanism which introduced by #2608 to solve the data race?

Copy link
Member

@zuston zuston Oct 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this PR's change looks good to me. I will take a deep look #2608 assp in this week

throw new RssException(
"Detected a ShuffleBlockInfo with a released buffer (refCnt=0) for blockId["
+ sbi.getBlockId()
+ "].");
}
shuffleBlocks.add(
ShuffleBlock.newBuilder()
.setBlockId(sbi.getBlockId())
Expand Down
Loading