Skip to content

Unlink delegate in ReleasableBytesReference once ref count reaches 0 #127058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 6, 2025
Merged

Unlink delegate in ReleasableBytesReference once ref count reaches 0 #127058

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
257494e
Unlink delegate in ReleasableBytesReference once ref count reaches 0
original-brownbear Apr 18, 2025
1a32b71
fix comment
original-brownbear Apr 18, 2025
2fd23a1
fix assertion
original-brownbear Apr 18, 2025
d97f3be
Merge remote-tracking branch 'elastic/main' into break-gc-chain-delegate
original-brownbear May 6, 2025
e09ceed
simplify assertion
original-brownbear May 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions server/src/main/java/org/elasticsearch/common/bytes/ReleasableBytesReference.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ public final class ReleasableBytesReference implements RefCounted, Releasable, B

private static final ReleasableBytesReference EMPTY = new ReleasableBytesReference(BytesArray.EMPTY, RefCounted.ALWAYS_REFERENCED);

private final BytesReference delegate;
private BytesReference delegate;
private final RefCounted refCounted;

public static ReleasableBytesReference empty() {
Expand Down Expand Up @@ -63,16 +63,24 @@ public boolean tryIncRef() {

@Override
public boolean decRef() {
return refCounted.decRef();
boolean res = refCounted.decRef();
if (res) {
delegate = null;
}
return res;
}

@Override
public boolean hasReferences() {
return refCounted.hasReferences();
boolean hasRef = refCounted.hasReferences();
// delegate is nulled out when the ref-count reaches zero but only via a plain store, and also we could be racing with a concurrent
// decRef so need to check #refCounted again in case we run into a non-null delegate but saw a reference before
assert delegate != null || hasRef == false || refCounted.hasReferences() == false;
Copy link
Contributor

@DaveCTurner DaveCTurner May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could just check refCounted.hasReferences() again, no need to check hasRef right?

Copy link
Contributor Author

@original-brownbear original-brownbear May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got this idea into my head that we might be introducing far more happens-before into everything than necessary via assertions and that as a result reproducing some prod bugs isn't happening. But on reflection, it doesn't make too much sense to do so here, a false hasRef read twice should actually never introduce a happens before, I'll clean it up before merging :)

return hasRef;
}

public ReleasableBytesReference retain() {
refCounted.incRef();
refCounted.mustIncRef();
return this;
}

Expand All @@ -82,6 +90,7 @@ public ReleasableBytesReference retain() {
* retaining unnecessary buffers.
*/
public ReleasableBytesReference retainedSlice(int from, int length) {
assert hasReferences();
if (from == 0 && length() == length) {
return retain();
}
Expand Down Expand Up @@ -136,6 +145,7 @@ public int indexOf(byte marker, int from) {

@Override
public int length() {
assert hasReferences();
return delegate.length();
}

Expand All @@ -154,6 +164,7 @@ public ReleasableBytesReference slice(int from, int length) {

@Override
public long ramBytesUsed() {
assert hasReferences();
return delegate.ramBytesUsed();
}

Expand Down Expand Up @@ -228,6 +239,7 @@ public XContentBuilder toXContent(XContentBuilder builder, Params params) throws

@Override
public boolean isFragment() {
assert hasReferences();
return delegate.isFragment();
}

Expand Down
3 changes: 1 addition & 2 deletions ...a/org/elasticsearch/xpack/remotecluster/RemoteClusterSecurityFcActionAuthorizationIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,6 @@
import static org.elasticsearch.xpack.remotecluster.AbstractRemoteClusterSecurityTestCase.performRequestWithAdminUser;
import static org.hamcrest.Matchers.arrayContaining;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.greaterThanOrEqualTo;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.instanceOf;
Expand Down Expand Up @@ -282,7 +281,7 @@ public void testIndicesPrivilegesAreEnforcedForCcrRestoreSessionActions() throws
GetCcrRestoreFileChunkAction.REMOTE_TYPE,
new GetCcrRestoreFileChunkRequest(response2.getNode(), sessionUUID2, leaderIndex2FileName, 1, shardId2)
);
assertThat(getChunkResponse.getChunk().length(), equalTo(1));
assertFalse(getChunkResponse.getChunk().hasReferences());
Copy link
Contributor Author

@original-brownbear original-brownbear Apr 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh this is pretty much exactly what this is about :P this assertion shouldn't have worked but worked as long as we didn't check for having a reference in the length() call. I see no value in the original version and there's a tiny bit of value in checking that we actually released the reference here but I guess we could just as well drop any assertion on the response.


// Clear restore session fails if index is unauthorized
final var e4 = expectThrows(
Expand Down