Skip to content

Conversation

jdconrad
Copy link
Contributor

CloseableByteBuffer currently only uses confined Arena for buffer allocation meaning only the thread creating the Arena is allowed to access the native memory. When using elasticsearch with systemd, our notify-extend message is executed on a thread separate from where the native memory was allocated. This is causing a RuntimeException to be thrown. This changes CloseableByteBuffer to allow for a shared Arena as well for systemd.

@jdconrad jdconrad requested a review from a team as a code owner September 22, 2025 21:35
@jdconrad jdconrad added >bug Team:Core/Infra Meta label for core/infra team :Core/Infra/Node Lifecycle Node startup, bootstrapping, and shutdown auto-backport Automatically create backport pull requests when merged v9.2.0 v8.19.5 v9.1.5 v8.18.8 v9.0.8 labels Sep 22, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine
Copy link
Collaborator

Hi @jdconrad, I've created a changelog YAML for you.


public non-sealed interface JavaLibrary extends NativeLibrary {
CloseableByteBuffer newBuffer(int len);
CloseableByteBuffer newBuffer(int len, boolean shared);
Copy link
Member

@rjernst rjernst Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of passing in a boolean, could we separate these two cases into two different methods? That is, have newSharedBuffer and newConfinedBuffer? I think this better matches the model of the jdk with ofShared() and ofConfined. The internal implementation can still use a flag, but the api surface area would be clearly distinct and easily understandable without remember what that flag means.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this.

Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@ldematte ldematte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++
I really like the new newSharedBuffer/newConfinedBuffer and ofShared/ofConfined.

@jdconrad jdconrad merged commit b34d068 into elastic:main Sep 23, 2025
35 checks passed
jdconrad added a commit to jdconrad/elasticsearch that referenced this pull request Sep 23, 2025
CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.
jdconrad added a commit to jdconrad/elasticsearch that referenced this pull request Sep 23, 2025
CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.
jdconrad added a commit to jdconrad/elasticsearch that referenced this pull request Sep 23, 2025
CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.
jdconrad added a commit to jdconrad/elasticsearch that referenced this pull request Sep 23, 2025
CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.19
9.1
8.18
9.0

elasticsearchmachine pushed a commit that referenced this pull request Sep 23, 2025
CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.
elasticsearchmachine pushed a commit that referenced this pull request Sep 23, 2025
CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.
szybia added a commit to szybia/elasticsearch that referenced this pull request Sep 23, 2025
…-dls

* upstream/main:
  Bump FLEET_AGENTS_MAPPINGS_VERSION so the new mapping applies on upgrades (elastic#134957)
  [ML] Adding custom headers support openai text embeddings (elastic#134960)
  Fix systemd notify to use a shared arena (elastic#135235)
elasticsearchmachine pushed a commit that referenced this pull request Sep 23, 2025
* Fix systemd notify to use a shared arena (#135235)

CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.

* fix backport
elasticsearchmachine pushed a commit that referenced this pull request Sep 23, 2025
* Fix systemd notify to use a shared arena (#135235)

CloseableByteBuffer currently only uses confined Arena 
for buffer allocation meaning only the thread creating 
the Arena is allowed to access the native memory. When 
using elasticsearch with systemd, our notify-extend 
message is executed on a thread separate from where 
the native memory was allocated. This is causing a 
RuntimeException to be thrown. This changes 
CloseableByteBuffer to allow for a shared Arena as 
well for systemd.

* fix backport
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged >bug :Core/Infra/Node Lifecycle Node startup, bootstrapping, and shutdown Team:Core/Infra Meta label for core/infra team v8.18.8 v8.19.5 v9.0.8 v9.1.5 v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants