Skip to content

Document Rest*Action and Transport*Action threading model #135043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Oct 2, 2025
Merged

Document Rest*Action and Transport*Action threading model #135043

Changes from 6 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e659a25
Documentation : Rest and Transport Action Threading Model
zhubotang-wq Sep 18, 2025
f1c7de0
Documentation : Rest and Transport Action Threading Model
zhubotang-wq Sep 18, 2025
f6c71ed
Documentation : Transport action type strings.
zhubotang-wq Sep 18, 2025
8310030
Documentation : Transport action type strings.
zhubotang-wq Sep 19, 2025
e0004ba
Documentation : Transport action type strings.
zhubotang-wq Sep 22, 2025
21f95b1
Documentation : Transport action type strings.
zhubotang-wq Sep 22, 2025
dd2ae71
Merge remote-tracking branch 'upstream/main' into fix/document_genera…
zhubotang-wq Sep 29, 2025
3c91c6f
Update docs/internal/GeneralArchitectureGuide.md
zhubotang-wq Sep 29, 2025
31c25a1
Update docs/internal/GeneralArchitectureGuide.md
zhubotang-wq Sep 29, 2025
b833f7e
Update docs/internal/GeneralArchitectureGuide.md
zhubotang-wq Sep 29, 2025
1a45418
Update docs/internal/GeneralArchitectureGuide.md
zhubotang-wq Sep 29, 2025
660570c
Documentation : Rest and Transport Action Threading Model
zhubotang-wq Sep 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions docs/internal/GeneralArchitectureGuide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,24 @@ are coordinated.
> does not hold, in those cases you can locate the transport action for a REST action by looking at the `NodeClient` invocation in the
> `Rest*Action`'s `prepareRequest` implementation, it should specify the `ActionType` being invoked which can then be used to locate
> the `Transport*Action` class that handles it.
>
> A netty [EventLoop] thread handles the initial steps of a Rest*Action request lifecycle such as decoding, validation and routing.
> Upon entry into the "transport layer", [NodeClient] delegates the decision of execution to individual [TransportAction]. Each action
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I prefer not to make every mention of a class name a code link. Maybe use a link the first time a class is mentioned, if you think it's helpful to go look at some particular piece of code for additional information.

When I see a link, I think I should click on it, and then I'm disappointed when it's redundant and I have too many tabs open (the last part is my own problem I suppose 😌).

Instead of highlighting text with a hyperlink, you can use back ticks (``) to emphasize that it's a code name. I'd replace all your new text's [TransportAction] with TransportAction, since there's already a link reference above the new text. Same for [ActionType]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Dialed down the use of code links more sparingly as to not distract readers.

> determines whether to execute synchronously on invoking thread-an approach reserved for lightweight processing-or to dispatch execution to an appropriate
> thread pool, which is recommended practice for heavy workloads. Comprehensive mechanisms are available for propagating [ThreadContext] when tasks
> are dispatched to alternate thread pools.
>
> [TransportAction] can also be initiated through peer-to-peer communication between nodes. In such cases, the [InboundHandler]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might also be worth mentioning about how the responses to these remote actions are threaded, using the executor defined in the response handler, see org.elasticsearch.transport.TransportResponseHandler#executor().

Also note that requests received remotely are always deserialized on the Netty event loop, but responses are deserialized after dispatching to the relevant executor. Outbound messages (both requests and responses) are serialized on the calling thread.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Added referenced to TransportResponseHandler and missing details on division of workload between IO and logic layer thread pools.

> locates the appropriate [TransportAction] by consulting the [NamedRegistry], then invokes its handleExecution() method. When a [TransportAction]
> is registered, it can specify an executor to control how the action is run. One option is the DIRECT_EXECUTOR_SERVICE, which executes the
> action on the calling thread. However, this should be used with caution—it's only appropriate when the action is lightweight.
> Otherwise, it risks blocking the peer-to-peer I/O thread, potentially degrading responsiveness and causing the node to become unresponsive.
> If [TransportAction] elects a separate executor, [InboundHandler] performs message deserialization before delegating execution to the executor,
> refer to [TransportResponseHandler] method executor() for more details.
>
> [TransportAction] requests received from remote nodes are always deserialized on the Netty [EventLoop]. In contrast, [TransportAction] responses are
> deserialized only after being dispatched to the designated executor. Outbound messages, including both requests and responses, are serialized
> synchronously on the calling thread.

### Action registration
Elasticsearch contains many [TransportAction]s, configured statically in [ActionModule#setupActions]. [ActionPlugin]s can
Expand All @@ -115,6 +133,17 @@ The actions themselves sometimes dispatch downstream actions to other nodes in t
[TransportService#registerRequestHandler]. [HandledTransportAction] is a common parent class that registers an action with the
`TransportService`.

> [!NOTE]
> TransportActions' [ActionType] naming conventions encode semantic information about the role, scope, plugins, modules and behaviours.
> [ActionType] instances are mapped to permission privileges via the [ClusterPrivilegeResolver]. Security interceptors enforce access
> control by invoking RBACEngine.checkPrivileges().
> Indices-level [ActionType] strings generally follows the pattern: `indices:[data|admin|monitor]/[read|write|get]/[index|bulk|update]`.
> Cluster-level [ActionType] strings are prefixed by `cluster:` are often followed by a domain-specific such as `autoscaling`, `logstash`, `ingest`,
> `xpack`.
> - `internal:` is meant to executed by `_system` user.
> - `cluster:internal/xpack/..` [ActionType] strings are plugin specific.
> - `internal:transport/proxy/indices:` [ActionType] strings are automatically wrapped actions when requests for CCR/CCS in proxy mode.

> [!NOTE]
> The name [TransportAction] can be misleading, as it suggests they are all invoke-able and invoked via the TCP transport. In fact,
> a majority of transport actions are only ever invoked locally via the [NodeClient]. The two key features of a `TransportAction` are:
Expand Down Expand Up @@ -179,6 +208,13 @@ capabilities.
[TransportService]:https://github.com/elastic/elasticsearch/blob/v9.0.1/server/src/main/java/org/elasticsearch/transport/TransportService.java
[TransportSingleShardAction]:https://github.com/elastic/elasticsearch/blob/v9.0.1/server/src/main/java/org/elasticsearch/action/support/single/shard/TransportSingleShardAction.java
[Transport]:https://github.com/elastic/elasticsearch/blob/v9.0.1/server/src/main/java/org/elasticsearch/transport/Transport.java
[EventLoop]:https://www.elastic.co/docs/reference/elasticsearch/configuration-reference/networking-settings#modules-network-threading-model
[TaskManager]:https://github.com/elastic/elasticsearch/blob/main/server/src/main/java/org/elasticsearch/tasks/TaskManager.java
[InboundHandler]:https://github.com/elastic/elasticsearch/blob/main/server/src/main/java/org/elasticsearch/transport/InboundHandler.java
[NamedRegistry]:https://github.com/elastic/elasticsearch/blob/main/server/src/main/java/org/elasticsearch/common/NamedRegistry.java
[IndexPrivilege]:https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authz/privilege/IndexPrivilege.java
[ClusterPrivilegeResolver]:https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authz/privilege/ClusterPrivilegeResolver.java
[TransportResponseHandler]:https://github.com/elastic/elasticsearch/blob/main/server/src/main/java/org/elasticsearch/transport/TransportResponseHandler.java

## Serializations

Expand Down