Skip to content

Conversation

@salvatore-campagna
Copy link
Contributor

@salvatore-campagna salvatore-campagna commented Feb 17, 2025

In PR #97557, we adopted the persistent task framework to execute downsampling tasks. Previously, these tasks were executed as regular, non-persistent tasks. In both cases, the master node was responsible for orchestrating and coordinating task execution. However, starting with version 8.10, the orchestration mechanism changed as part of the transition to the persistent task framework.

With the introduction of persistent tasks, TransportDownsampleAction now uses PersistentTasksService to request the execution of a persistent downsampling task. Instead of assigning the task directly to the node hosting the primary shard and going through TransportDownsampleIndexerAction, the request is now handled by DownsampleShardPersistentTaskExecutor, which takes full ownership of task execution and lifecycle management.

For backward compatibility, we initially retained TransportDownsampleIndexerAction. Depending on the master node's version, one of two orchestration mechanisms was selected:

  • For versions prior to 8.10, a non-persistent task was used.
  • Starting from 8.10, a persistent task was created and managed by a persistent task executor.

Now that newer versions of Elasticsearch no longer need to maintain backward compatibility with 8.10, we can safely remove the class responsible for the old, non-persistent downsampling task execution.

@salvatore-campagna salvatore-campagna marked this pull request as ready for review February 17, 2025 14:52
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@salvatore-campagna salvatore-campagna merged commit 7f7967b into elastic:main Feb 18, 2025
17 checks passed
@lkts
Copy link
Contributor

lkts commented Feb 18, 2025

Does this need to be backported to 9.0?

@martijnvg
Copy link
Member

Does this need to be backported to 9.0?

I don't think it has to be back ported. Maybe we should do this for consistency?

@lkts
Copy link
Contributor

lkts commented Feb 18, 2025

Yes, i meant just for consistency.

@salvatore-campagna
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
9.0

Questions ?

Please refer to the Backport tool documentation

salvatore-campagna added a commit to salvatore-campagna/elasticsearch that referenced this pull request Feb 20, 2025
(cherry picked from commit 7f7967b)

# Conflicts:
#	x-pack/plugin/downsample/src/main/java/org/elasticsearch/xpack/downsample/TransportDownsampleIndexerAction.java
salvatore-campagna added a commit that referenced this pull request Feb 20, 2025
(cherry picked from commit 7f7967b)

# Conflicts:
#	x-pack/plugin/downsample/src/main/java/org/elasticsearch/xpack/downsample/TransportDownsampleIndexerAction.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants