Skip to content

Conversation

@parkertimmins
Copy link
Contributor

@parkertimmins parkertimmins commented Feb 13, 2025

When reindexing data stream indices, parts of the index metadata needs to be copied from the source index to destination index, so that ILM and Data stream lifecycle continue to function properly. This adds a new CopyLifecycleIndexMetadataTransportAction which copies the following metadata:

  • creation date
  • rollover info
  • ILM custom metadata

elasticsearchmachine and others added 2 commits February 13, 2025 21:09
- add action to internal user for reindex data stream
- make request an IndicesRequest
- rename to CopyLifecycleIndexMetadataAction
- fix ILM state test
}

public Builder putRolloverInfos(Map<String, RolloverInfo> rolloverInfos) {
this.rolloverInfos.clear();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clearing the existing infos is a bit weirds, but seems better than making rolloverInfos non-final

@parkertimmins parkertimmins changed the title Add action to copy index metadata when reindex data stream indices Add action to copy index metadata when reindexing data stream indices Feb 13, 2025
@parkertimmins parkertimmins added the auto-backport Automatically create backport pull requests when merged label Feb 13, 2025
@parkertimmins parkertimmins marked this pull request as ready for review February 13, 2025 23:00
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Feb 13, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Copy link
Member

@masseyke masseyke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@parkertimmins parkertimmins merged commit aba25c6 into elastic:main Feb 14, 2025
18 checks passed
@parkertimmins parkertimmins deleted the reindex-data-stream-copy-index-metadata branch February 14, 2025 01:14
parkertimmins added a commit to parkertimmins/elasticsearch that referenced this pull request Feb 14, 2025
…elastic#122535)

When reindexing data stream indices, parts of the index metadata needs to be copied from the source index to destination index, so that ILM and data stream lifecycle function properly. This adds a new CopyLifecycleIndexMetadataTransportAction which copies the following metadata from a source index to a destination index:
- creation date setting
- rollover info
- ILM custom metadata
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
9.0
8.18 Commit could not be cherrypicked due to conflicts
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 122535

@parkertimmins
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x
8.18

Questions ?

Please refer to the Backport tool documentation

parkertimmins added a commit to parkertimmins/elasticsearch that referenced this pull request Feb 14, 2025
…elastic#122535)

When reindexing data stream indices, parts of the index metadata needs to be copied from the source index to destination index, so that ILM and data stream lifecycle function properly. This adds a new CopyLifecycleIndexMetadataTransportAction which copies the following metadata from a source index to a destination index:
- creation date setting
- rollover info
- ILM custom metadata

(cherry picked from commit aba25c6)

# Conflicts:
#	x-pack/plugin/migrate/build.gradle
parkertimmins added a commit to parkertimmins/elasticsearch that referenced this pull request Feb 14, 2025
…elastic#122535)

When reindexing data stream indices, parts of the index metadata needs to be copied from the source index to destination index, so that ILM and data stream lifecycle function properly. This adds a new CopyLifecycleIndexMetadataTransportAction which copies the following metadata from a source index to a destination index:
- creation date setting
- rollover info
- ILM custom metadata

(cherry picked from commit aba25c6)

# Conflicts:
#	x-pack/plugin/migrate/build.gradle
elasticsearchmachine pushed a commit that referenced this pull request Feb 14, 2025
…#122535) (#122555)

When reindexing data stream indices, parts of the index metadata needs to be copied from the source index to destination index, so that ILM and data stream lifecycle function properly. This adds a new CopyLifecycleIndexMetadataTransportAction which copies the following metadata from a source index to a destination index:
- creation date setting
- rollover info
- ILM custom metadata
elasticsearchmachine pushed a commit that referenced this pull request Feb 14, 2025
…ndices (#122535) (#122559)

* Add action to copy index metadata when reindexing data stream indices (#122535)

When reindexing data stream indices, parts of the index metadata needs to be copied from the source index to destination index, so that ILM and data stream lifecycle function properly. This adds a new CopyLifecycleIndexMetadataTransportAction which copies the following metadata from a source index to a destination index:
- creation date setting
- rollover info
- ILM custom metadata

(cherry picked from commit aba25c6)

# Conflicts:
#	x-pack/plugin/migrate/build.gradle

* remove timeouts which were not present in 8.x
elasticsearchmachine pushed a commit that referenced this pull request Feb 14, 2025
…indices (#122535) (#122561)

* Add action to copy index metadata when reindexing data stream indices (#122535)

When reindexing data stream indices, parts of the index metadata needs to be copied from the source index to destination index, so that ILM and data stream lifecycle function properly. This adds a new CopyLifecycleIndexMetadataTransportAction which copies the following metadata from a source index to a destination index:
- creation date setting
- rollover info
- ILM custom metadata

(cherry picked from commit aba25c6)

# Conflicts:
#	x-pack/plugin/migrate/build.gradle

* remove timeouts which were not present in 8.18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged backport pending :Data Management/Data streams Data streams and their lifecycles >non-issue Team:Data Management Meta label for data/management team v8.18.0 v8.19.0 v9.0.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants