You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
navigation_title: Migrate Elasticsearch data with minimal downtime
2
+
navigation_title: Migrate with minimal downtime
3
3
applies_to:
4
4
stack: ga
5
-
deployment:
6
-
ess: ga
7
-
ece: ga
8
-
eck: ga
9
5
products:
10
6
- id: elasticsearch
11
7
- id: cloud-hosted
@@ -14,8 +10,7 @@ products:
14
10
---
15
11
16
12
# Migrate {{es}} data with minimal downtime [migrate-elasticsearch-data-with-minimal-downtime]
17
-
18
-
When moving your data and services from one {{es}} cluster to another, such as to {{ech}}, {{ece}}, new on-premises hardware, or any other {{es}} environment, you can use incremental snapshots to minimize downtime.
13
+
When moving your data and services from one {{es}} cluster to another, such as to {{ech}}, {{ece}}, new on-premises hardware, or any other {{es}} environment, you can migrate with minimal downtime using incremental snapshots.
19
14
20
15
Migrating with incremental snapshots is useful when you want to:
21
16
@@ -24,61 +19,52 @@ Migrating with incremental snapshots is useful when you want to:
24
19
* Maintain data consistency and minimize disruption.
25
20
26
21
## How incremental snapshots work [how-incremental-snapshots-work]
27
-
28
22
Incremental snapshots capture only the data that has changed since the previous snapshot.
29
23
30
24
After the initial full snapshot, each subsequent snapshot contains only the differences, which makes the snapshot process faster over time. When you restore snapshots, only the missing data segments are copied from the snapshot repository to the cluster local storage, speeding up restores when the changes between snapshots are small.
31
25
32
26
When you incrementally create and restore snapshots, you can repeatedly synchronize the new cluster with the old cluster by taking and restoring multiple snapshots before performing the final cutover. This mechanism enables you to copy the vast majority of your data into the new cluster ahead of time, making the future transition from the old cluster to your new one as quick as possible, since only few segments will need to go through the snapshot-and-restore procedure upon cutover.
While incremental snapshots allow efficient migration with minimal downtime, consider the limitations when planning your migration.
30
+
31
+
Limitations include the following:
32
+
***Storage requirements** – Sufficient repository storage is required, and usage can grow based on snapshot frequency and data volume.
33
+
***Network overhead** – Transferring snapshots across networks, regions, or providers can be time-consuming and incur costs.
34
+
***Version compatibility** – Old and new clusters must use compatible {{es}} versions. To check if your cluster versions are compatible, check [Snapshot version compatibility](/deploy-manage/tools/snapshot-and-restore.md#snapshot-restore-version-compatibility).
35
+
***Custom integrations** – Some custom integrations that directly use the {{es}} API, such as the [Elasticsearch Java Client library](elasticsearch-java://reference/index.md), can require additional handling during the cutover from the old cluster to the new cluster.
36
+
***Resource usage** – Initial and incremental snapshot and restore operations can be resource-intensive, potentially affecting cluster performance.
35
37
36
-
Complete the minimal-downtime migration using incremental snapshots. While the exact sequence may differ depending on your infrastructure and operational requirements, you can use the recommended migration timeline as a reliable baseline that you can adapt. Adjust the steps and times to fit your own operational needs.
Tp complete the migration with minimal downtime, use incremental snapshots. While the exact sequence may differ depending on your infrastructure and operational requirements, you can use the recommended migration timeline as a reliable baseline that you can adapt. Adjust the steps and times to fit your own operational needs.
37
40
38
41
1.**09:00**: Take the initial full snapshot of the old cluster. You can also take the initial full snapshot the day before.
39
42
2.**09:30**: Restore the snapshot to the new cluster.
40
43
3.**09:55**: Take another snapshot of the old cluster and restore it to the new cluster. Repeat this process until the snapshot and restore operations take only a few seconds or minutes.
41
-
4.**10:15**: Perform the final cutover:
44
+
4.**10:15**: Perform the final cutover.
42
45
1. In the old cluster, pause indexing or set indices to read-only.
43
46
2. Take a final snapshot.
44
47
3. Restore the snapshot to the new cluster.
45
48
4. Change ingestion and querying to the new cluster.
While incremental snapshots allow efficient migration with minimal downtime, consider the limitations when planning your migration.
51
-
52
-
Limitations include the following:
53
-
***Storage requirements** – Sufficient repository storage is required, and usage can grow based on snapshot frequency and data volume.
54
-
***Network overhead** – Transferring snapshots across networks, regions, or providers can be time-consuming and incur costs.
55
-
***Version compatibility** – Old and new clusters must use compatible {{es}} versions. To check if your cluster versions are compatible, check [Snapshot version compatibility](/deploy-manage/tools/snapshot-and-restore.md#snapshot-restore-version-compatibility).
56
-
***Custom integrations** – Some custom integrations that directly use the {{es}} API can require additional handling during the cutover from the old cluster to the new cluster.
57
-
***Resource usage** – Initial and incremental snapshot and restore operations can be resource-intensive, potentially affecting cluster performance.
## Related resources [related-incremental-snapshot-resources]
61
52
For more information on migrating {{es}} data with minimal downtime using incremental snapshots, review the related resources.
62
53
63
54
### Snapshot and restore
64
-
65
55
* For more information about snapshot and restore concepts, check [Snapshot and restore](/deploy-manage/tools/snapshot-and-restore.md).
66
56
* To learn how to configure snapshot repositories before taking or restoring snapshots, check [Manage snapshot repositories](/deploy-manage/tools/snapshot-and-restore/manage-snapshot-repositories.md).
67
57
* To learn how to restore snapshots to clusters other than the source, check [Restore to a different cluster](/deploy-manage/tools/snapshot-and-restore/restore-snapshot.md#restore-different-cluster).
68
58
69
59
### Cluster and index management
70
-
71
60
* For details on setting indices to read-only to safely pause indexing during migration, check [Index lifecycle actions: Read-only](elasticsearch://reference/elasticsearch/index-lifecycle-actions/ilm-readonly.md).
72
61
73
62
### Data ingestion
74
-
75
63
* For information about using {{ls}} for data ingestion, check the [{{ls}} documentation](logstash://reference/index.md).
76
64
* For information about using {{beats}} for data ingestion, check the [{{beats}} documentation](beats://reference/index.md).
77
65
78
66
### Alternative migration methods
79
-
80
67
* To learn about reindexing from remote clusters as an alternative migration method, check [Reindex from a remote cluster](/manage-data/migrate.md#ech-reindex-remote).
81
68
82
69
### Additional support
83
-
84
70
* To get expert assistance for your {{es}} migrations, go to [Elastic Professional Services](https://www.elastic.co/consulting).
0 commit comments