You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
navigation_title: {{ech}} data to {{serverless-full}} with {{ls}}
3
3
applies_to:
4
4
stack: ga
5
5
deployment:
@@ -13,26 +13,19 @@ products:
13
13
- id: cloud-kubernetes
14
14
---
15
15
16
-
# Migrate data with {{ls}} [migrate-with-ls]
16
+
# Migrate {{ech}} data to {{serverless-full}} with {{ls}} [migrate-with-ls]
17
17
18
18
You can use {{ls}} to migrate data from an {{ech}} deployment to an {{serverless-full}} project.
19
-
20
-
The Logstash input has cursor like-functionality that keeps your place.
21
-
22
-
[Tracking a field's value across runs](https://www.elastic.co/docs/reference/logstash/plugins/plugins-inputs-elasticsearch)
23
-
24
19
Familiarity with {{ech}}, {{es}}, and {{ls}} is helpful, but not required.
25
20
26
-
:::{note}
27
-
This guide focuses on data migration. Dashboards, visualizations, pipelines, templates, and other {{kib}} assets must be migrated separately using the {{kib}} [export/import APIs](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-saved-objects) or recreated manually.
21
+
:::{admonition} Basic migration
22
+
This guide focuses on a basic data migration scenario for moving static data from an {{ech}} deployment to a {{serverless-full}} project. Dashboards, visualizations, pipelines, templates, and other {{kib}} assets must be migrated separately using the {{kib}} [export/import APIs](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-saved-objects) or recreated manually.
28
23
:::
29
24
30
25
## Prerequisites [migrate-prereqs]
31
26
32
-
Ensure that you have:
33
-
34
27
- {{ech}} deployment with data to migrate
35
-
- {{serverless-full}} project configured and running
28
+
-[{{serverless-full}}](/deploy-manage/deploy/elastic-cloud/serverless.md) project configured and running
36
29
- {{ls}} [installed](https://www.elastic.co/downloads/logstash) on your local machine or server
37
30
- API keys in {{ls}} format for authentication with both deployments
38
31
@@ -43,9 +36,8 @@ Ensure that you have:
43
36
*[Verify data migration](#verify-migration)
44
37
45
38
46
-
47
39
## Step 1: Configure {{ls}} [configure-ls]
48
-
Create a new {{ls}} configuration file (migration.conf) with these settings:
40
+
Create a new {{ls}} [pipeline configuration file](logstash://reference/creating-logstash-pipeline.md) (_migration.conf_) with these settings:
49
41
50
42
```
51
43
input {
@@ -62,15 +54,20 @@ output {
62
54
hosts => [ "https://<SERVERLESS_HOST_URL>:443" ] # URL for your Serverless project URL, set port as 443
63
55
api_key => "<SERVERLESS_API_KEY>" # API key (in Logstash format) for your Serverless project
64
56
ssl_enabled => true
65
-
index => "%{[@metadata][input][elasticsearch][_index]}" # Retain original index names
57
+
index => "%{[@metadata][input][elasticsearch][_index]}" # Instruction to retain original index names
When you create an API key for Logstash, be sure to select **Logstash** from the **API key** format dropdown. This option formats the API key in the correct `id:api_key` format required by Logstash.
64
+
:::{admonition} Tips
73
65
66
+
- When you create an [API key for {{ls}}](logstash://reference/connecting-to-serverless.md#api-key), be sure to select **Logstash** from the **API key** format dropdown. This option formats the API key in the correct `id:api_key` format required by {{ls}}.
67
+
68
+
- To migrate multiple indexes at the same time, use a wildcard in the index name.
69
+
For example, `index => "logs-*"` migrates all indices starting with `logs-`.
70
+
:::
74
71
75
72
## Step 2: Run {{ls}} [run-ls]
76
73
@@ -88,27 +85,11 @@ After running {{ls}}, verify that the data has been successfully migrated:
88
85
2. Navigate to Index Management and select the index.
89
86
3. Verify that the migrated data is visible.
90
87
88
+
<br>
89
+
:::{admonition} Advanced migration
90
+
:applies_to: stack: preview
91
91
92
+
{{ls}} can handle more advanced migrations with field tracking settings in the elasticsearch input. The field tracking feature adds "cursor-like" functionality that can support more complex migrations and ongoing data migration over time.
92
93
93
-
94
-
95
-
:::{tip}
96
-
To migrate multiple indexes at the same time, use a wildcard in the index name.
97
-
For example, `index => "logs-*"` migrates all indices starting with `logs-`.
94
+
More information is available in the Elasticsearch input plugin documentation: [Tracking a field's value across runs](https://www.elastic.co/docs/reference/logstash/plugins/plugins-inputs-elasticsearch#plugins-inputs-elasticsearch-cursor).
0 commit comments