1
1
[role="xpack"]
2
2
[[tutorial-migrate-data-stream-from-ilm-to-dsl]]
3
- === Tutorial: Migrate ILM managed data stream to Data stream lifecycle
3
+ === Tutorial: Migrate ILM managed data stream to data stream lifecycle
4
4
5
5
preview::[]
6
6
7
- In this tutorial we'll look at migrating an existing data stream from {ilm-init} to
8
- Data stream lifecycle. The existing {ilm-init} managed backing indices will continue
7
+ In this tutorial we'll look at migrating an existing data stream from Index Lifecycle Management ( {ilm-init}) to
8
+ data stream lifecycle. The existing {ilm-init} managed backing indices will continue
9
9
to be managed by {ilm-init} until they age out and get deleted by {ilm-init}; however,
10
- the new backing indices will be managed by Data stream lifecycle.
11
- This way, a data stream is gradually migrated away from being managed by {ilm-cap } to
12
- being managed by Data stream lifecycle. As we'll see, {ilm-cap } and Data stream lifecycle
10
+ the new backing indices will be managed by data stream lifecycle.
11
+ This way, a data stream is gradually migrated away from being managed by {ilm-init } to
12
+ being managed by data stream lifecycle. As we'll see, {ilm-init } and data stream lifecycle
13
13
can co-manage a data stream; however, an index can only be managed by one system at
14
14
a time.
15
15
16
- Let's first create a data stream with two backing indices managed by {ilm-cap}.
17
- We first create an {ilm-cap} policy:
16
+ [discrete]
17
+ [[migrate-dsl-ilm-tldr]]
18
+ ==== TL;DR
19
+ To migrate a data stream from {ilm-init} to data stream lifecycle we'll have to execute
20
+ two steps:
21
+
22
+ 1. Update the index template that's backing the data stream to set <<index-lifecycle-prefer-ilm, prefer_ilm>>
23
+ to `false`, and to configure data stream lifecycle.
24
+ 2. Configure the data stream lifecycle for the _existing_ data stream using
25
+ the <<data-streams-put-lifecycle, lifecycle API>>.
26
+
27
+ For more details see the <<migrate-from-ilm-to-dsl, migrate to data stream lifecycle>> section.
28
+
29
+ [discrete]
30
+ [[setup-test-data]]
31
+ ==== Setup ILM managed data stream
32
+ Let's first create a data stream with two backing indices managed by {ilm-init}.
33
+ We first create an {ilm-init} policy:
18
34
19
35
[source,console]
20
36
----
@@ -40,7 +56,7 @@ PUT _ilm/policy/pre-dsl-ilm-policy
40
56
}
41
57
----
42
58
43
- And let's create an index template that'll back the data stream and configures {ilm-cap }:
59
+ And let's create an index template that'll back the data stream and configures {ilm-init }:
44
60
45
61
[source,console]
46
62
----
@@ -153,10 +169,13 @@ currently richer in features).
153
169
stream (which will be configured on all the new backing indices, as long as it exists
154
170
in the index template).
155
171
172
+ [discrete]
173
+ [[migrate-from-ilm-to-dsl]]
174
+ ==== Migrate data stream to data stream lifecycle
156
175
To migrate the `dsl-data-stream` to data stream lifecycle we'll have to execute
157
176
two steps:
158
177
159
- 1. Update the index template that's backing the index template to configure <<index-lifecycle-prefer-ilm, prefer_ilm>>
178
+ 1. Update the index template that's backing the data stream to set <<index-lifecycle-prefer-ilm, prefer_ilm>>
160
179
to `false`, and to configure data stream lifecycle.
161
180
2. Configure the data stream lifecycle for the _existing_ `dsl-data-stream` using
162
181
the <<data-streams-put-lifecycle, lifecycle API>>.
@@ -192,13 +211,13 @@ PUT _index_template/dsl-data-stream-template
192
211
193
212
<1> The `prefer_ilm` setting will now be configured on the **new** backing indices
194
213
(created by rolling over the data stream) such that {ilm-init} does _not_ take
195
- precedence over Data stream lifecycle.
214
+ precedence over data stream lifecycle.
196
215
<2> We're configuring the data stream lifecycle so _new_ data streams will be
197
- managed by Data stream lifecycle.
216
+ managed by data stream lifecycle.
198
217
199
- We've now made sure that new data streams will be managed by Data stream lifecycle.
218
+ We've now made sure that new data streams will be managed by data stream lifecycle.
200
219
201
- Let's update our existing `dsl-data-stream` and configure Data stream lifecycle:
220
+ Let's update our existing `dsl-data-stream` and configure data stream lifecycle:
202
221
203
222
[source,console]
204
223
----
@@ -210,7 +229,7 @@ PUT _data_stream/dsl-data-stream/_lifecycle
210
229
// TEST[continued]
211
230
212
231
We can inspect the data stream to check that the next generation will indeed be
213
- managed by Data stream lifecycle:
232
+ managed by data stream lifecycle:
214
233
215
234
[source,console]
216
235
--------------------------------------------------
@@ -274,7 +293,7 @@ GET _data_stream/dsl-data-stream
274
293
and will be configured accordingly for new backing indices.
275
294
276
295
We'll now rollover the data stream to see the new generation index being managed by
277
- Data stream lifecycle:
296
+ data stream lifecycle:
278
297
279
298
[source,console]
280
299
----
@@ -352,14 +371,17 @@ GET _data_stream/dsl-data-stream
352
371
in the index template
353
372
<4> The new write index is managed by `Data stream lifecycle`
354
373
355
- We can easily change this data stream to be managed by {ilm-cap} because we didn't remove
356
- the {ilm-cap} policy when we <<update-index-template-for-dsl, updated
374
+ [discrete]
375
+ [[migrate-from-dsl-to-ilm]]
376
+ ==== Migrate data stream back to ILM
377
+ We can easily change this data stream to be managed by {ilm-init} because we didn't remove
378
+ the {ilm-init} policy when we <<update-index-template-for-dsl, updated
357
379
the index template>>.
358
380
359
381
We can achieve this in two ways:
360
382
361
383
1. <<data-streams-delete-lifecycle, Delete the lifecycle>> from the data streams
362
- 2. Disable Data stream lifecycle by configuring the `enabled` flag to `false`.
384
+ 2. Disable data stream lifecycle by configuring the `enabled` flag to `false`.
363
385
364
386
Let's implement option 2 and disable the data stream lifecycle:
365
387
@@ -439,13 +461,13 @@ GET _data_stream/dsl-data-stream
439
461
// TESTRESPONSE[s/"index_name": ".ds-dsl-data-stream-2023.10.19-000003"/"index_name": $body.data_streams.0.indices.2.index_name/]
440
462
// TESTRESPONSE[s/"index_uuid": "PA_JquKGSiKcAKBA8abcd1"/"index_uuid": $body.data_streams.0.indices.2.index_uuid/]
441
463
// TESTRESPONSE[s/"status": "GREEN"/"status": "YELLOW"/]
442
- <1> The write index is now managed by {ilm-cap }
443
- <2> The `lifecycle` configured on the data stream is now disabled.
444
- <3> The next write index will be managed by {ilm-cap }
464
+ <1> The write index is now managed by {ilm-init }
465
+ <2> The `lifecycle` configured on the data stream is now disabled.
466
+ <3> The next write index will be managed by {ilm-init }
445
467
446
- Had we removed the {ilm-cap } policy from the index template when we <<update-index-template-for-dsl, updated>>
447
- it, the write index of the data stream will now be `Unmanaged` because the index
448
- wouldn't have the {ilm-cap } policy configured to fallback onto.
468
+ Had we removed the {ilm-init } policy from the index template when we <<update-index-template-for-dsl, updated>>
469
+ it, the write index of the data stream will now be `Unmanaged` because the index
470
+ wouldn't have the {ilm-init } policy configured to fallback onto.
449
471
450
472
//////////////////////////
451
473
[source,console]
0 commit comments