You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: manage-data/lifecycle/data-stream.md
+5-4Lines changed: 5 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -48,11 +48,12 @@ Steps `2-4` apply only to backing indices that are not already managed by {{ilm-
48
48
49
49
Since the lifecycle is configured on the data stream level, the process to configure a lifecycle on a new data stream and on an existing one differ.
50
50
51
-
In the following sections, we will go through the following tutorials:
51
+
Four tutorials are available to help you set up and manage data streams with data stream lifecycle:
52
52
53
-
* To create a new data stream with a lifecycle, you need to add the data stream lifecycle as part of the index template that matches the name of your data stream (see [Tutorial: Create a data stream with a lifecycle](data-stream/tutorial-create-data-stream-with-lifecycle.md)). When a write operation with the name of your data stream reaches {{es}} then the data stream will be created with the respective data stream lifecycle.
54
-
* To update the lifecycle of an existing data stream you need to use the [data stream lifecycle APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-data-stream) to edit the lifecycle on the data stream itself (see [Tutorial: Update existing data stream](data-stream/tutorial-update-existing-data-stream.md)).
55
-
* Migrate an existing {{ilm-init}} managed data stream to Data stream lifecycle using [Tutorial: Migrate ILM managed data stream to data stream lifecycle](data-stream/tutorial-migrate-ilm-managed-data-stream-to-data-stream-lifecycle.md).
53
+
* To create a new data stream with a lifecycle, add the data stream lifecycle as part of the index template that matches the name of your data stream. See [Tutorial: Create a data stream with a lifecycle](data-stream/tutorial-create-data-stream-with-lifecycle.md) for the detailed steps. When a write operation with the name of your data stream reaches {{es}} then the data stream will be created with the respective data stream lifecycle.
54
+
* To update the lifecycle settings for an individual, existing data stream, use the [data stream lifecycle APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-data-stream). See [Tutorial: Update existing data stream](data-stream/tutorial-update-existing-data-stream.md) for details.
55
+
* Retention settings for data streams can be configured both individually, at the data stream level, and globally, for all data streams in a cluster. To learn more, refer to [Tutorial: Configure data stream retention](/manage-data/lifecycle/data-stream/tutorial-data-stream-retention.md).
56
+
* To migrate an existing {{ilm-init}} managed data stream to data stream lifecycle, follow the steps in [Tutorial: Migrate ILM managed data stream to data stream lifecycle](data-stream/tutorial-migrate-ilm-managed-data-stream-to-data-stream-lifecycle.md).
56
57
57
58
::::{note}
58
59
Updating the data stream lifecycle of an existing data stream is different from updating the settings or the mapping, because it is applied on the data stream level and not on the individual backing indices.
Copy file name to clipboardExpand all lines: manage-data/lifecycle/data-stream/tutorial-create-data-stream-with-lifecycle.md
+13-9Lines changed: 13 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,7 +30,7 @@ You can use the [create index template API](https://www.elastic.co/docs/api/doc/
30
30
```console
31
31
PUT _index_template/my-index-template
32
32
{
33
-
"index_patterns": ["my-data-stream*"],
33
+
"index_patterns": ["my-data-stream-test"], <1>
34
34
"data_stream": { },
35
35
"priority": 500,
36
36
"template": {
@@ -43,7 +43,7 @@ PUT _index_template/my-index-template
43
43
}
44
44
}
45
45
```
46
-
46
+
1. In this case the index template will be applied to a data stream named `my-data-stream-test`. You can optionally use a wildcard (`*`) in the index pattern to match all data streams created (either manually or using an indexing request) that have a name matching the specified pattern.
47
47
48
48
## Create a data stream [create-data-stream-with-lifecycle]
49
49
@@ -52,13 +52,13 @@ You can create a data stream in two ways:
52
52
1. By manually creating the stream using the [create data stream API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create-data-stream). The stream’s name must still match one of your template’s index patterns.
53
53
54
54
```console
55
-
PUT _data_stream/my-data-stream
55
+
PUT _data_stream/my-data-stream-test
56
56
```
57
57
58
58
2. By [indexing requests](../../data-store/data-streams/use-data-stream.md#add-documents-to-a-data-stream) that target the stream’s name. This name must match one of your index template’s index patterns.
@@ -72,7 +72,7 @@ You can create a data stream in two ways:
72
72
You can use the [get data stream lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-data-lifecycle) to see the data stream lifecycle of your data stream and the [explain data stream lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle) to see the exact state of each backing index.
73
73
74
74
```console
75
-
GET _data_stream/my-data-stream/_lifecycle
75
+
GET _data_stream/my-data-stream-test/_lifecycle
76
76
```
77
77
78
78
The result will look like this:
@@ -81,7 +81,7 @@ The result will look like this:
81
81
{
82
82
"data_streams": [
83
83
{
84
-
"name": "my-data-stream", <1>
84
+
"name": "my-data-stream-test", <1>
85
85
"lifecycle": {
86
86
"enabled": true, <2>
87
87
"data_retention": "7d", <3>
@@ -103,16 +103,20 @@ The result will look like this:
103
103
If you want to see more information about how the data stream lifecycle is applied on individual backing indices use the [explain data stream lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle):
104
104
105
105
```console
106
-
GET .ds-my-data-stream-*/_lifecycle/explain
106
+
GET .ds-my-data-stream-test/_lifecycle/explain
107
107
```
108
108
109
+
:::{tip}
110
+
You can use a wildcard (`*`) in the data stream name to retrieve the lifecycle status for all data streams matching the pattern.
Copy file name to clipboardExpand all lines: manage-data/lifecycle/data-stream/tutorial-data-stream-retention.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ products:
8
8
- id: elasticsearch
9
9
---
10
10
11
-
# Tutorial: Data stream retention [tutorial-manage-data-stream-retention]
11
+
# Tutorial: Configure data stream retention [tutorial-manage-data-stream-retention]
12
12
13
13
In this tutorial, we are going to go over the data stream lifecycle retention; we will define it, go over how it can be configured and how it can gets applied. Keep in mind, the following options apply only to data streams that are managed by the data stream lifecycle.
Copy file name to clipboardExpand all lines: manage-data/lifecycle/data-stream/tutorial-migrate-ilm-managed-data-stream-to-data-stream-lifecycle.md
+1-3Lines changed: 1 addition & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -312,8 +312,6 @@ GET _data_stream/dsl-data-stream
312
312
3. The new write index received the `false` value for the `prefer_ilm` setting, as we configured in the index template
313
313
4. The new write index is managed by `Data stream lifecycle`
314
314
315
-
316
-
317
315
## Migrate data stream back to ILM [migrate-from-dsl-to-ilm]
318
316
319
317
We can easily change this data stream to be managed by {{ilm-init}} because we didn’t remove the {{ilm-init}} policy when we [updated the index template](#update-index-template-for-dsl).
@@ -333,7 +331,7 @@ PUT _data_stream/dsl-data-stream/_lifecycle
333
331
}
334
332
```
335
333
336
-
1. The `enabled` flag can be ommitted and defaults to `true` however, here we explicitly configure it to `false` Let’s check the state of the data stream:
334
+
1. The `enabled` flag can be omitted and defaults to `true` however, here we explicitly configure it to `false` Let’s check the state of the data stream:
Copy file name to clipboardExpand all lines: manage-data/lifecycle/data-stream/tutorial-update-existing-data-stream.md
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,13 +8,14 @@ products:
8
8
- id: elasticsearch
9
9
---
10
10
11
-
# Tutorial: Update existing data stream [tutorial-manage-existing-data-stream]
11
+
# Tutorial: Update the lifecycle of a data stream [tutorial-manage-existing-data-stream]
12
12
13
-
To update the lifecycle of an existing data stream, perform the following actions:
13
+
Follow these steps to configure or remove data stream lifecycle settings for an existing, individual data stream:
14
14
15
-
1.[Set a data stream’s lifecycle](#set-lifecycle)
16
-
2.[Remove lifecycle for a data stream](#delete-lifecycle)
15
+
-[Set a data stream’s lifecycle](#set-lifecycle)
16
+
-[Remove the lifecycle for a data stream](#delete-lifecycle)
17
17
18
+
Note that these steps are for data stream lifecycle only. For the steps to configure {{ilm}}, refer to the [{{ilm-init}} documentation](/manage-data/lifecycle/index-lifecycle-management.md). For a comparison between the two, refer to [](/manage-data/lifecycle.md).
18
19
19
20
## Set a data stream’s lifecycle [set-lifecycle]
20
21
@@ -143,7 +144,7 @@ The response will look like:
143
144
:::
144
145
:::::
145
146
146
-
## Remove lifecycle for a data stream [delete-lifecycle]
147
+
## Remove the lifecycle for a data stream [delete-lifecycle]
147
148
148
149
To remove the lifecycle of a data stream you can use the **Index Management** tools in {{kib}} or the {{es}} [delete lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete-data-lifecycle).
0 commit comments