Skip to content

Commit 015b6a5

Browse files
Fix up data stream lifecycle tutorials (#2884)
A few updates to the DLM tutorials, corresponding to Edu's feedback in elastic/docs-content-internal#190 --------- Co-authored-by: Vlada Chirmicci <[email protected]>
1 parent 662b7e0 commit 015b6a5

File tree

5 files changed

+26
-22
lines changed

5 files changed

+26
-22
lines changed

manage-data/lifecycle/data-stream.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,12 @@ Steps `2-4` apply only to backing indices that are not already managed by {{ilm-
4848

4949
Since the lifecycle is configured on the data stream level, the process to configure a lifecycle on a new data stream and on an existing one differ.
5050

51-
In the following sections, we will go through the following tutorials:
51+
Four tutorials are available to help you set up and manage data streams with data stream lifecycle:
5252

53-
* To create a new data stream with a lifecycle, you need to add the data stream lifecycle as part of the index template that matches the name of your data stream (see [Tutorial: Create a data stream with a lifecycle](data-stream/tutorial-create-data-stream-with-lifecycle.md)). When a write operation with the name of your data stream reaches {{es}} then the data stream will be created with the respective data stream lifecycle.
54-
* To update the lifecycle of an existing data stream you need to use the [data stream lifecycle APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-data-stream) to edit the lifecycle on the data stream itself (see [Tutorial: Update existing data stream](data-stream/tutorial-update-existing-data-stream.md)).
55-
* Migrate an existing {{ilm-init}} managed data stream to Data stream lifecycle using [Tutorial: Migrate ILM managed data stream to data stream lifecycle](data-stream/tutorial-migrate-ilm-managed-data-stream-to-data-stream-lifecycle.md).
53+
* To create a new data stream with a lifecycle, add the data stream lifecycle as part of the index template that matches the name of your data stream. See [Tutorial: Create a data stream with a lifecycle](data-stream/tutorial-create-data-stream-with-lifecycle.md) for the detailed steps. When a write operation with the name of your data stream reaches {{es}} then the data stream will be created with the respective data stream lifecycle.
54+
* To update the lifecycle settings for an individual, existing data stream, use the [data stream lifecycle APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-data-stream). See [Tutorial: Update existing data stream](data-stream/tutorial-update-existing-data-stream.md) for details.
55+
* Retention settings for data streams can be configured both individually, at the data stream level, and globally, for all data streams in a cluster. To learn more, refer to [Tutorial: Configure data stream retention](/manage-data/lifecycle/data-stream/tutorial-data-stream-retention.md).
56+
* To migrate an existing {{ilm-init}} managed data stream to data stream lifecycle, follow the steps in [Tutorial: Migrate ILM managed data stream to data stream lifecycle](data-stream/tutorial-migrate-ilm-managed-data-stream-to-data-stream-lifecycle.md).
5657

5758
::::{note}
5859
Updating the data stream lifecycle of an existing data stream is different from updating the settings or the mapping, because it is applied on the data stream level and not on the individual backing indices.

manage-data/lifecycle/data-stream/tutorial-create-data-stream-with-lifecycle.md

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ You can use the [create index template API](https://www.elastic.co/docs/api/doc/
3030
```console
3131
PUT _index_template/my-index-template
3232
{
33-
"index_patterns": ["my-data-stream*"],
33+
"index_patterns": ["my-data-stream-test"], <1>
3434
"data_stream": { },
3535
"priority": 500,
3636
"template": {
@@ -43,7 +43,7 @@ PUT _index_template/my-index-template
4343
}
4444
}
4545
```
46-
46+
1. In this case the index template will be applied to a data stream named `my-data-stream-test`. You can optionally use a wildcard (`*`) in the index pattern to match all data streams created (either manually or using an indexing request) that have a name matching the specified pattern.
4747

4848
## Create a data stream [create-data-stream-with-lifecycle]
4949

@@ -52,13 +52,13 @@ You can create a data stream in two ways:
5252
1. By manually creating the stream using the [create data stream API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create-data-stream). The stream’s name must still match one of your template’s index patterns.
5353

5454
```console
55-
PUT _data_stream/my-data-stream
55+
PUT _data_stream/my-data-stream-test
5656
```
5757

5858
2. By [indexing requests](../../data-store/data-streams/use-data-stream.md#add-documents-to-a-data-stream) that target the stream’s name. This name must match one of your index template’s index patterns.
5959

6060
```console
61-
PUT my-data-stream/_bulk
61+
PUT my-data-stream-test/_bulk
6262
{ "create":{ } }
6363
{ "@timestamp": "2099-05-06T16:21:15.000Z", "message": "192.0.2.42 - - [06/May/2099:16:21:15 +0000] \"GET /images/bg.jpg HTTP/1.0\" 200 24736" }
6464
{ "create":{ } }
@@ -72,7 +72,7 @@ You can create a data stream in two ways:
7272
You can use the [get data stream lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-data-lifecycle) to see the data stream lifecycle of your data stream and the [explain data stream lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle) to see the exact state of each backing index.
7373

7474
```console
75-
GET _data_stream/my-data-stream/_lifecycle
75+
GET _data_stream/my-data-stream-test/_lifecycle
7676
```
7777

7878
The result will look like this:
@@ -81,7 +81,7 @@ The result will look like this:
8181
{
8282
"data_streams": [
8383
{
84-
"name": "my-data-stream", <1>
84+
"name": "my-data-stream-test", <1>
8585
"lifecycle": {
8686
"enabled": true, <2>
8787
"data_retention": "7d", <3>
@@ -103,16 +103,20 @@ The result will look like this:
103103
If you want to see more information about how the data stream lifecycle is applied on individual backing indices use the [explain data stream lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle):
104104

105105
```console
106-
GET .ds-my-data-stream-*/_lifecycle/explain
106+
GET .ds-my-data-stream-test/_lifecycle/explain
107107
```
108108

109+
:::{tip}
110+
You can use a wildcard (`*`) in the data stream name to retrieve the lifecycle status for all data streams matching the pattern.
111+
:::
112+
109113
The result will look like this:
110114

111115
```console-result
112116
{
113117
"indices": {
114-
".ds-my-data-stream-2023.04.19-000001": {
115-
"index": ".ds-my-data-stream-2023.04.19-000001", <1>
118+
".ds-my-data-stream-test-2023.04.19-000001": {
119+
"index": ".ds-my-data-stream-test-2023.04.19-000001", <1>
116120
"managed_by_lifecycle": true, <2>
117121
"index_creation_date_millis": 1681918009501,
118122
"time_since_index_creation": "1.6m", <3>

manage-data/lifecycle/data-stream/tutorial-data-stream-retention.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ products:
88
- id: elasticsearch
99
---
1010

11-
# Tutorial: Data stream retention [tutorial-manage-data-stream-retention]
11+
# Tutorial: Configure data stream retention [tutorial-manage-data-stream-retention]
1212

1313
In this tutorial, we are going to go over the data stream lifecycle retention; we will define it, go over how it can be configured and how it can gets applied. Keep in mind, the following options apply only to data streams that are managed by the data stream lifecycle.
1414

manage-data/lifecycle/data-stream/tutorial-migrate-ilm-managed-data-stream-to-data-stream-lifecycle.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -312,8 +312,6 @@ GET _data_stream/dsl-data-stream
312312
3. The new write index received the `false` value for the `prefer_ilm` setting, as we configured in the index template
313313
4. The new write index is managed by `Data stream lifecycle`
314314

315-
316-
317315
## Migrate data stream back to ILM [migrate-from-dsl-to-ilm]
318316

319317
We can easily change this data stream to be managed by {{ilm-init}} because we didn’t remove the {{ilm-init}} policy when we [updated the index template](#update-index-template-for-dsl).
@@ -333,7 +331,7 @@ PUT _data_stream/dsl-data-stream/_lifecycle
333331
}
334332
```
335333

336-
1. The `enabled` flag can be ommitted and defaults to `true` however, here we explicitly configure it to `false` Let’s check the state of the data stream:
334+
1. The `enabled` flag can be omitted and defaults to `true` however, here we explicitly configure it to `false` Let’s check the state of the data stream:
337335

338336

339337
```console

manage-data/lifecycle/data-stream/tutorial-update-existing-data-stream.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,14 @@ products:
88
- id: elasticsearch
99
---
1010

11-
# Tutorial: Update existing data stream [tutorial-manage-existing-data-stream]
11+
# Tutorial: Update the lifecycle of a data stream [tutorial-manage-existing-data-stream]
1212

13-
To update the lifecycle of an existing data stream, perform the following actions:
13+
Follow these steps to configure or remove data stream lifecycle settings for an existing, individual data stream:
1414

15-
1. [Set a data stream’s lifecycle](#set-lifecycle)
16-
2. [Remove lifecycle for a data stream](#delete-lifecycle)
15+
- [Set a data stream’s lifecycle](#set-lifecycle)
16+
- [Remove the lifecycle for a data stream](#delete-lifecycle)
1717

18+
Note that these steps are for data stream lifecycle only. For the steps to configure {{ilm}}, refer to the [{{ilm-init}} documentation](/manage-data/lifecycle/index-lifecycle-management.md). For a comparison between the two, refer to [](/manage-data/lifecycle.md).
1819

1920
## Set a data stream’s lifecycle [set-lifecycle]
2021

@@ -143,7 +144,7 @@ The response will look like:
143144
:::
144145
:::::
145146

146-
## Remove lifecycle for a data stream [delete-lifecycle]
147+
## Remove the lifecycle for a data stream [delete-lifecycle]
147148

148149
To remove the lifecycle of a data stream you can use the **Index Management** tools in {{kib}} or the {{es}} [delete lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete-data-lifecycle).
149150

0 commit comments

Comments
 (0)