Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,42 @@ products:

# Tutorial: Update existing data stream [tutorial-manage-existing-data-stream]

To update the lifecycle of an existing data stream you do the following actions:
To update the lifecycle of an existing data stream, perform the following actions:

1. [Set a data stream’s lifecycle](#set-lifecycle)
2. [Remove lifecycle for a data stream](#delete-lifecycle)


## Set a data stream’s lifecycle [set-lifecycle]

To add or to change the retention period of your data stream you can use the [PUT lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-data-lifecycle).
To add or to change the retention period of your data stream you can use the **Index Management** tools in {{kib}} or the {{es}} [lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-data-lifecycle).


:::::{tab-set}
:group: kibana-api
:::{tab-item} {{kib}}
:sync: kibana

To change the data retention settings for a data stream:

1. Go to **Stack Management > Index Management** and open the **Data Streams** tab.
1. Use the search tool to find the data stream you're looking for.
1. Select the data stream to view its details.
1. In the data stream details pane, select **Manage > Edit data retention** to adjust the settings. You can do any of the following:

- Select how long to retain your data, in days, hours, minutes, or seconds.
- Choose to **Keep data indefinitely**, so that your data will not be deleted. Your data stream is still managed but the data will never be deleted. Managing a time series data stream such as for logs or metrics enables {{es}} to better store your data even if you do not use a retention period.
- Disable **Enable data retention** to turn off data stream lifecycle management for your data stream.

Note that if the data stream is already managed by [{{ilm-init}}](/manage-data/lifecycle/index-lifecycle-management.md), to edit the data retention settings you must edit the associated {{ilm-init}} policy.


:::

:::{tab-item} API
:sync: api

To change the data retention settings for a data stream:

* You can set infinite retention period, meaning that your data should never be deleted. For example:

Expand All @@ -27,7 +54,7 @@ To add or to change the retention period of your data stream you can use the [PU
{ } <1>
```

1. An empty payload means that your data stream is still managed but the data will never be deleted. Managing a time series data stream such as logs or metrics enables {{es}} to better store your data even if you do not use a retention period.
1. An empty payload means that your data stream is still managed but the data will never be deleted. Managing a time series data stream such as for logs or metrics enables {{es}} to better store your data even if you do not use a retention period.

* Or you can set the retention period of your choice. For example:

Expand All @@ -39,9 +66,32 @@ To add or to change the retention period of your data stream you can use the [PU
```

1. The retention period of this data stream is set to 30 days. This means that {{es}} is allowed to delete data that is older than 30 days at its own discretion.
:::
:::::

The changes in the lifecycle are applied on all backing indices of the data stream.

You can see the effect of the change in {{kib}} or using the {{es}} [explain API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle):

:::::{tab-set}
:group: kibana-api
:::{tab-item} {{kib}}
:sync: kibana
To check the data retention settings for a data stream:

1. Go to **Stack Management > Index Management** and open the **Data Streams** tab.
1. Use the search tool to find the data stream you're looking for.
1. Select the data stream to view its details. The flyout shows the data retention settings for the data stream. Note that if the data stream is currently managed by an [{{ilm-init}} policy](/manage-data/lifecycle/index-lifecycle-management.md), the **Effective data retention** may differ from the retention value that you've set in the data stream, as indicated by the **Data retention**.

:::{image} /manage-data/images/elasticsearch-reference-lifecycle-status.png
:alt: Index lifecycle status page
:width: 500px
:::

:::{tab-item} API
:sync: api

The changes in the lifecycle are applied on all backing indices of the data stream. You can see the effect of the change via the [explain API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle):
To check the data retention settings for a data stream:

```console
GET .ds-my-data-stream-*/_lifecycle/explain
Expand Down Expand Up @@ -90,16 +140,48 @@ The response will look like:
9. The time that will be used to determine when it’s safe to delete this index and all its data.
10. The data retention for this index as well is at least 30 days, as it was recently updated.


:::
:::::

## Remove lifecycle for a data stream [delete-lifecycle]

To remove the lifecycle of a data stream you can use the [delete lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete-data-lifecycle). As consequence, the maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all its backing indices. For example:
To remove the lifecycle of a data stream you can use the **Index Management** tools in {{kib}} or the {{es}} [delete lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete-data-lifecycle).


:::::{tab-set}
:group: kibana-api
:::{tab-item} {{kib}}
:sync: kibana

To remove a data stream's lifecycle:

1. Go to **Stack Management > Index Management** and open the **Data Streams** tab.
1. Use the search tool to find the data stream you're looking for.
1. Select the data stream to view its details.
1. In the data stream details pane, select **Manage > Edit data retention**.
1. Turn off the **Enable data retention** option and save your changes. The maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all of its backing indices.

You can confirm your changes by reopening the data stream pane. The **Effective data retention** will show a **Disabled** status.

::::{image} /manage-data/images/elasticsearch-reference-lifecycle-disabled.png
:alt: Index lifecycle status is disabled
:width: 500px
::::


:::

:::{tab-item} API
:sync: api

To remove a data stream's lifecycle:

```console
DELETE _data_stream/my-data-stream/_lifecycle
```

After running the API request, the maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all of its backing indices.

You can then use the [explain API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-explain-data-lifecycle) again to see that the indices are no longer managed.

```console
Expand All @@ -125,5 +207,5 @@ GET .ds-my-data-stream-*/_lifecycle/explain
2. Indication that the index is not managed by the data stream lifecycle.
3. The name of another backing index.
4. Indication that the index is not managed by the data stream lifecycle.


:::
:::::
Loading