Consolidate ILM tutorials

Author: kilfoyle

manage-data/_snippets/create-lifecycle-policy.md

@@ -0,0 +1,66 @@
+A lifecycle policy specifies the phases in the index lifecycle and the actions to perform in each phase. A lifecycle can have up to five phases: `hot`, `warm`, `cold`, `frozen`, and `delete`.
+
+For example, you might define a policy named `timeseries_policy` that has the following two phases:
+
+* A `hot` phase that defines a rollover action to specify that an index rolls over when it reaches either a `max_primary_shard_size` of 50 gigabytes or a `max_age` of 30 days.
+* A `delete` phase that sets `min_age` to remove the index 90 days after rollover.
+
+::::{note}
+The `min_age` value is relative to the rollover time, not the index creation time. [Learn more](../../../troubleshoot/elasticsearch/index-lifecycle-management-errors.md#min-age-calculation).
+
+::::
+
+
+You can create the policy in {{kib}} or with the [create or update policy](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-put-lifecycle) API. 
+
+::::{tab-set}
+:group: kibana-api
+:::{tab-item} {{kib}}
+:sync: kibana
+To create the policy from {{kib}}, open the menu and go to **Stack Management > Index Lifecycle Policies**. Click **Create policy**.
+
+:::{image} /manage-data/images/elasticsearch-reference-create-policy.png
+:alt: Create policy page
+:screenshot:
+:::
+:::
+
+:::{tab-item} API
+:sync: api
+Use the [Create or update policy](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-put-lifecycle) API to add an ILM policy to the {{es}} cluster:
+
+```console
+PUT _ilm/policy/timeseries_policy
+{
+  "policy": {
+    "phases": {
+      "hot": {                                <1>
+        "actions": {
+          "rollover": {
+            "max_primary_shard_size": "50GB", <2>
+            "max_age": "30d"
+          }
+        }
+      },
+      "delete": {
+        "min_age": "90d",                     <3>
+        "actions": {
+          "delete": {}                        <4>
+        }
+      }
+    }
+  }
+}
+```
+
+1. The `min_age` defaults to `0ms`, so new indices enter the `hot` phase immediately.
+2. Trigger the `rollover` action when either of the conditions are met.
+3. Move the index into the `delete` phase 90 days after rollover.
+4. Trigger the `delete` action when the index enters the delete phase.
+
+:::
+::::
+
+:::{tip}
+For more details about default {{ilm-init}} policy settings, refer to [Create a lifecycle policy](/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md#ilm-create-policy).
+:::

manage-data/data-store/aliases.md

@@ -268,7 +268,7 @@ POST _aliases
 If an alias points to multiple indices or data streams and `is_write_index` isn’t set, the alias rejects write requests. If an index alias points to one index and `is_write_index` isn’t set, the index automatically acts as the write index. Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.
 
 ::::{tip}
-We recommend using data streams to store append-only time series data. If you need to update or delete existing time series data, you can perform update or delete operations directly on the data stream backing index. If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. See [Manage time series data without data streams](../lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams).
+We recommend using data streams to store append-only time series data. If you need to update or delete existing time series data, you can perform update or delete operations directly on the data stream backing index. If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. See the tutorial [](../lifecycle/index-lifecycle-management/tutorial-time-series-without-data-streams.md).
 ::::
 
 

manage-data/data-store/data-streams.md

@@ -26,7 +26,7 @@ To determine whether you should use a data stream for your data, you should cons
 * You mostly perform indexing requests, with occasional updates and deletes.
 * You index documents without an `_id`, or when indexing documents with an explicit `_id` you expect first-write-wins behavior.
 
-For most time series data use-cases, a data stream will be a good fit. However, if you find that your data doesn’t fit into these categories (for example, if you frequently send multiple documents using the same `_id` expecting last-write-wins), you may want to use an index alias with a write index instead. See documentation for [managing time series data without a data stream](../lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) for more information.
+For most time series data use-cases, a data stream will be a good fit. However, if you find that your data doesn’t fit into these categories (for example, if you frequently send multiple documents using the same `_id` expecting last-write-wins), you may want to use an index alias with a write index instead. See the tutorial [](../lifecycle/index-lifecycle-management/tutorial-time-series-without-data-streams.md) for more information.
 
 Keep in mind that some features such as [Time Series Data Streams (TSDS)](../data-store/data-streams/time-series-data-stream-tsds.md) and [data stream lifecycles](../lifecycle/data-stream.md) require a data stream.
 
@@ -106,7 +106,7 @@ Data streams are designed for use cases where existing data is rarely updated. Y
 If you need to update a larger number of documents in a data stream, you can use the [update by query](data-streams/use-data-stream.md#update-docs-in-a-data-stream-by-query) and [delete by query](data-streams/use-data-stream.md#delete-docs-in-a-data-stream-by-query) APIs.
 
 ::::{tip}
-If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. See [Manage time series data without data streams](../lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams).
+If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. See the tutorial [](../lifecycle/index-lifecycle-management/tutorial-time-series-without-data-streams.md).
 ::::
 
 

manage-data/data-store/data-streams/modify-data-stream.md

@@ -344,7 +344,7 @@ Follow these steps:
     ```
 
 4. If you do not want to mix new and old data in your new data stream, pause the indexing of new documents. While mixing old and new data is safe, it could interfere with data retention. See [Mixing new and old data in a data stream](../data-streams/modify-data-stream.md#data-stream-mix-new-old-data).
-5. If you use {{ilm-init}} to [automate rollover](../../lifecycle/index-lifecycle-management/tutorial-automate-rollover.md), reduce the {{ilm-init}} poll interval. This ensures the current write index doesn’t grow too large while waiting for the rollover check. By default, {{ilm-init}} checks rollover conditions every 10 minutes.
+5. If you use {{ilm-init}} to [automate rollover](../../lifecycle/index-lifecycle-management/tutorial-time-series-with-data-streams.md), reduce the {{ilm-init}} poll interval. This ensures the current write index doesn’t grow too large while waiting for the rollover check. By default, {{ilm-init}} checks rollover conditions every 10 minutes.
 
     The following [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings) request lowers the `indices.lifecycle.poll_interval` setting to `1m` (one minute).
 

manage-data/data-store/data-streams/set-up-data-stream.md

@@ -213,7 +213,7 @@ For an example, see [Data stream privileges](../../../deploy-manage/users-roles/
 
 ## Convert an index alias to a data stream [convert-index-alias-to-data-stream]
 
-Prior to {{es}} 7.9, you’d typically use an [index alias with a write index](../../lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) to manage time series data. Data streams replace this functionality, require less maintenance, and automatically integrate with [data tiers](../../lifecycle/data-tiers.md).
+Prior to {{es}} 7.9, you’d typically use an [index alias with a write index](../../lifecycle/index-lifecycle-management/tutorial-time-series-without-data-streams.md) to manage time series data. Data streams replace this functionality, require less maintenance, and automatically integrate with [data tiers](../../lifecycle/data-tiers.md).
 
 To convert an index alias with a write index to a data stream with the same name, use the [migrate to data stream API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-migrate-to-data-stream). During conversion, the alias’s indices become hidden backing indices for the stream. The alias’s write index becomes the stream’s write index. The stream still requires a matching index template with data stream enabled.
 

manage-data/lifecycle/index-lifecycle-management/ilm-tutorials.md

@@ -0,0 +1,26 @@
+---
+navigation_title: ILM tutorials
+applies_to:
+  stack: ga
+products:
+  - id: elasticsearch
+---
+
+# {{ilm}} tutorials
+
+A collection of tutorials is available to help you control the lifecycle of your data using {{ilm}}.
+
+## Configuring rollover
+
+When you continuously index timestamped documents into {{es}}, you typically use a [data stream](../../data-store/data-streams.md) so you can periodically [roll over](rollover.md) to a new index. This enables you to implement a [hot-warm-cold architecture](../data-tiers.md) to meet the performance requirements for your newest data, control costs over time, enforce retention policies, and still get the most out of your data.
+
+To simplify index management and automate rollover, select one of the scenarios that best applies to your situation:
+
+* **Roll over data streams with ILM.** When ingesting write-once, timestamped data that doesn't change, follow the steps in [](/manage-data/lifecycle/index-lifecycle-management/tutorial-time-series-with-data-streams.md) for simple, automated data stream rollover. ILM-managed backing indices are automatically created under a single data stream alias. ILM also tracks and transitions the backing indices through the lifecycle automatically. 
+* **Roll over time series indices with ILM.** Data streams are best suited for [append-only](../../data-store/data-streams.md#data-streams-append-only) use cases. If you need to update or delete existing time series data, you can perform update or delete operations directly on the data stream backing index. If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. You can still use {{ilm-init}} to manage and roll over the alias’s indices. Follow the steps in [](/manage-data/lifecycle/index-lifecycle-management/tutorial-time-series-without-data-streams.md) for more information.
+* **Roll over general content as data streams with ILM.** If some of your indices store data that isn't timestamped, but you would like to get the benefits of automatic rotation when the index reaches a certain size or age, or delete already rotated indices after a certain amount of time, follow the steps in [](/manage-data/lifecycle/index-lifecycle-management/tutorial-general-content-with-data-streams.md). These steps include injecting a timestamp field during indexing time to mimic time series data.
+
+## Customizing a built-in {{ilm-init}} policy
+
+When your data streams have been set up automatically, for example when you're ingesting data by means of an [Elastic integration](https://docs.elastic.co/en/integrations), the data is typically managed using a built-in {{ilm-init}} policy. To customize the lifecycle policy for managed indices, refer to: [](/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md).
+

manage-data/lifecycle/index-lifecycle-management/rollover.md

@@ -22,6 +22,16 @@ The rollover feature is an important part of how [index lifecycle](../index-life
 
 Rollover can be triggered via the [API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover), {{ilm-init}}, or {{dlm-init}}.
 
+:::{tip}
+The following tutorials are available to help you configure rollover for your indices, for three different scenarios:
+* [](/manage-data/lifecycle/index-lifecycle-management/tutorial-time-series-with-data-streams.md)
+* [](/manage-data/lifecycle/index-lifecycle-management/tutorial-time-series-without-data-streams.md)
+* [](/manage-data/lifecycle/index-lifecycle-management/tutorial-general-content-with-data-streams.md)
+
+Refer to [](/manage-data/lifecycle/index-lifecycle-management/ilm-tutorials.md) for an overview of these.
+
+:::
+
 ## How rollover works in {{ilm-init}}
 
 You define a rollover action in the hot phase of an index lifecycle policy. It will run when any of the configured conditions are met and the write index contains at least one document.
@@ -70,7 +80,7 @@ Each data stream requires an [index template](../../data-store/templates.md) tha
 * A configuration that indicates a data stream is used for the index pattern.
 * Optional: The mappings and settings applied to each backing index when it’s created.
 
-For more information about this approach, refer to the [Manage time series data with data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-with-data-streams) tutorial.
+For more information about this approach, refer to the [](../index-lifecycle-management/tutorial-time-series-with-data-streams.md) tutorial.
 
 :::{tip}
 Data streams are designed for append-only data, where the data stream name can be used as the operations (read, write, rollover, shrink etc.) target. If your use case requires data to be updated in place, you can perform [update or delete operations directly on the backing indices](../../data-store/data-streams/use-data-stream.md#update-delete-docs-in-a-backing-index).
@@ -86,7 +96,7 @@ For more information about the data stream naming pattern, refer to the [Generat
 
 ### Rotating your indices with aliases [rollover-with-aliases]
 
- Rotating indices with aliases requires additional configuration steps, including bootstrapping the initial index. For more details about this approach, refer to the [Manage time series data without data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) tutorial.
+ Rotating indices with aliases requires additional configuration steps, including bootstrapping the initial index. For more details about this approach, refer to the [](../index-lifecycle-management/tutorial-time-series-without-data-streams.md) tutorial.
 
 :::{important}
 The use of aliases for rollover requires meeting certain conditions. Review these considerations before applying this approach: