Skip to content

Commit 45c4266

Browse files
committed
Applying review recommendations
1 parent 0708c30 commit 45c4266

File tree

1 file changed

+27
-22
lines changed
  • manage-data/lifecycle/index-lifecycle-management

1 file changed

+27
-22
lines changed

manage-data/lifecycle/index-lifecycle-management/rollover.md

Lines changed: 27 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -10,66 +10,71 @@ products:
1010
# Rollover [index-rollover]
1111

1212
In {{es}}, the [rollover action](elasticsearch://reference/elasticsearch/index-lifecycle-actions/ilm-rollover.md) replaces your active write index with a new one whenever your index grows too large, too old, or stores too many documents.
13-
This is particularly useful for time-series data, such as logs or metrics, where index growth is continuous and performance can degrade over time.
13+
This is particularly useful for time-series data, such as logs or metrics where index growth is continuous, in order to meet performance and retention requirements.
1414

15-
Without rollover, a single index can grow until searches slow down, shard relocation takes hours, and nodes run low on disk space.
15+
Without rollover, a single index would continue to grow, causing search performance to drop and having a higher administrative burden on the cluster.
1616

17-
The rollover feature is an important part of how [index lifecycle](../index-lifecycle-management/index-lifecycle.md) and [data stream lifecycles](../data-stream.md) work to keep your indices fast and manageable. By switching the write target of an index, the rollover action provides the following benefits:
17+
The rollover feature is an important part of how [index lifecycle](../index-lifecycle-management/index-lifecycle.md) (ILM) and [data stream lifecycles](../data-stream.md) (DLM) work to keep your indices fast and manageable. By switching the write target of an index, the rollover action provides the following benefits:
1818

19-
* **Automation** - works with ILM to remove manual index rotation tasks and allows for granular control over retention cycles
19+
* **Lifecycle** - works with lifecycle management (ILM or DLM) to transition the index through its lifecycle actions and allows for granular control over retention cycles
2020
* **Optimized performance** - keeps shard sizes within recommended limits (10-50 GB)
2121
* **Queries run faster** - improves search performance
2222

23-
Rollover can be triggered via the [API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover) or ILM.
23+
Rollover can be triggered via the [API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover), ILM, or DLM.
2424

2525
## How rollover works in ILM
2626

27-
You define a rollover action in the hot phase of an index lifecycle policy. It will run when any of the configured thresholds are met.
27+
You define a rollover action in the hot phase of an index lifecycle policy. It will run when any of the configured thresholds are met and the write index contains at least one document.
2828
You can configure the following rollover conditions:
2929

30-
* **Size** - an index will rollover when it reaches a set size, for example 50 GB.
30+
* **Size** - an index will rollover when its shards reach a set size, for example 50 GB.
3131
* **Age** - an index will rollover when an index reaches a certain age, for example 7 days.
3232
* **Document count** - an index will rollover when a shard contains a certain number of documents, for example 2 million.
3333

3434
::::{tip}
3535
Rolling over to a new index based on size, document count, or age is preferable to time-based rollovers. Rolling over at an arbitrary time often results in many small indices, which can have a negative impact on performance and resource usage.
3636
::::
3737

38-
After rollover occurs in the hot phase of an ILM policy, indices move to other index lifecycle phases like warm, cold, frozen, and delete. Rollover creates a new write index while the old one continues through the lifecycle phases.
38+
After rollover, indices move to other index lifecycle phases like warm, cold, frozen, and delete. Rollover creates a new write index while the old one continues through the lifecycle phases.
3939

4040
**Special rules:**
4141

42-
* Empty indices rollover is blocked even if they have an associated `max_age` that would otherwise result in a roll over occurring. A policy can override this behavior if you set `min_docs: 0` in the rollover conditions. This can also be disabled on a cluster-wide basis if you set `indices.lifecycle.rollover.only_if_has_documents` to `false`.
42+
* Rollover for an empty write index is skipped even if they have an associated `max_age` that would otherwise result in a roll over occurring. A policy can override this behavior if you set `min_docs: 0` in the rollover conditions. This can also be disabled on a cluster-wide basis if you set `indices.lifecycle.rollover.only_if_has_documents` to `false`.
4343
* Forced rollover occurs if any shard reaches 200 million documents. Usually, a shard will reach 50 GB long before it reaches 200 million documents, but this isn’t the case for space efficient data sets.
4444

4545
## Recommended approaches
4646

4747
Decide your approach to index rotation based on your use case and requirements.
4848

49-
| Use case | Recommended approach | Benefits |
50-
| ---------------------- | --------------------------------------------------------- | ------------------------------------- |
51-
| Logs, metrics | [Data streams](rollover.md#rollover-data-stream) | Automatic rollover, minimal setup |
52-
| Legacy indexing setup | [Alias-based rollover](rollover.md#rollover-with-aliases) | Granular control over rollover timing |
53-
| Small, static datasets | No rollover | Simpler management |
49+
| Use case | Recommended approach | Benefits and limitations |
50+
| ---------------------- | --------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
51+
| Logs, metrics | [Data streams](rollover.md#rollover-data-stream) | Automatic rollover with lifecycle management, minimal setup, control over rollover timing <1> |
52+
| Legacy indexing setup | [Alias-based rollover](rollover.md#rollover-with-aliases) | Granular control over rollover timing.<br> Note that this option requires advanced configuration steps. |
53+
| Small, static datasets | No rollover | Simpler management |
54+
55+
1. Rollover is handled automatically in Serverless projects, therefore rollover timing is abstracted from the user.
5456

5557
:::{tip}
56-
For new projects, use data streams. They're simple to manage and handle rollover automatically.
58+
For new projects, use data streams. They're simple to manage with lifecycle policies where you define phases and actions that handle rollover automatically.
5759
:::
5860

5961

6062
### Rotating your indices with data streams [rollover-data-stream]
6163

62-
We recommend using [data streams](../../data-store/data-streams.md) to manage time series data. Data streams automatically track the write index while keeping configuration to a minimum.
64+
We recommend using [data streams](../../data-store/data-streams.md) to manage time series data. Data streams set up with lifecycle policies automatically track the write index while keeping configuration to a minimum.
6365
When targeting a data stream, the new backing index becomes the data stream's writing index. The generation of new backing indices is incremented automatically when it reaches a specified age or size.
6466

6567
Each data stream requires an [index template](../../data-store/templates.md) that contains the following:
6668

6769
* A name or wildcard (`*`) pattern for the data stream.
68-
* The data stream’s timestamp field. This field must be mapped as a [`date`](elasticsearch://reference/elasticsearch/mapping-reference/date.md) or [`date_nanos`](elasticsearch://reference/elasticsearch/mapping-reference/date_nanos.md) field data type and must be included in every document indexed to the data stream.
69-
* The mappings and settings applied to each backing index when it’s created.
70+
* Optional: The mappings and settings applied to each backing index when it’s created.
7071

7172
For more information about this approach, refer to the [Manage time series data with data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-with-data-streams) tutorial.
7273

74+
:::{tip}
75+
Data streams are designed for append-only data, where the data stream name can be used as the operations (read, write, rollover, shrink etc.) target. If your use case requires data to be updated in place, you can perform [update or delete operations directly on the backing indices](../../data-store/data-streams/use-data-stream.md#update-delete-docs-in-a-backing-index).
76+
:::
77+
7378
**Data streams naming pattern**<br>
7479
{{es}} uses a structured naming convention for the backing indices of data streams, following this pattern:
7580

@@ -80,18 +85,18 @@ For more information about data stream naming patterns, refer to the [Generation
8085

8186
### Rotating your indices with aliases [rollover-with-aliases]
8287

83-
Data streams are designed for append-only data, where the data stream name can be used as the operations (read, write, rollover, shrink etc.) target. If your use case requires data to be updated in place, you can instead manage your time series data using [index aliases](../../data-store/aliases.md). However, this approach requires additional configuration steps, including bootstrapping the initial index. For more details about this approach, refer to the [Manage time series data without data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) tutorial.
88+
Rotating indices with aliases requires additional configuration steps, including bootstrapping the initial index. For more details about this approach, refer to the [Manage time series data without data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) tutorial.
8489

8590
:::{important}
86-
To roll over an index alias, the alias and its [write index](../../data-store/aliases.md#write-index) must meet the following conditions:
91+
The use of aliases for rollover requires meeting certain conditions. Review these considerations before applying this approach:
8792

8893
* The index name must match the pattern `^.-\d+$*`, for example `my-index-000001`.
8994
* The `index.lifecycle.rollover_alias` must be configured as the alias to roll over.
90-
* The index must be the write index for the alias.
95+
* The index must be the [write index](../../data-store/aliases.md#write-index) for the alias.
9196
:::
9297

9398
::::{note}
94-
When an index is rolled over, the previous index’s age is updated to reflect the rollover time. This date, rather than the index’s `creation_date`, is used in {{ilm}} `min_age` phase calculations. [Learn more](../../../troubleshoot/elasticsearch/index-lifecycle-management-errors.md#min-age-calculation).
99+
When an alias or data stream is rolled over, the previous write index’s rollover time is stored. This date, rather than the index’s `creation_date`, is used in {{ilm}} `min_age` phase calculations. [Learn more](../../../troubleshoot/elasticsearch/index-lifecycle-management-errors.md#min-age-calculation).
95100

96101
::::
97102

0 commit comments

Comments
 (0)