You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
***Queries run faster** - improves search performance
22
22
23
+
Rollover can be triggered via the [API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover) or ILM.
24
+
25
+
## How rollover works in ILM
26
+
27
+
You define a rollover action in the hot phase of an index lifecycle policy. It will run when any of the configured thresholds are met.
28
+
You can configure the following rollover conditions:
29
+
30
+
***Size** - an index will rollover when it reaches a set size, for example 50 GB.
31
+
***Age** - an index will rollover when an index reaches a certain age, for example 7 days.
32
+
***Document count** - an index will rollover when a shard contains a certain number of documents, for example 2 million.
33
+
34
+
::::{tip}
35
+
Rolling over to a new index based on size, document count, or age is preferable to time-based rollovers. Rolling over at an arbitrary time often results in many small indices, which can have a negative impact on performance and resource usage.
36
+
::::
37
+
38
+
After rollover occurs in the hot phase of an ILM policy, indices move to other index lifecycle phases like warm, cold, frozen, and delete. Rollover creates a new write index while the old one continues through the lifecycle phases.
39
+
40
+
**Special rules:**
41
+
42
+
* Empty indices rollover is blocked even if they have an associated `max_age` that would otherwise result in a roll over occurring. A policy can override this behavior if you set `min_docs: 0` in the rollover conditions. This can also be disabled on a cluster-wide basis if you set `indices.lifecycle.rollover.only_if_has_documents` to `false`.
43
+
* Forced rollover occurs if any shard reaches 200 million documents. Usually, a shard will reach 50 GB long before it reaches 200 million documents, but this isn’t the case for space efficient data sets.
23
44
24
45
## Recommended approaches
25
46
@@ -39,52 +60,40 @@ For new projects, use data streams. They're simple to manage and handle rollover
39
60
### Rotating your indices with data streams [rollover-data-stream]
40
61
41
62
We recommend using [data streams](../../data-store/data-streams.md) to manage time series data. Data streams automatically track the write index while keeping configuration to a minimum.
42
-
When targeting a data stream, the new index becomes the data stream's writing index and it's generation is incremented.
63
+
When targeting a data stream, the new backing index becomes the data stream's writing index. The generation of new backing indices is incremented automatically when it reaches a specified age or size.
43
64
44
65
Each data stream requires an [index template](../../data-store/templates.md) that contains the following:
45
66
46
67
* A name or wildcard (`*`) pattern for the data stream.
47
68
* The data stream’s timestamp field. This field must be mapped as a [`date`](elasticsearch://reference/elasticsearch/mapping-reference/date.md) or [`date_nanos`](elasticsearch://reference/elasticsearch/mapping-reference/date_nanos.md) field data type and must be included in every document indexed to the data stream.
48
69
* The mappings and settings applied to each backing index when it’s created.
49
70
71
+
For more information about this approach, refer to the [Manage time series data with data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-with-data-streams) tutorial.
72
+
73
+
**Data streams naming pattern**<br>
74
+
{{es}} uses a structured naming convention for the backing indices of data streams, following this pattern:
50
75
76
+
```console
77
+
.ds-<DATA-STREAM-NAME>-<yyyy.MM.dd>-<GENERATION>
78
+
```
79
+
For more information about data stream naming patterns, refer to the [Generation](../../data-store/data-streams.md#data-streams-generation) section of the Data streams page.
51
80
52
81
### Rotating your indices with aliases [rollover-with-aliases]
53
82
54
-
Data streams are designed for append-only data, where the data stream name can be used as the operations (read, write, rollover, shrink etc.) target. If your use case requires data to be updated in place, you can instead manage your time series data using [index aliases](../../data-store/aliases.md). However, this approach requires additional configuration steps.
83
+
Data streams are designed for append-only data, where the data stream name can be used as the operations (read, write, rollover, shrink etc.) target. If your use case requires data to be updated in place, you can instead manage your time series data using [index aliases](../../data-store/aliases.md). However, this approach requires additional configuration steps, including bootstrapping the initial index. For more details about this approach, refer to the [Manage time series data without data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) tutorial.
55
84
85
+
:::{important}
56
86
To roll over an index alias, the alias and its [write index](../../data-store/aliases.md#write-index) must meet the following conditions:
57
87
58
88
* The index name must match the pattern `^.-\d+$*`, for example `my-index-000001`.
59
89
* The `index.lifecycle.rollover_alias` must be configured as the alias to roll over.
60
90
* The index must be the write index for the alias.
61
-
91
+
:::
62
92
63
93
::::{note}
64
94
When an index is rolled over, the previous index’s age is updated to reflect the rollover time. This date, rather than the index’s `creation_date`, is used in {{ilm}} `min_age` phase calculations. [Learn more](../../../troubleshoot/elasticsearch/index-lifecycle-management-errors.md#min-age-calculation).
65
95
66
96
::::
67
97
68
-
69
-
## How rollover works in ILM
70
-
71
-
You define a rollover action in the hot phase of an ILM policy. It will run when any of the configured thresholds are met.
72
-
You can configure the following rollover conditions:
73
-
74
-
***Size** - an index will rollover when it reaches a set size, for example 50 GB.
75
-
***Age** - an index will rollover when an index reaches a certain age, for example 7 days.
76
-
***Document count** - an index will rollover when a shard contains a certain number of documents, for example 2 million.
77
-
78
-
::::{tip}
79
-
Rolling over to a new index based on size, document count, or age is preferable to time-based rollovers. Rolling over at an arbitrary time often results in many small indices, which can have a negative impact on performance and resource usage.
80
-
::::
81
-
82
-
**Special rules:**
83
-
84
-
* Empty indices rollover is blocked even if they have an associated `max_age` that would otherwise result in a roll over occurring. A policy can override this behavior if you set `min_docs: 0` in the rollover conditions. This can also be disabled on a cluster-wide basis if you set `indices.lifecycle.rollover.only_if_has_documents` to `false`.
85
-
* Forced rollover occurs if any shard reaches 200 million documents. Usually, a shard will reach 50 GB long before it reaches 200 million documents, but this isn’t the case for space efficient data sets.
86
-
87
-
88
-
89
-
90
-
??? link to [data streams API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create-data-stream)
98
+
**Alias-based naming pattern**<br>
99
+
When configured correctly, the newly created write index will have a similar name to one that's been rolled over, however the six-digit, zero-padded suffix will be incremented. For example before rollover, the write index was called `my-index-000001` and after rollover, the newly created index becomes `my-index-000002` and also becomes the new write index. The alias typically shares the base name which in this example is `my-index`.
0 commit comments