You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md
+9-276Lines changed: 9 additions & 276 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,9 +16,9 @@ There a few things to note about how an {{ilm-init}} policy works:
16
16
17
17
* For {{ilm-init}} to manage an index, you need to specify a valid policy in the `index.lifecycle.name` index setting.
18
18
19
-
* To configure a lifecycle policy for [rolling indices](rollover.md), you create the policy and add it to the [index template](../../data-store/templates.md).
19
+
* To configure a lifecycle policy for [rolling indices](rollover.md) or data streams, you create the policy and add it to the [index template](../../data-store/templates.md).
20
20
21
-
* To use a policy to manage an index that doesn’t roll over, you can specify a lifecycle policy when you create the index, or apply a policy directly to an existing index.
21
+
* To use a policy to manage a single index, you can specify a lifecycle policy when you create the index, or apply a policy directly to an existing index.
22
22
23
23
* {{ilm-init}} policies are stored in the global cluster state and can be included in snapshots by setting `include_global_state` to `true` when you [take the snapshot](../../../deploy-manage/tools/snapshot-and-restore/create-snapshots.md). When the snapshot is restored, all of the policies in the global state are restored and any local policies with the same names are overwritten.
24
24
@@ -32,12 +32,6 @@ To set up ILM to manage one or more indices, the general procedure is as follows
32
32
If you're configuring ILM for rolling indices and not using [data streams](../../data-store/data-streams.md), you additionally need to:
33
33
34
34
3.[Create an initial managed index](#create-initial-index)
35
-
4.[Apply a lifecycle policy manually](#apply-policy-manually)
36
-
37
-
Once ILM is set up, at any time you can:
38
-
39
-
*[View the lifecycle status for an index](#view-lifecycle-status)
You can perform these actions in {{kib}} or using the {{es}} API.
43
37
@@ -59,12 +53,12 @@ To add an ILM policy to an {{es}} cluster:
59
53
60
54
1. In the **Hot phase**, by default an ILM-managed index [rolls over](elasticsearch://reference/elasticsearch/index-lifecycle-actions/ilm-rollover.md) when either:
61
55
* It reaches 30 days of age.
62
-
* The primary shard reaches 50 gigabytes in size.
56
+
* One or more primary shards reach 50 gigabytes in size.
63
57
64
-
Disaable**Use recommended defaults** to adjust these values or to roll over based on the size of the primary shard, the number of documents in the primary shard, or the total number of documents in the index.
58
+
Disable**Use recommended defaults** to adjust these values or to roll over based on the size of the primary shard, the number of documents in the primary shard, or the total number of documents in the index.
65
59
66
60
::::{important}
67
-
The rollover action implicitly always rolls over a data stream or alias if one or more shards contain 200000000 or more documents. Normally a shard will reach 25GB long before it reaches 200M documents, but this isn’t the case for space efficient data sets. Search performance will very likely suffer if a shard contains more than 200M documents. This is the reason for the built-in limit.
61
+
The rollover action implicitly always rolls over a data stream or alias if one or more shards contain 200000000 or more documents. Normally a shard will reach 25GB long before it reaches 200M documents, but this isn’t the case for space efficient data sets. Search performance will very likely suffer if a shard contains more than 200M documents. This is the reason for the built-in limit.
68
62
::::
69
63
70
64
1. By default, only the "hot" index lifecycle phase is enabled. Enable each additional lifecycle phase that you'd like, and for each choose the [index lifecycle actions](elasticsearch://reference/elasticsearch/index-lifecycle-actions/index.md) to perform on indices when they enter that phase.
@@ -129,10 +123,11 @@ To add an index template to a cluster and apply the lifecycle policy to indices
129
123
1. Specify a pattern to match the indices you want to manage with the lifecycle policy. For example, `my-index-*`.
130
124
1. If you're storing continuously generated, append-only data, you can opt to create [data streams](/manage-data/data-store/data-streams.md) instead of indices for more efficient storage. If you enable this option, you can also enable **Data retention** to configure how long your indexed data is kept.
131
125
132
-
:::{important}
133
-
When the **Data retention** option is set, data is guaranteed to be stored for the specified retention duration. Elasticsearch is allowed at a later time to delete data older than this duration. This setting replaces any data retention settings that may be defined in an ILM policy. Refer to the [Data stream retention](/manage-data/lifecycle/data-stream/tutorial-data-stream-retention.md) tutorial to learn more.
126
+
:::{note}
127
+
Since you're creating an index lifecycle policy to manage indices, the **Data retention** option should be left disabled. Data retention is applicable only if you're using a data stream lifecycle, which is an alternative to ILM. Refer to the [Data stream lifecycle](/manage-data/lifecycle/data-stream.md) to learn more.
134
128
:::
135
129
130
+
136
131
1. Configure any other options you'd like, including:
137
132
* The [index mode](elasticsearch://reference/elasticsearch/index-settings/time-series.md) to use for the created indices.
138
133
* The template priority, version, and any metadata.
@@ -222,7 +217,7 @@ To create the initial managed index:
222
217
223
218
Create an alias for the index:
224
219
225
-
1. Open **Developer tools**.
220
+
1. Open **Dev tools**.
226
221
2. Send the following request:
227
222
228
223
```console
@@ -262,265 +257,3 @@ Now you can start indexing data to the rollover alias specified in the lifecycle
262
257
:::
263
258
::::
264
259
265
-
## Apply a lifecycle policy manually [apply-policy-manually]
266
-
267
-
You can specify a policy when you create an index or apply a lifecycle policy to an existing index.
268
-
269
-
::::{important}
270
-
Do not manually apply a policy that uses the rollover action. Policies that use rollover must be applied by the [index template](#apply-policy-template). Otherwise, the policy is not carried forward when the rollover action creates a new index.
271
-
::::
272
-
273
-
::::{tab-set}
274
-
:group: kibana-api
275
-
:::{tab-item} {{kib}}
276
-
:sync: kibana
277
-
278
-
To apply a lifecycle policy to an existing index:
279
-
280
-
1. Go to **Stack Management > Index Management**. In the **Indices** tab, search for and select the index that you created, for example `test-00001`. Note that to view system indices you need to enable **Include hidden indices**.
281
-
282
-
1. From the **Manage index** dropdown menu select **Add lifecycle policy**.
283
-
284
-
1. Choose a lifecycle policy and confirm your changes.
285
-
286
-
Once the policy is applied, {{ilm-init}} starts managing the index immediately.
287
-
288
-
% QUESTION: Is there a way in the Kibana UI to add a policy to multiple indices?
289
-
:::
290
-
291
-
:::{tab-item} API
292
-
:sync: api
293
-
Use the [update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings) to apply a lifecycle policy to an existing index.
294
-
295
-
### Apply an policy to a single index [apply-policy-single]
296
-
297
-
In the following request, the `index.lifecycle.name` setting specifies an index’s policy.
298
-
299
-
```console
300
-
PUT my-index
301
-
{
302
-
"settings": {
303
-
"number_of_shards": 1,
304
-
"number_of_replicas": 1,
305
-
"index.lifecycle.name": "my_policy" <1>
306
-
}
307
-
}
308
-
```
309
-
310
-
1. Sets the lifecycle policy for the index.
311
-
312
-
Once the policy is applied, {{ilm-init}} starts managing the index immediately.
313
-
314
-
### Apply a policy to multiple indices [apply-policy-multiple]
315
-
316
-
You can apply the same policy to multiple indices by using wildcards in the index name when you call the API.
317
-
318
-
```console
319
-
PUT mylogs-pre-ilm*/_settings <1>
320
-
{
321
-
"index": {
322
-
"lifecycle": {
323
-
"name": "mylogs_policy_existing"
324
-
}
325
-
}
326
-
}
327
-
```
328
-
329
-
1. Updates all indices with names that start with `mylogs-pre-ilm`.
330
-
331
-
::::::{warning}
332
-
Be careful not to inadvertently match indices that you don’t want to modify.
333
-
::::::
334
-
:::
335
-
::::
336
-
337
-
## View the lifecycle status for an index [view-lifecycle-status]
338
-
339
-
For any existing managed index in your cluster, you can access the ILM policy applied to it and details such as its current phase.
340
-
341
-
::::{tab-set}
342
-
:group: kibana-api
343
-
:::{tab-item} {{kib}}
344
-
:sync: kibana
345
-
To view the current lifecycle status for one or more indices:
346
-
347
-
1. Go to **Stack Management > Index Management** and open the **Indices** tab.
348
-
1. Enable **Include hidden indices** to view all indices, including those managed by ILM. Note that if you're using data streams, you can find the data stream associated with any index listed in the **Data stream** column.
349
-
1. Use the search tool to find the index you're looking for. You can also filter the results by lifecycle status and lifecycle phase.
350
-
1. Select the index to view details.
351
-
1. Open the **Index lifecycle** tab to view ILM details such as the current lifecycle phase, the ILM policy name, the current [index lifecycle action](elasticsearch://reference/elasticsearch/index-lifecycle-actions/index.md), and other details.
352
-
353
-

354
-
355
-
To view the current lifecycle status for a datastream:
356
-
357
-
1. Go to **Stack Management > Index Management** and open the **Data Streams** tab.
358
-
1. Use the search tool to find the data stream you're looking for.
359
-
1. Select the data stream to view details. In the flyout that opens, you have direct links to access the ILM policy and the index template.
360
-
361
-

362
-
:::
363
-
364
-
:::{tab-item} API
365
-
:sync: api
366
-
Use the [Explain the lifecycle state API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-explain-lifecycle) to view the current lifecycle status for an index:
367
-
368
-
```console
369
-
GET my-index/_ilm/explain
370
-
```
371
-
372
-
The API response shows the current ILM phase and other details:
You may want to change the ILM policy applied to an index. For example, if you're ingesting a large set of log data you may want to apply a different policy to a subset of indices containing only the most critical logs.
463
-
464
-
:::::{warning}
465
-
Be careful when changing either the `logs@lifecycle` or `metrics@lifecycle` policies as these typically manage many indices. In {{kib}}, the **Index Lifecycle Policies** table shows the number of indices currently associated with each policy.
466
-
:::::
467
-
468
-
::::{tab-set}
469
-
:group: kibana-api
470
-
:::{tab-item} {{kib}}
471
-
:sync: kibana
472
-
To switch an index’s lifecycle policy:
473
-
474
-
1. Go to **Stack Management > Index Management**. In the **Indices** tab, search for and select the index that you that you want to switch to a new policy. You can use the **Lifecycle status** filter to narrow the list.
475
-
476
-
1. From the **Manage index** dropdown menu select **Remove lifecycle policy**. Confirm your choice and then the ILM policy is removed.
477
-
478
-
1. From the **Manage index** dropdown menu select **Add lifecycle policy**, and then select a new policy to apply.
479
-
480
-
:::
481
-
482
-
:::{tab-item} API
483
-
:sync: api
484
-
Use the {{es}} [remove policy](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-remove-policy) and [update settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings) APIs to switch an index’s lifecycle policy:
485
-
486
-
1. Remove the existing policy using the [remove policy API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-remove-policy). Target a data stream or alias to remove the policies of all its indices.
487
-
488
-
```console
489
-
POST logs-my_app-default/_ilm/remove
490
-
```
491
-
492
-
2. The remove policy API removes all {{ilm-init}} metadata from the index and doesn’t consider the index’s lifecycle status. This can leave indices in an undesired state.
493
-
494
-
For example, the [`forcemerge`](elasticsearch://reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md) action temporarily closes an index before reopening it. Removing an index’s {{ilm-init}} policy during a `forcemerge` can leave the index closed indefinitely.
495
-
496
-
After policy removal, use the [get index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get) to check an index’s state. Target a data stream or alias to get the state of all its indices.
497
-
498
-
```console
499
-
GET logs-my_app-default
500
-
```
501
-
502
-
You can then change the index as needed. For example, you can re-open any closed indices using the [open index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-open).
503
-
504
-
```console
505
-
POST logs-my_app-default/_open
506
-
```
507
-
508
-
3. Assign a new policy using the [update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings). Target a data stream or alias to assign a policy to all its indices.
509
-
510
-
::::::{warning}
511
-
* Don’t assign a new policy without first removing the existing policy. This can cause [phase execution](index-lifecycle.md#ilm-phase-execution) to silently fail.
0 commit comments