Skip to content

Commit edb5347

Browse files
jrodewigdebadair
andauthored
[DOCS] Update ILM screenshots and tutorial (#68482) (#68585)
Changes: - Reworks the ILM tutorial to focus on the Elastic Agent and a built-in ILM policy - Updates several screenshots in the docs for the new ILM UI Co-authored-by: debadair <[email protected]>
1 parent f864bcf commit edb5347

9 files changed

+73
-118
lines changed

docs/reference/data-streams/set-up-a-data-stream.asciidoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,10 @@ While optional, we recommend you configure an <<set-up-lifecycle-policy,{ilm}
2121
indices.
2222

2323
In {kib}, open the menu and go to *Stack Management > Index Lifecycle Policies*.
24-
Click *Index Lifecycle Policies*.
24+
Click *Create policy*.
2525

2626
[role="screenshot"]
27-
image::images/ilm/create-policy.png[Index Lifecycle Policies page]
27+
image::images/ilm/create-policy.png[Create Policy page]
2828

2929
[%collapsible]
3030
.API example
Lines changed: 69 additions & 114 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,38 @@
11
[role="xpack"]
22

33
[[example-using-index-lifecycle-policy]]
4-
=== Tutorial: Manage {filebeat} time-based indices
4+
=== Tutorial: Customize built-in {ilm-init} policies
55
++++
6-
<titleabbrev>Manage {filebeat} time-based indices</titleabbrev>
6+
<titleabbrev>Customize built-in {ilm-init} policies</titleabbrev>
77
++++
88

9-
With {ilm} ({ilm-init}), you can create policies that perform actions automatically
10-
on indices as they age and grow. {ilm-init} policies help you to manage
11-
performance, resilience, and retention of your data during its lifecycle. This tutorial shows
12-
you how to use {kib}’s *Index Lifecycle Policies* to modify and create {ilm-init}
13-
policies. You can learn more about all of the actions, benefits, and lifecycle
14-
phases in the <<overview-index-lifecycle-management, {ilm-init} overview>>.
9+
{es} includes the following built-in {ilm-init} policies:
10+
11+
- `logs`
12+
- `metrics`
13+
- `synthetics`
14+
15+
The {agent} uses these policies to manage backing indices for its data streams.
16+
This tutorial shows you how to use {kib}’s **Index Lifecycle Policies** to
17+
customize these policies based on your application's performance, resilience,
18+
and retention requirements.
1519

1620

1721
[discrete]
1822
[[example-using-index-lifecycle-policy-scenario]]
1923
==== Scenario
2024

21-
You’re tasked with sending syslog files to an {es} cluster. This
22-
log data has the following data retention guidelines:
25+
You want to send log files to an {es} cluster so you can visualize and analyze
26+
the data. This data has the following retention requirements:
2327

24-
* Keep logs on hot data nodes for 30 days
25-
* Roll over to a new index if the size reaches 50GB
26-
* After 30 days:
27-
** Move the logs to warm data nodes
28-
** Set <<glossary-replica-shard, replica shards>> to 1
29-
** <<indices-forcemerge, Force merge>> multiple index segments to free up the space used by deleted documents
30-
* Delete logs after 90 days
28+
* When the write index reaches 50GB or is 30 days old, roll over to a new index.
29+
* After rollover, keep indices in the hot data tier for 30 days.
30+
* 30 days after rollover:
31+
** Move indices to the warm data tier.
32+
** Set <<glossary-replica-shard, replica shards>> to 1.
33+
** <<indices-forcemerge, Force merge>> multiple index segments to free up the
34+
space used by deleted documents.
35+
* Delete indices 90 days after rollover.
3136

3237

3338
[discrete]
@@ -36,130 +41,80 @@ log data has the following data retention guidelines:
3641

3742
To complete this tutorial, you'll need:
3843

39-
* An {es} cluster with hot and warm nodes configured for shard allocation
40-
awareness.
44+
* An {es} cluster with hot and warm data tiers.
4145

42-
** {ess}:
43-
Choose the Elastic Stack and then the {cloud}/ec-getting-started-profiles.html#ec-getting-started-profiles-hot-warm[hot-warm architecture] hardware profile.
46+
** {ess}:
47+
Elastic Stack deployments on the {ess} include a hot tier by default. To add a
48+
warm tier, edit your deployment and click **Add capacity** for the warm data
49+
tier.
50+
+
51+
[role="screenshot"]
52+
image::images/ilm/tutorial-ilm-ess-add-warm-data-tier.png[Add a warm data tier to your deployment]
4453

45-
** Self-managed cluster:
46-
Add node attributes as described for {ref}/shard-allocation-filtering.html[shard allocation filtering].
54+
** Self-managed cluster:
55+
Assign `data_hot` and `data_warm` roles to nodes as described in
56+
<<data-tiers>>.
4757
+
48-
For example, you can set this in your `elasticsearch.yml` for each data node:
58+
For example, include the `data_warm` node role in the `elasticsearch.yml` file
59+
of each node in the warm tier:
4960
+
5061
[source,yaml]
51-
--------------------------------------------------------------------------------
52-
node.attr.data: "warm"
53-
--------------------------------------------------------------------------------
62+
----
63+
node.roles: [ "data_warm" ]
64+
----
5465

55-
* A server with {filebeat} installed and configured to send logs to the `elasticsearch`
56-
output as described in the {filebeat-ref}/filebeat-installation-configuration.html[{filebeat} quick start].
66+
* A host with the {agent} installed and configured to send logs to your {es}
67+
cluster.
5768

5869
[discrete]
59-
[[example-using-index-lifecycle-policy-view-fb-ilm-policy]]
60-
==== View the {filebeat} {ilm-init} policy
70+
[[example-using-index-lifecycle-policy-view-ilm-policy]]
71+
==== View the policy
6172

62-
{filebeat} includes a default {ilm-init} policy that enables rollover. {ilm-init}
63-
is enabled automatically if you’re using the default `filebeat.yml` and index template.
73+
The {agent} uses data streams with an index pattern of `logs-*-*` to store log
74+
monitoring data. The built-in `logs` {ilm-init} policy automatically manages
75+
backing indices for these data streams.
6476

65-
To view the default policy in {kib}:
77+
To view the `logs` policy in {kib}:
6678

67-
. Go to Management and select *Index Lifecycle Policies*.
68-
. Search for _filebeat_
69-
. Select the _filebeat-version_ policy.
79+
. Open the menu and go to **Stack Management > Index Lifecycle Policies**.
80+
. Select the `logs` policy.
7081

71-
This policy initiates the rollover action when the index size reaches 50GB or
72-
becomes 30 days old.
73-
74-
[role="screenshot"]
75-
image::images/ilm/tutorial-ilm-hotphaserollover-default.png["Default policy"]
82+
The `logs` policy uses the recommended rollover defaults: Start writing to a new
83+
index when the current write index reaches 50GB or becomes 30 days old.
7684

85+
To view or change the rollover settings, click **Advanced settings** for the hot
86+
phase. Then disable **Use recommended defaults** to display the rollover
87+
settings.
7788

78-
[discrete]
79-
==== Modify the policy
80-
81-
The default policy is enough to prevent the creation of many tiny daily indices.
82-
You can modify the policy to meet more complex requirements.
83-
84-
. Activate the warm phase.
85-
+
86-
--
8789
[role="screenshot"]
88-
image::images/ilm/tutorial-ilm-modify-default-warm-phase-rollover.png["Modify to add warm phase"]
89-
90-
.. Set one of the following options to control when the index moves to the warm phase:
91-
92-
*** Provide a value for *Timing for warm phase*. Setting this to *15* keeps the
93-
indices on hot nodes for a range of 15-45 days, depending on when the initial
94-
rollover occurred.
95-
96-
*** Enable *Move to warm phase on rollover*. The index might move to the warm phase
97-
more quickly than intended if it reaches the *Maximum index size* before the
98-
the *Maximum age*.
99-
100-
.. In the *Select a node attribute to control shard allocation* dropdown, select
101-
*data:warm(2)* to migrate shards to warm data nodes.
102-
103-
.. Change *Number of replicas* to *1*.
104-
105-
.. Enable *Force merge data* and set *Number of segments* to *1*.
106-
107-
NOTE: When rollover is enabled in the hot phase, action timing in the other phases
108-
is based on the rollover date.
109-
--
110-
111-
. Activate the delete phase and set *Timing for delete phase* to *90* days.
112-
+
113-
[role="screenshot"]
114-
image::images/ilm/tutorial-ilm-delete-rollover.png["Add a delete phase"]
90+
image::images/ilm/tutorial-ilm-hotphaserollover-default.png[View rollover defaults]
11591

11692
[discrete]
117-
==== Create a custom policy
118-
119-
If meeting a specific retention time period is most important, you can create a
120-
custom policy. For this option, you use {filebeat} daily indices without
121-
rollover.
93+
[[ilm-ex-modify-policy]]
94+
==== Modify the policy
12295

123-
To create a custom policy:
96+
The default `logs` policy is designed to prevent the creation of many tiny daily
97+
indices. You can modify the policy to meet your performance requirements and
98+
manage resource usage.
12499

125-
. Go to Management and select *Index Lifecycle Policies*.
126-
. Click *Create policy*.
127-
. Activate the warm phase and configure it as follows:
100+
. Activate the warm phase and click **Advanced settings**.
128101
+
129102
--
130-
**Timing for warm phase**: 30 days from index creation
131-
132-
**Node attribute**: `data:warm`
103+
.. Disable **Move to warm phase on rollover**. Set **Timing for warm phase** to
104+
**30 days from rollover**.
133105

134-
**Number of replicas**: 1
106+
.. Enable **Set replicas** and change **Number of replicas** to **1**.
135107

136-
**Force merge data**: enable
137-
138-
**Number of segments**: 1
108+
.. Enable **Force merge data** and set **Number of segments** to **1**.
139109

140110
[role="screenshot"]
141-
image::images/ilm/tutorial-ilm-custom-policy.png["Modify the custom policy to add a warm phase"]
111+
image::images/ilm/tutorial-ilm-modify-default-warm-phase-rollover.png[Add a warm phase with custom settings]
142112
--
143113

144-
. Activate the delete phase and set the timing to 90 days.
114+
. Activate the delete phase and set **Timing for delete phase** to **90 days from
115+
rollover**.
145116
+
146117
[role="screenshot"]
147-
image::images/ilm/tutorial-ilm-delete-phase-creation.png["Delete phase"]
148-
149-
To configure the index to use the new policy:
118+
image::images/ilm/tutorial-ilm-delete-rollover.png[Add a delete phase]
150119

151-
. Go to Management and select *Index Lifecycle Policies*.
152-
. Find your {ilm-init} policy and click its *Actions* link.
153-
. Choose *Add policy to index template*.
154-
. Select your {filebeat} index template name from the *Index template* list. For example, `filebeat-7.5.x`.
155-
. Click *Add Policy* to save the changes.
156-
+
157-
NOTE: If you initially used the default {filebeat} {ilm-init} policy, you will
158-
see a notice that the template already has a policy associated with it. Confirm
159-
that you want to overwrite that configuration.
160-
161-
When you change the policy associated with the index template, the active
162-
index will continue to use the policy it was associated with at index creation
163-
unless you manually update it. The next new index will use the updated policy.
164-
For more reasons that your {ilm-init} policy changes might be delayed, see
165-
<<update-lifecycle-policy, Update Lifecycle Policy>>.
120+
. Click **Save Policy**.

docs/reference/ilm/ilm-tutorial.asciidoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,10 +58,10 @@ Note that this value is relative to the rollover time, not the index creation ti
5858
You can create the policy through {kib} or with the
5959
<<ilm-put-lifecycle, put policy>> API.
6060
To create the policy from {kib}, open the menu and go to *Stack Management >
61-
Index Lifecycle Policies*. Click *Index Lifecycle Policies*.
61+
Index Lifecycle Policies*. Click *Create policy*.
6262

6363
[role="screenshot"]
64-
image:images/ilm/create-policy.png[]
64+
image::images/ilm/create-policy.png[Create policy page]
6565

6666
.API example
6767
[%collapsible]
-76.5 KB
Loading
-48.5 KB
Loading
-94.9 KB
Loading
60.4 KB
Loading
-218 KB
Loading
-353 KB
Loading

0 commit comments

Comments
 (0)