7
7
++++
8
8
9
9
When you continuously index timestamped documents into {es},
10
- you typically use a <<data-streams, data stream>> so you can periodically roll over to a
10
+ you typically use a <<data-streams,data stream>> so you can periodically <<index-rollover, roll over>> to a
11
11
new index.
12
- This enables you to implement a hot-warm-cold architecture to meet your performance
12
+ This enables you to implement a <<data-tiers, hot-warm-cold architecture>> to meet your performance
13
13
requirements for your newest data, control costs over time, enforce retention policies,
14
14
and still get the most out of your data.
15
15
16
- TIP: Data streams are best suited for
16
+ TIP: <<data-streams, Data streams>> are best suited for
17
17
<<data-streams-append-only,append-only>> use cases. If you need to update or delete existing time
18
18
series data, you can perform update or delete operations directly on the data stream backing index.
19
19
If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may
20
- want to use an index alias with a write index instead. You can still use ILM to manage and rollover
20
+ want to use an index alias with a write index instead. You can still use <<index-lifecycle-management, ILM>> to manage and <<index- rollover,roll over>>
21
21
the alias's indices. Skip to <<manage-time-series-data-without-data-streams>>.
22
22
23
+ [discrete]
24
+ [[manage-time-series-data-with-data-streams]]
25
+ === Manage time series data with data streams
26
+
23
27
To automate rollover and management of a data stream with {ilm-init}, you:
24
28
25
29
. <<ilm-gs-create-policy, Create a lifecycle policy>> that defines the appropriate
26
- phases and actions.
27
- . <<ilm-gs-apply-policy, Create an index template>> to create the data stream and
30
+ <<ilm-index-lifecycle, phases>> and <<ilm- actions,actions>> .
31
+ . <<ilm-gs-apply-policy, Create an index template>> to <<ilm-gs- create-the-data-stream,create the data stream>> and
28
32
apply the ILM policy and the indices settings and mappings configurations for the backing
29
33
indices.
30
34
. <<ilm-gs-check-progress, Verify indices are moving through the lifecycle phases>>
31
35
as expected.
32
36
33
- For an introduction to rolling indices, see <<index-rollover>>.
34
-
35
37
IMPORTANT: When you enable {ilm} for {beats} or the {ls} {es} output plugin,
36
38
lifecycle policies are set up automatically.
37
39
You do not need to take any other actions.
@@ -41,7 +43,7 @@ or the {ilm-init} APIs.
41
43
42
44
[discrete]
43
45
[[ilm-gs-create-policy]]
44
- === Create a lifecycle policy
46
+ ==== Create a lifecycle policy
45
47
46
48
A lifecycle policy specifies the phases in the index lifecycle
47
49
and the actions to perform in each phase. A lifecycle can have up to five phases:
@@ -101,7 +103,7 @@ PUT _ilm/policy/timeseries_policy
101
103
102
104
[discrete]
103
105
[[ilm-gs-apply-policy]]
104
- === Create an index template to create the data stream and apply the lifecycle policy
106
+ ==== Create an index template to create the data stream and apply the lifecycle policy
105
107
106
108
To set up a data stream, first create an index template to specify the lifecycle policy. Because
107
109
the template is for a data stream, it must also include a `data_stream` definition.
@@ -148,7 +150,7 @@ PUT _index_template/timeseries_template
148
150
149
151
[discrete]
150
152
[[ilm-gs-create-the-data-stream]]
151
- === Create the data stream
153
+ ==== Create the data stream
152
154
153
155
To get things started, index a document into the name or wildcard pattern defined
154
156
in the `index_patterns` of the <<index-templates,index template>>. As long
@@ -184,12 +186,12 @@ stream's write index.
184
186
This process repeats each time a rollover condition is met.
185
187
You can search across all of the data stream's backing indices, managed by the `timeseries_policy`,
186
188
with the `timeseries` data stream name.
187
- Write operations are routed to the current write index. Read operations will be handled by all
188
- backing indices.
189
+ Write operations should be sent to the data stream name, which will route them to its current write index.
190
+ Read operations against the data stream will be handled by all its backing indices.
189
191
190
192
[discrete]
191
193
[[ilm-gs-check-progress]]
192
- === Check lifecycle progress
194
+ ==== Check lifecycle progress
193
195
194
196
To get status information for managed indices, you use the {ilm-init} explain API.
195
197
This lets you find out things like:
@@ -300,7 +302,7 @@ as expected.
300
302
301
303
[discrete]
302
304
[[ilm-gs-alias-apply-policy]]
303
- === Create an index template to apply the lifecycle policy
305
+ ==== Create an index template to apply the lifecycle policy
304
306
305
307
To automatically apply a lifecycle policy to the new write index on rollover,
306
308
specify the policy in the index template used to create new indices.
@@ -358,7 +360,7 @@ DELETE _index_template/timeseries_template
358
360
359
361
[discrete]
360
362
[[ilm-gs-alias-bootstrap]]
361
- === Bootstrap the initial time series index with a write index alias
363
+ ==== Bootstrap the initial time series index with a write index alias
362
364
363
365
To get things started, you need to bootstrap an initial index and
364
366
designate it as the write index for the rollover alias specified in your index template.
@@ -389,11 +391,11 @@ This matches the `timeseries-*` pattern, so the settings from `timeseries_templa
389
391
390
392
This process repeats each time rollover conditions are met.
391
393
You can search across all of the indices managed by the `timeseries_policy` with the `timeseries` alias.
392
- Write operations are routed to the current write index.
394
+ Write operations should be sent towards the alias, which will route them to its current write index.
393
395
394
396
[discrete]
395
397
[[ilm-gs-alias-check-progress]]
396
- === Check lifecycle progress
398
+ ==== Check lifecycle progress
397
399
398
400
Retrieving the status information for managed indices is very similar to the data stream case.
399
401
See the data stream <<ilm-gs-check-progress, check progress section>> for more information.
0 commit comments