You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/datatiers.asciidoc
+29-29Lines changed: 29 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,22 +2,22 @@
2
2
[[data-tiers]]
3
3
== Data tiers
4
4
5
-
A _data tier_ is a collection of <<modules-node,nodes>> within a cluster that share the same
6
-
<<node-roles,data node role>>, and a hardware profile that's appropriately sized for the role. Elastic recommends that nodes in the same tier share the same
7
-
hardware profile to avoid <<hotspotting,hot spotting>>.
5
+
A _data tier_ is a collection of <<modules-node,nodes>> within a cluster that share the same
6
+
<<node-roles,data node role>>, and a hardware profile that's appropriately sized for the role. Elastic recommends that nodes in the same tier share the same
7
+
hardware profile to avoid <<hotspotting,hot spotting>>.
8
8
9
9
The data tiers that you use, and the way that you use them, depends on the data's <<data-management,category>>.
10
10
11
11
The following data tiers are can be used with each data category:
12
12
13
13
Content data:
14
14
15
-
* <<content-tier,Content tier>> nodes handle the indexing and query load for non-timeseries
15
+
* <<content-tier,Content tier>> nodes handle the indexing and query load for non-timeseries
16
16
indices, such as a product catalog.
17
17
18
18
Time series data:
19
19
20
-
* <<hot-tier,Hot tier>> nodes handle the indexing load for time series data,
20
+
* <<hot-tier,Hot tier>> nodes handle the indexing load for time series data,
21
21
such as logs or metrics. They hold your most recent, most-frequently-accessed data.
22
22
* <<warm-tier,Warm tier>> nodes hold time series data that is accessed less-frequently
23
23
and rarely needs to be updated.
@@ -27,30 +27,30 @@ infrequently and not normally updated. To save space, you can keep
27
27
<<ilm-searchable-snapshot,{search-snaps}>> on the cold tier. These fully mounted
28
28
indices eliminate the need for replicas, reducing required disk space by
29
29
approximately 50% compared to the regular indices.
30
-
* <<frozen-tier,Frozen tier>> nodes hold time series data that is accessed
30
+
* <<frozen-tier,Frozen tier>> nodes hold time series data that is accessed
31
31
rarely and never updated. The frozen tier stores <<partially-mounted,partially
32
32
mounted indices>> of <<ilm-searchable-snapshot,{search-snaps}>> exclusively.
33
33
This extends the storage capacity even further — by up to 20 times compared to
34
-
the warm tier.
34
+
the warm tier.
35
35
36
-
TIP: The performance of an {es} node is often limited by the performance of the underlying storage and hardware profile.
37
-
For example hardware profiles, refer to Elastic Cloud's {cloud}/ec-reference-hardware.html[instance configurations].
36
+
TIP: The performance of an {es} node is often limited by the performance of the underlying storage and hardware profile.
37
+
For example hardware profiles, refer to Elastic Cloud's {cloud}/ec-reference-hardware.html[instance configurations].
38
38
Review our recommendations for optimizing your storage for <<indexing-use-faster-hardware,indexing>> and <<search-use-faster-hardware,search>>.
39
39
40
-
IMPORTANT: {es} generally expects nodes within a data tier to share the same
41
-
hardware profile. Variations not following this recommendation should be
40
+
IMPORTANT: {es} generally expects nodes within a data tier to share the same hardware profile
41
+
and have the same disk capacity. Variations not following this recommendation should be
42
42
carefully architected to avoid <<hotspotting,hot spotting>>.
43
43
44
44
The way data tiers are used often depends on the data's category:
45
45
46
46
- Content data remains on the <<content-tier,content tier>> for its entire
47
-
data lifecycle.
47
+
data lifecycle.
48
48
49
-
- Time series data may progress through the
50
-
descending temperature data tiers (hot, warm, cold, and frozen) according to your
51
-
performance, resiliency, and data retention requirements.
52
-
+
53
-
You can automate these lifecycle transitions using the <<data-streams,data stream lifecycle>>, or custom <<index-lifecycle-management,{ilm}>>.
49
+
- Time series data may progress through the
50
+
descending temperature data tiers (hot, warm, cold, and frozen) according to your
51
+
performance, resiliency, and data retention requirements.
52
+
+
53
+
You can automate these lifecycle transitions using the <<data-streams,data stream lifecycle>>, or custom <<index-lifecycle-management,{ilm}>>.
54
54
55
55
[discrete]
56
56
[[available-tier]]
@@ -75,9 +75,9 @@ While they are also responsible for indexing, content data is generally not inge
75
75
as time series data such as logs and metrics. From a resiliency perspective the indices in this
76
76
tier should be configured to use one or more replicas.
77
77
78
-
The content tier is required and is often deployed within the same node
78
+
The content tier is required and is often deployed within the same node
79
79
grouping as the hot tier. System indices and other indices that aren't part
80
-
of a data stream are automatically allocated to the content tier.
80
+
of a data stream are automatically allocated to the content tier.
81
81
// end::content-tier[]
82
82
83
83
[discrete]
@@ -215,26 +215,26 @@ When {es} creates an index as part of a <<data-streams, data stream>>,
215
215
by default {es} sets the `_tier_preference`
216
216
to `data_hot` to automatically allocate the index shards to the hot tier.
217
217
218
-
At the time of index creation, you can override the default setting by explicitly setting
218
+
At the time of index creation, you can override the default setting by explicitly setting
219
219
the preferred value in one of two ways:
220
220
221
221
- Using an <<index-templates,index template>>. Refer to <<getting-started-index-lifecycle-management,Automate rollover with ILM>> for details.
222
-
- Within the <<indices-create-index,create index>> request body.
222
+
- Within the <<indices-create-index,create index>> request body.
223
223
224
-
You can override this
225
-
setting after index creation by <<indices-update-settings,updating the index setting>> to the preferred
226
-
value.
224
+
You can override this
225
+
setting after index creation by <<indices-update-settings,updating the index setting>> to the preferred
226
+
value.
227
227
228
228
This setting also accepts multiple tiers in order of preference. This prevents indices from remaining unallocated if no nodes are available in the preferred tier. For example, when {ilm} migrates an index to the cold phase, it sets the index `_tier_preference` to `data_cold,data_warm,data_hot`.
229
229
230
-
To remove the data tier preference
231
-
setting, set the `_tier_preference` value to `null`. This allows the index to allocate to any data node within the cluster. Setting the `_tier_preference` to `null` does not restore the default value. Note that, in the case of managed indices, a <<ilm-migrate,migrate>> action might apply a new value in its place.
230
+
To remove the data tier preference
231
+
setting, set the `_tier_preference` value to `null`. This allows the index to allocate to any data node within the cluster. Setting the `_tier_preference` to `null` does not restore the default value. Note that, in the case of managed indices, a <<ilm-migrate,migrate>> action might apply a new value in its place.
232
232
233
233
[discrete]
234
234
[[data-tier-allocation-value]]
235
235
==== Determine the current data tier preference
236
236
237
-
You can check an existing index's data tier preference by <<indices-get-settings,polling its
237
+
You can check an existing index's data tier preference by <<indices-get-settings,polling its
238
238
settings>> for `index.routing.allocation.include._tier_preference`:
239
239
240
240
[source,console]
@@ -247,8 +247,8 @@ GET /my-index-000001/_settings?filter_path=*.settings.index.routing.allocation.i
247
247
[[data-tier-allocation-troubleshooting]]
248
248
==== Troubleshooting
249
249
250
-
The `_tier_preference` setting might conflict with other allocation settings. This conflict might prevent the shard from allocating. A conflict might occur when a cluster has not yet been completely <<troubleshoot-migrate-to-tiers,migrated
251
-
to data tiers>>.
250
+
The `_tier_preference` setting might conflict with other allocation settings. This conflict might prevent the shard from allocating. A conflict might occur when a cluster has not yet been completely <<troubleshoot-migrate-to-tiers,migrated
251
+
to data tiers>>.
252
252
253
253
This setting will not unallocate a currently allocated shard, but might prevent it from migrating from its current location to its designated data tier. To troubleshoot, call the <<cluster-allocation-explain,cluster allocation explain API>> and specify the suspected problematic shard.
0 commit comments