You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: manage-data/data-store/data-streams/time-series-data-stream-tsds.md
+25-27Lines changed: 25 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,33 +19,13 @@ Before setting up a time series data stream, make sure you're familiar with gene
19
19
20
20
## When to use a time series data stream [when-to-use-tsds]
21
21
22
-
Both a regular data stream and a time series data stream can store timestamped metrics data.
22
+
_Metrics_ consist of data point–timestamp pairs, identified by [dimension fields]() that can be used in aggregation queries. Both a regular data stream and a time series data stream can store metrics data.
23
23
24
24
Choose a time series data stream if you typically add metrics data to {{es}} in near real-time and in `@timestamp` order. For other timestamped data, such as logs or traces, use a [logs data stream](logs-data-stream.md) or [regular data stream](/manage-data/data-store/data-streams.md).
25
25
26
-
### Differences from a regular data stream [differences-from-regular-data-stream]
27
-
28
-
A time series data stream works like a regular data stream, with some key differences:
29
-
30
-
***Time series index mode:** The matching index template for a TSDS must include a `data_stream` object with `index.mode` set to `time_series`. This option enables most TSDS-related functionality.
31
-
***Required fields:** In a TSDS, each document contains:
32
-
* A `@timestamp` field
33
-
* One or more [dimension fields](#time-series-dimension), set with `time_series_dimension: true`
34
-
* One or more [metric fields](#time-series-metric) (not strictly required, but typical for a TSDS)
35
-
***Document IDs:** Time series documents use two IDs:
36
-
* An internal [`_tsid`](#tsid) metadata field, generated by {{es}} for each document in a TSDS and used for sorting and compression
37
-
* The document `_id`, a generated hash of the document's dimensions and `@timestamp` (custom `_id` values are not supported)
38
-
***Backing indices:** A TSDS uses [time-bound indices](/manage-data/data-store/data-streams/time-bound-tsds.md) to store data from the same time period in the same backing index.
39
-
***Dimension-based routing:** The matching index template for a TSDS must contain the `index.routing_path` index setting, which specifies dimensions for routing documents to shards.
40
-
***Sorting:** A TSDS uses internal [index sorting](elasticsearch://reference/elasticsearch/index-settings/sorting.md) to order shard segments by `_tsid` and `@timestamp`, for better compression. Time series data streams do not use `index.sort.*` settings.
41
-
***Synthetic source:** A TSDS uses [synthetic `_source`](elasticsearch://reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source), which has some [restrictions](elasticsearch://reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source-restrictions) and [modifications](elasticsearch://reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source-modifications).
26
+
To make sure a TSDS is right for your use case, review the list of [differences from a regular data stream](#differences-from-regular-data-stream) on this page.
42
27
43
-
44
-
## Query time series data
45
-
46
-
You can use the {{esql}} [`TS` command](elasticsearch://reference/query-languages/esql/commands/ts.md) to query time series data streams. The `TS` command is optimized for time series data. It also enables the use of aggregation functions that efficiently process metrics per time series, before aggregating results.
47
-
48
-
## Time series concepts [time-series]
28
+
## Time series overview [time-series]
49
29
50
30
A time series is a sequence of observations for a specific entity. Together, these observations let you track changes to the entity over time. For example, a time series can track:
51
31
@@ -91,7 +71,7 @@ You can also simplify dimension definitions by using [pass-through](elasticsearc
91
71
92
72
#### Metrics [time-series-metric]
93
73
94
-
Metrics are numeric measurements that change over time. Although metrics are not required, documents in a TSDS typically contain one or more metric fields.
74
+
Metrics are numeric measurements that change over time. Documents in a TSDS contain one or more metric fields.
95
75
96
76
:::{tip}
97
77
Metrics are expected to change (even if rarely or slowly), while dimensions generally remain constant.
@@ -110,9 +90,6 @@ The valid values for `time_series_metric` are `counter` and `gauge`:
110
90
`gauge`
111
91
: A metric that represents a single numeric that can arbitrarily increase or decrease. For example, a temperature or available disk space.
112
92
113
-
114
-
:::::
115
-
116
93
#### `_tsid` metadata field [tsid]
117
94
118
95
The `_tsid` is an automatically generated object containing the document’s dimensions. It's intended for internal {{es}} use, so in most cases you won't need to work with it.
@@ -122,6 +99,27 @@ The `_tsid` is an automatically generated object containing the document’s dim
122
99
- To retrieve the value of `_tsid`, use the fields parameter in a search. The `_tsid` is not included in get document responses.
123
100
- The format of the `_tsid` field is subject to change.
124
101
102
+
### Differences from a regular data stream [differences-from-regular-data-stream]
103
+
104
+
A time series data stream works like a regular data stream, with some key differences:
105
+
106
+
***Time series index mode:** The matching index template for a TSDS must include a `data_stream` object with `index.mode` set to `time_series`. This option enables most TSDS-related functionality.
107
+
***Required fields:** In a TSDS, each document contains:
108
+
* A `@timestamp` field
109
+
* One or more [dimension fields](#time-series-dimension), set with `time_series_dimension: true`
110
+
* One or more [metric fields](#time-series-metric) (not strictly required, but typical for a TSDS)
111
+
***Document IDs:** Time series documents use two IDs:
112
+
* An internal [`_tsid`](#tsid) metadata field, generated by {{es}} for each document in a TSDS and used for sorting and compression
113
+
* The document `_id`, a generated hash of the document's dimensions and `@timestamp` (custom `_id` values are not supported)
114
+
***Backing indices:** A TSDS uses [time-bound indices](/manage-data/data-store/data-streams/time-bound-tsds.md) to store data from the same time period in the same backing index.
115
+
***Dimension-based routing:** The matching index template for a TSDS must contain the `index.routing_path` index setting, which specifies dimensions for routing documents to shards.
116
+
***Sorting:** A TSDS uses internal [index sorting](elasticsearch://reference/elasticsearch/index-settings/sorting.md) to order shard segments by `_tsid` and `@timestamp`, for better compression. Time series data streams do not use `index.sort.*` settings.
117
+
***Synthetic source:** A TSDS uses [synthetic `_source`](elasticsearch://reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source), which has some [restrictions](elasticsearch://reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source-restrictions) and [modifications](elasticsearch://reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source-modifications).
118
+
119
+
## Query time series data
120
+
121
+
You can use the {{esql}} [`TS` command](elasticsearch://reference/query-languages/esql/commands/ts.md) to query time series data streams. The `TS` command is optimized for time series data. It also enables the use of aggregation functions that efficiently process metrics per time series, before aggregating results.
0 commit comments