Skip to content

Commit f1ae88c

Browse files
authored
Simplify tsds getting started (elastic#108090) (elastic#108134)
* By only using index templates. * Removing the usage and mentioning of index.routing_path index setting.
1 parent 9b9c9ca commit f1ae88c

File tree

3 files changed

+41
-103
lines changed

3 files changed

+41
-103
lines changed

docs/reference/data-streams/downsampling.asciidoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ and it inherits its settings (for example, the number of shards and replicas).
190190
supported.
191191

192192
* The downsampling configuration is extracted from the time series data stream
193-
<<tsds-create-mappings-component-template,index mapping>>. The only additional
193+
<<create-tsds-index-template,index mapping>>. The only additional
194194
required setting is the downsampling `fixed_interval`.
195195

196196
[discrete]

docs/reference/data-streams/set-up-tsds.asciidoc

Lines changed: 36 additions & 98 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,6 @@ To set up a <<tsds,time series data stream (TSDS)>>, follow these steps:
88

99
. Check the <<tsds-prereqs,prerequisites>>.
1010
. <<tsds-ilm-policy>>.
11-
. <<tsds-create-mappings-component-template>>.
12-
. <<tsds-create-index-settings-component-template>>.
1311
. <<create-tsds-index-template>>.
1412
. <<create-tsds>>.
1513
. <<secure-tsds>>.
@@ -109,34 +107,52 @@ PUT _ilm/policy/my-weather-sensor-lifecycle-policy
109107
----
110108

111109
[discrete]
112-
[[tsds-create-mappings-component-template]]
113-
==== Create a mappings component template
110+
[[create-tsds-index-template]]
111+
==== Create an index template
112+
113+
To setup a TSDS create an index template with the following details:
114+
115+
* One or more index patterns that match the TSDS's name. We recommend
116+
using our {fleet-guide}/data-streams.html#data-streams-naming-scheme[data stream
117+
naming scheme].
118+
119+
* Enable data streams.
120+
121+
* Specify a mapping that defines your dimensions and metrics:
114122

115-
A TSDS requires a matching index template. In most cases, you compose this index
116-
template using one or more component templates. You typically use separate
117-
component templates for mappings and index settings. This lets you reuse the
118-
component templates in multiple index templates.
123+
** One or more <<time-series-dimension,dimension fields>> with a `time_series_dimension` value of `true`.
124+
At least one of these dimensions must be a plain `keyword` field.
119125

120-
For a TSDS, the mappings component template must include mappings for:
126+
** One or more <<time-series-metric,metric fields>>, marked using the `time_series_metric` mapping parameter.
121127

122-
* One or more <<time-series-dimension,dimension fields>> with a
123-
`time_series_dimension` value of `true`. At least one of these dimensions must
124-
be a plain `keyword` field.
128+
** Optional: A `date` or `date_nanos` mapping for the `@timestamp` field. If you don’t specify a mapping,
129+
Elasticsearch maps `@timestamp` as a `date` field with default options.
125130

126-
Optionally, the template can also include mappings for:
131+
* Define index settings:
127132

128-
* One or more <<time-series-metric,metric fields>>, marked using the
129-
`time_series_metric` mapping parameter.
133+
** Set `index.mode` setting to `time_series`.
130134

131-
* A `date` or `date_nanos` mapping for the `@timestamp` field. If you don’t
132-
specify a mapping, Elasticsearch maps `@timestamp` as a `date` field with
133-
default options.
135+
** Your lifecycle policy in the `index.lifecycle.name` index setting.
136+
137+
** Optional: Other index settings, such as <<dynamic-index-number-of-replicas,`index.number_of_replicas`>>,
138+
for your TSDS's backing indices.
139+
140+
* A priority higher than `200` to avoid collisions with built-in templates.
141+
See <<avoid-index-pattern-collisions>>.
142+
143+
* Optional: Component templates containing your mappings and other index settings.
134144

135145
[source,console]
136146
----
137-
PUT _component_template/my-weather-sensor-mappings
147+
PUT _index_template/my-weather-sensor-index-template
138148
{
149+
"index_patterns": ["metrics-weather_sensors-*"],
150+
"data_stream": { },
139151
"template": {
152+
"settings": {
153+
"index.mode": "time_series",
154+
"index.lifecycle.name": "my-lifecycle-policy"
155+
},
140156
"mappings": {
141157
"properties": {
142158
"sensor_id": {
@@ -156,88 +172,11 @@ PUT _component_template/my-weather-sensor-mappings
156172
"time_series_metric": "gauge"
157173
},
158174
"@timestamp": {
159-
"type": "date",
160-
"format": "strict_date_optional_time"
175+
"type": "date"
161176
}
162177
}
163178
}
164179
},
165-
"_meta": {
166-
"description": "Mappings for weather sensor data"
167-
}
168-
}
169-
----
170-
// TEST[continued]
171-
172-
[discrete]
173-
[[tsds-create-index-settings-component-template]]
174-
==== Create an index settings component template
175-
176-
Optionally, the index settings component template for a TSDS can include:
177-
178-
* Your lifecycle policy in the `index.lifecycle.name` index setting.
179-
* Other index settings, such as <<dynamic-index-number-of-replicas,`index.number_of_replicas`>>, for your TSDS's
180-
backing indices.
181-
182-
IMPORTANT: Don't specify the `index.routing_path` index setting in a component
183-
template. If you choose, you can configure `index.routing_path` directly in the
184-
index template, as shown in the following step.
185-
186-
[source,console]
187-
----
188-
PUT _component_template/my-weather-sensor-settings
189-
{
190-
"template": {
191-
"settings": {
192-
"index.lifecycle.name": "my-lifecycle-policy"
193-
}
194-
},
195-
"_meta": {
196-
"description": "Index settings for weather sensor data"
197-
}
198-
}
199-
----
200-
// TEST[continued]
201-
202-
[discrete]
203-
[[create-tsds-index-template]]
204-
==== Create an index template
205-
206-
Use your component templates to create an index template. In the index template,
207-
specify:
208-
209-
* One or more index patterns that match the TSDS's name. We recommend
210-
using our {fleet-guide}/data-streams.html#data-streams-naming-scheme[data stream
211-
naming scheme].
212-
213-
* That the template is data stream enabled.
214-
215-
* An `index.mode` object set to `time_series`.
216-
217-
* Optional: The `index.routing_path` index setting. The setting value should
218-
only match plain `keyword` dimension fields and should be set directly in the
219-
index template. When not defined explicitly, the `index.routing_path` setting is
220-
generated from the mapping using all mappings that are set with
221-
`time_series_dimension` set to `true`.
222-
223-
* The component templates containing your mappings and other index settings.
224-
225-
* A priority higher than `200` to avoid collisions with built-in templates.
226-
See <<avoid-index-pattern-collisions>>.
227-
228-
[source,console]
229-
----
230-
PUT _index_template/my-weather-sensor-index-template
231-
{
232-
"index_patterns": ["metrics-weather_sensors-*"],
233-
"data_stream": { },
234-
"template": {
235-
"settings": {
236-
"index.mode": "time_series",
237-
"index.routing_path": [ "sensor_id", "location" ]
238-
}
239-
},
240-
"composed_of": [ "my-weather-sensor-mappings", "my-weather-sensor-settings" ],
241180
"priority": 500,
242181
"_meta": {
243182
"description": "Template for my weather sensor data"
@@ -251,7 +190,6 @@ PUT _index_template/my-weather-sensor-index-template
251190
----
252191
DELETE _data_stream/*
253192
DELETE _index_template/*
254-
DELETE _component_template/my-*
255193
DELETE _ilm/policy/my-weather-sensor-lifecycle-policy
256194
----
257195
// TEST[continued]

docs/reference/data-streams/tsds.asciidoc

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -226,9 +226,9 @@ to not be actively written anymore in order to provide good performance. These a
226226
- <<ilm-readonly>>
227227
- <<ilm-searchable-snapshot>>
228228
- <<ilm-shrink>>
229-
{ilm-cap} will **not** proceed with executing these actions until the upper time-bound
230-
for accepting writes, represented by the <<index-time-series-end-time,`index.time_series.end_time`>>
231-
index setting, has lapsed.
229+
{ilm-cap} will **not** proceed with executing these actions until the upper time-bound
230+
for accepting writes, represented by the <<index-time-series-end-time,`index.time_series.end_time`>>
231+
index setting, has lapsed.
232232

233233
If no backing index can accept a document's `@timestamp` value, {es} rejects the
234234
document.
@@ -290,7 +290,7 @@ created the initial backing index has:
290290

291291
Only data that falls inside that range can be indexed.
292292

293-
In our <<tsds-create-index-settings-component-template,TSDS example>>,
293+
In our <<create-tsds-index-template,TSDS example>>,
294294
`index.look_ahead_time` is set to three hours, so only documents with a
295295
`@timestamp` value that is within three hours previous or subsequent to the
296296
present time are accepted for indexing.

0 commit comments

Comments
 (0)