|
| 1 | +[[passthrough]] |
| 2 | +=== Pass-through object field type |
| 3 | +++++ |
| 4 | +<titleabbrev>Pass-through object</titleabbrev> |
| 5 | +++++ |
| 6 | + |
| 7 | +Pass-through objects extend the functionality of <<object, objects>> by allowing to access |
| 8 | +their subfields without including the name of the pass-through object as prefix. For instance: |
| 9 | + |
| 10 | +[source,console] |
| 11 | +-------------------------------------------------- |
| 12 | +PUT my-index-000001 |
| 13 | +{ |
| 14 | + "mappings": { |
| 15 | + "properties": { |
| 16 | + "attributes": { |
| 17 | + "type": "passthrough", <1> |
| 18 | + "priority": 10, |
| 19 | + "properties": { |
| 20 | + "id": { |
| 21 | + "type": "keyword" |
| 22 | + } |
| 23 | + } |
| 24 | + } |
| 25 | + } |
| 26 | + } |
| 27 | +} |
| 28 | +
|
| 29 | +PUT my-index-000001/_doc/1 |
| 30 | +{ |
| 31 | + "attributes" : { <2> |
| 32 | + "id": "foo", |
| 33 | + "zone": 10 |
| 34 | + } |
| 35 | +} |
| 36 | +
|
| 37 | +GET my-index-000001/_search |
| 38 | +{ |
| 39 | + "query": { |
| 40 | + "bool": { |
| 41 | + "must": [ |
| 42 | + { "match": { "id": "foo" }}, <3> |
| 43 | + { "match": { "zone": 10 }} |
| 44 | + ] |
| 45 | + } |
| 46 | + } |
| 47 | +} |
| 48 | +
|
| 49 | +GET my-index-000001/_search |
| 50 | +{ |
| 51 | + "query": { |
| 52 | + "bool": { |
| 53 | + "must": [ |
| 54 | + { "match": { "attributes.id": "foo" }}, <4> |
| 55 | + { "match": { "attributes.zone": 10 }} |
| 56 | + ] |
| 57 | + } |
| 58 | + } |
| 59 | +} |
| 60 | +
|
| 61 | +-------------------------------------------------- |
| 62 | + |
| 63 | +<1> An object is defined as pass-through. Its priority (required) is used for conflict resolution. |
| 64 | +<2> Object contents get indexed as usual, including dynamic mappings. |
| 65 | +<3> Sub-fields can be referenced in queries as if they're defined at the root level. |
| 66 | +<4> Sub-fields can also be referenced including the object name as prefix. |
| 67 | + |
| 68 | +[[passthrough-conflicts]] |
| 69 | +==== Conflict resolution |
| 70 | + |
| 71 | +It's possible for conflicting names to arise, for fields that are defined within different scopes: |
| 72 | + |
| 73 | + 1. A pass-through object is defined next to a field that has the same name as one of the pass-through object |
| 74 | + sub-fields, e.g. |
| 75 | + |
| 76 | +[source,console] |
| 77 | +-------------------------------------------------- |
| 78 | +PUT my-index-000001/_doc/1 |
| 79 | +{ |
| 80 | + "attributes" : { |
| 81 | + "id": "foo" |
| 82 | + }, |
| 83 | + "id": "bar" |
| 84 | +} |
| 85 | +-------------------------------------------------- |
| 86 | + |
| 87 | + In this case, references to `id` point to the field at the root level, while field `attributes.id` |
| 88 | + can only be accessed using the full path. |
| 89 | + |
| 90 | + 1. Two (or more) pass-through objects are defined within the same object and contain fields with the same name, e.g. |
| 91 | + |
| 92 | +[source,console] |
| 93 | +-------------------------------------------------- |
| 94 | +PUT my-index-000002 |
| 95 | +{ |
| 96 | + "mappings": { |
| 97 | + "properties": { |
| 98 | + "attributes": { |
| 99 | + "type": "passthrough", |
| 100 | + "priority": 10, |
| 101 | + "properties": { |
| 102 | + "id": { |
| 103 | + "type": "keyword" |
| 104 | + } |
| 105 | + } |
| 106 | + }, |
| 107 | + "resource.attributes": { |
| 108 | + "type": "passthrough", |
| 109 | + "priority": 20, |
| 110 | + "properties": { |
| 111 | + "id": { |
| 112 | + "type": "keyword" |
| 113 | + } |
| 114 | + } |
| 115 | + } |
| 116 | + } |
| 117 | + } |
| 118 | +} |
| 119 | +-------------------------------------------------- |
| 120 | + |
| 121 | +In this case, param `priority` is used for conflict resolution, with the higher values taking precedence. In the |
| 122 | +example above, `resource.attributes` has higher priority than `attributes`, so references to `id` point to the field |
| 123 | +within `resource.attributes`. `attributes.id` can still be accessed using its full path. |
| 124 | + |
| 125 | +[[passthrough-dimensions]] |
| 126 | +==== Defining sub-fields as time-series dimensions |
| 127 | + |
| 128 | +It is possible to configure a pass-through field as a container for <<time-series-dimension,time-series dimensions>>. |
| 129 | +In this case, all sub-fields get annotated with the same parameter under the covers, and they're also |
| 130 | +included in <<dimension-based-routing, routing path>> and <<tsid, tsid>> calculations, thus simplifying |
| 131 | +the <<tsds,TSDS>> setup: |
| 132 | + |
| 133 | +[source,console] |
| 134 | +-------------------------------------------------- |
| 135 | +PUT _index_template/my-metrics |
| 136 | +{ |
| 137 | + "index_patterns": ["metrics-mymetrics-*"], |
| 138 | + "priority": 200, |
| 139 | + "data_stream": { }, |
| 140 | + "template": { |
| 141 | + "settings": { |
| 142 | + "index.mode": "time_series" |
| 143 | + }, |
| 144 | + "mappings": { |
| 145 | + "properties": { |
| 146 | + "attributes": { |
| 147 | + "type": "passthrough", |
| 148 | + "priority": 10, |
| 149 | + "time_series_dimension": true, |
| 150 | + "properties": { |
| 151 | + "host.name": { |
| 152 | + "type": "keyword" |
| 153 | + } |
| 154 | + } |
| 155 | + }, |
| 156 | + "cpu": { |
| 157 | + "type": "integer", |
| 158 | + "time_series_metric": "counter" |
| 159 | + } |
| 160 | + } |
| 161 | + } |
| 162 | + } |
| 163 | +} |
| 164 | +
|
| 165 | +POST metrics-mymetrics-test/_doc |
| 166 | +{ |
| 167 | + "@timestamp": "2020-01-01T00:00:00.000Z", |
| 168 | + "attributes" : { |
| 169 | + "host.name": "foo", |
| 170 | + "zone": "bar" |
| 171 | + }, |
| 172 | + "cpu": 10 |
| 173 | +} |
| 174 | +-------------------------------------------------- |
| 175 | +// TEST[skip: The @timestamp value won't match an accepted range in the TSDS] |
| 176 | + |
| 177 | +In the example above, `attributes` is defined as a dimension container. Its sub-fields `host.name` (static) and `zone` |
| 178 | +(dynamic) get included in the routing path and tsid, and can be referenced in queries without the `attributes.` prefix. |
| 179 | + |
| 180 | +[[passthrough-flattening]] |
| 181 | +==== Sub-field auto-flattening |
| 182 | + |
| 183 | +Pass-through fields apply <<subobjects-auto-flattening, auto-flattening>> to sub-fields by default, to reduce dynamic |
| 184 | +mapping conflicts. As a consequence, no sub-object definitions are allowed within pass-through fields. |
| 185 | + |
| 186 | +[[passthrough-params]] |
| 187 | +==== Parameters for `passthrough` fields |
| 188 | + |
| 189 | +The following parameters are accepted by `passthrough` fields: |
| 190 | + |
| 191 | +[horizontal] |
| 192 | + |
| 193 | +<<passthrough-conflicts,`priority`>>:: |
| 194 | + |
| 195 | + (Required) used for naming conflict resolution between pass-through fields. The field with the highest value wins. |
| 196 | + Accepts non-negative integer values. |
| 197 | + |
| 198 | +<<passthrough-dimensions,`time_series_dimension`>>:: |
| 199 | + |
| 200 | + Whether or not to treat sub-fields as <<time-series-dimension,time-series dimensions>>. |
| 201 | + Accepts `false` (default) or `true`. |
| 202 | + |
| 203 | +<<dynamic,`dynamic`>>:: |
| 204 | + |
| 205 | + Whether or not new `properties` should be added dynamically to an existing object. |
| 206 | + Accepts `true` (default), `runtime`, `false` and `strict`. |
| 207 | + |
| 208 | +<<enabled,`enabled`>>:: |
| 209 | + |
| 210 | + Whether the JSON value given for the object field should be parsed and indexed (`true`, default) |
| 211 | + or completely ignored (`false`). |
| 212 | + |
| 213 | +<<properties,`properties`>>:: |
| 214 | + |
| 215 | + The fields within the object, which can be of any <<mapping-types,data type>>, including `object`. |
| 216 | + New properties may be added to an existing object. |
| 217 | + |
| 218 | +IMPORTANT: If you need to index arrays of objects instead of single objects, read <<nested>> first. |
0 commit comments