You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: explore-analyze/aggregations.md
+8-23Lines changed: 8 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,8 +21,7 @@ An aggregation summarizes your data as metrics, statistics, or other analytics.
21
21
*[Bucket](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket.html) aggregations that group documents into buckets, also called bins, based on field values, ranges, or other criteria.
22
22
*[Pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline.html) aggregations that take input from other aggregations instead of documents or fields.
23
23
24
-
25
-
## Run an aggregation [run-an-agg]
24
+
## Run an aggregation [run-an-agg]
26
25
27
26
You can run aggregations as part of a [search](../solutions/search/querying-for-search.md) by specifying the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html)'s `aggs` parameter. The following search runs a [terms aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html) on `my-field`:
28
27
@@ -71,9 +70,7 @@ Aggregation results are in the response’s `aggregations` object:
71
70
72
71
1. Results for the `my-agg-name` aggregation.
73
72
74
-
75
-
76
-
## Change an aggregation’s scope [change-agg-scope]
73
+
## Change an aggregation’s scope [change-agg-scope]
77
74
78
75
Use the `query` parameter to limit the documents on which an aggregation runs:
79
76
@@ -98,8 +95,7 @@ GET /my-index-000001/_search
98
95
}
99
96
```
100
97
101
-
102
-
## Return only aggregation results [return-only-agg-results]
98
+
## Return only aggregation results [return-only-agg-results]
103
99
104
100
By default, searches containing an aggregation return both search hits and aggregation results. To return only aggregation results, set `size` to `0`:
105
101
@@ -117,7 +113,6 @@ GET /my-index-000001/_search
117
113
}
118
114
```
119
115
120
-
121
116
## Run multiple aggregations [run-multiple-aggs]
122
117
123
118
You can specify multiple aggregations in the same request:
@@ -140,8 +135,7 @@ GET /my-index-000001/_search
140
135
}
141
136
```
142
137
143
-
144
-
## Run sub-aggregations [run-sub-aggs]
138
+
## Run sub-aggregations [run-sub-aggs]
145
139
146
140
Bucket aggregations support bucket or metric sub-aggregations. For example, a terms aggregation with an [avg](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-avg-aggregation.html) sub-aggregation calculates an average value for each bucket of documents. There is no level or depth limit for nesting sub-aggregations.
147
141
@@ -191,8 +185,6 @@ The response nests sub-aggregation results under their parent aggregation:
191
185
1. Results for the parent aggregation, `my-agg-name`.
192
186
2. Results for `my-agg-name`'s sub-aggregation, `my-sub-agg-name`.
193
187
194
-
195
-
196
188
## Add custom metadata [add-metadata-to-an-agg]
197
189
198
190
Use the `meta` object to associate custom metadata with an aggregation:
@@ -231,8 +223,7 @@ The response returns the `meta` object in place:
231
223
}
232
224
```
233
225
234
-
235
-
## Return the aggregation type [return-agg-type]
226
+
## Return the aggregation type [return-agg-type]
236
227
237
228
By default, aggregation results include the aggregation’s name but not its type. To return the aggregation type, use the `typed_keys` query parameter.
238
229
@@ -252,11 +243,10 @@ GET /my-index-000001/_search?typed_keys
252
243
253
244
The response returns the aggregation type as a prefix to the aggregation’s name.
254
245
255
-
::::{important}
246
+
::::{important}
256
247
Some aggregations return a different aggregation type from the type in the request. For example, the terms, [significant terms](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-significantterms-aggregation.html), and [percentiles](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-percentile-aggregation.html) aggregations return different aggregations types depending on the data type of the aggregated field.
257
248
::::
258
249
259
-
260
250
```console-result
261
251
{
262
252
...
@@ -270,8 +260,6 @@ Some aggregations return a different aggregation type from the type in the reque
270
260
271
261
1. The aggregation type, `histogram`, followed by a `#` separator and the aggregation’s name, `my-agg-name`.
272
262
273
-
274
-
275
263
## Use scripts in an aggregation [use-scripts-in-an-agg]
276
264
277
265
When a field doesn’t exactly match the aggregation you need, you should aggregate on a [runtime field](../manage-data/data-store/mapping/runtime-fields.md):
@@ -298,15 +286,12 @@ GET /my-index-000001/_search?size=0
298
286
299
287
Scripts calculate field values dynamically, which adds a little overhead to the aggregation. In addition to the time spent calculating, some aggregations like [`terms`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html) and [`filters`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-filters-aggregation.html) can’t use some of their optimizations with runtime fields. In total, performance costs for using a runtime field varies from aggregation to aggregation.
300
288
301
-
302
-
## Aggregation caches [agg-caches]
289
+
## Aggregation caches [agg-caches]
303
290
304
291
For faster responses, {{es}} caches the results of frequently run aggregations in the [shard request cache](https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-request-cache.html). To get cached results, use the same [`preference` string](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-shard-routing.html#shard-and-node-preference) for each search. If you don’t need search hits, [set `size` to `0`](#return-only-agg-results) to avoid filling the cache.
305
292
306
293
{{es}} routes searches with the same preference string to the same shards. If the shards' data doesn’t change between searches, the shards return cached aggregation results.
307
294
308
-
309
-
## Limits for `long` values [limits-for-long-values]
295
+
## Limits for `long` values [limits-for-long-values]
310
296
311
297
When running aggregations, {{es}} uses [`double`](https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html) values to hold and represent numeric data. As a result, aggregations on [`long`](https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html) numbers greater than `253` are approximate.
* Select the **Other sample data sets** collapsible.
43
39
* Add the **Sample eCommerce orders** data set. This will create and populate an index called `kibana_sample_data_ecommerce`.
44
40
45
-
46
-
47
41
## Inspect index structure [aggregations-tutorial-inspect-data]
48
42
49
43
Before we start analyzing the data, let’s examine the structure of the documents in our sample eCommerce index. Run this command to see the field [mappings](../../manage-data/data-store/index-basics.md#elasticsearch-intro-documents-fields-mappings):
@@ -55,6 +49,7 @@ GET kibana_sample_data_ecommerce/_mapping
55
49
The response shows the field mappings for the `kibana_sample_data_ecommerce` index.
56
50
57
51
::::{dropdown} Example response
52
+
58
53
```console-response
59
54
{
60
55
"kibana_sample_data_ecommerce": {
@@ -271,34 +266,28 @@ The response shows the field mappings for the `kibana_sample_data_ecommerce` ind
271
266
3. `geoip.location`: Geographic coordinates stored as geo_point for location-based queries
272
267
4. `products.properties`: Nested structure containing details about items in each order
273
268
274
-
275
269
::::
276
270
277
-
278
271
The sample data includes the following [field data types](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html):
279
272
280
273
* [`text`](https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html) and [`keyword`](https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html) for text fields
281
-
282
-
* Most `text` fields have a `.keyword` subfield for exact matching using [multi-fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html)
274
+
* Most `text` fields have a `.keyword` subfield for exact matching using [multi-fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html)
283
275
284
276
* [`date`](https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html) for date fields
* [`geo_point`](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html) for geographic coordinates
292
283
* [`object`](https://www.elastic.co/guide/en/elasticsearch/reference/current/object.html) for nested structures such as `products`, `geoip`, `event`
293
284
294
285
Now that we understand the structure of our sample data, let’s start analyzing it.
295
286
296
-
297
287
## Get key business metrics [aggregations-tutorial-basic-metrics]
298
288
299
289
Let’s start by calculating important metrics about orders and customers.
300
290
301
-
302
291
### Get average order size [aggregations-tutorial-order-value]
303
292
304
293
Calculate the average order value across all orders in the dataset using the [`avg`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-avg-aggregation.html) aggregation.
@@ -321,8 +310,8 @@ GET kibana_sample_data_ecommerce/_search
321
310
2. A meaningful name that describes what this metric represents
322
311
3. Configures an `avg` aggregation, which calculates a simple arithmetic mean
323
312
324
-
325
313
::::{dropdown} Example response
314
+
326
315
```console-result
327
316
{
328
317
"took": 0,
@@ -354,11 +343,8 @@ GET kibana_sample_data_ecommerce/_search
354
343
3. Results appear under the name we specified in the request
355
344
4. The average order value is calculated dynamically from all the orders in the dataset
356
345
357
-
358
346
::::
359
347
360
-
361
-
362
348
### Get multiple order statistics at once [aggregations-tutorial-order-stats]
363
349
364
350
Calculate multiple statistics about orders in one request using the [`stats`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-stats-aggregation.html) aggregation.
@@ -380,8 +366,8 @@ GET kibana_sample_data_ecommerce/_search
380
366
1. A descriptive name for this set of statistics
381
367
2. `stats` returns count, min, max, avg, and sum at once
382
368
383
-
384
369
::::{dropdown} Example response
370
+
385
371
```console-result
386
372
{
387
373
"aggregations": {
@@ -402,22 +388,17 @@ GET kibana_sample_data_ecommerce/_search
402
388
4. `"avg"`: Average value per order across all orders
403
389
5. `"sum"`: Total revenue from all orders combined
404
390
405
-
406
391
::::
407
392
408
-
409
393
::::{tip}
410
394
The [stats aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-stats-aggregation.html) is more efficient than running individual min, max, avg, and sum aggregations.
Let’s group orders in different ways to understand sales patterns.
419
401
420
-
421
402
### Break down sales by category [aggregations-tutorial-category-breakdown]
422
403
423
404
Group orders by category to see which product categories are most popular, using the [`terms`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html) aggregation.
@@ -444,8 +425,8 @@ GET kibana_sample_data_ecommerce/_search
444
425
4. Limit to top 5 categories
445
426
5. Order by number of orders (descending)
446
427
447
-
448
428
::::{dropdown} Example response
429
+
449
430
```console-result
450
431
{
451
432
"took": 4,
@@ -501,11 +482,8 @@ GET kibana_sample_data_ecommerce/_search
Group orders by day to track daily sales patterns using the [`date_histogram`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html) aggregation.
@@ -533,8 +511,8 @@ GET kibana_sample_data_ecommerce/_search
533
511
4. Formats dates in response using [date patterns](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html) (e.g. "yyyy-MM-dd"). Refer to [date math expressions](https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#date-math) for additional options.
534
512
5. When `min_doc_count` is 0, returns buckets for days with no orders, useful for continuous time series visualization.
535
513
536
-
537
514
::::{dropdown} Example response
515
+
538
516
```console-result
539
517
{
540
518
"took": 2,
@@ -723,16 +701,12 @@ GET kibana_sample_data_ecommerce/_search
723
701
4. `key` is the same date represented as the Unix timestamp for this bucket
724
702
5. `doc_count` counts the number of documents that fall into this time bucket
725
703
726
-
727
704
::::
728
705
729
-
730
-
731
706
## Combine metrics with groupings [aggregations-tutorial-combined-analysis]
732
707
733
708
Now let’s calculate [metrics](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics.html) within each group to get deeper insights.
Let’s combine metrics to track daily trends: daily revenue, unique customers, and average basket size.
@@ -859,8 +830,8 @@ GET kibana_sample_data_ecommerce/_search
859
830
2. Uses the [`cardinality`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html) aggregation to count unique customers per day
860
831
3. Average number of items per order
861
832
862
-
863
833
::::{dropdown} Example response
834
+
864
835
```console-result
865
836
{
866
837
"took": 119,
@@ -1324,13 +1295,10 @@ GET kibana_sample_data_ecommerce/_search
1324
1295
1325
1296
::::
1326
1297
1327
-
1328
-
1329
1298
## Track trends and patterns [aggregations-tutorial-trends]
1330
1299
1331
1300
You can use [pipeline aggregations](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline.html) on the results of other aggregations. Let’s analyze how metrics change over time.
1332
1301
1333
-
1334
1302
### Smooth out daily fluctuations [aggregations-tutorial-moving-average]
1335
1303
1336
1304
Moving averages help identify trends by reducing day-to-day noise in the data. Let’s observe sales trends more clearly by smoothing daily revenue variations, using the [Moving Function](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-movfn-aggregation.html) aggregation.
@@ -1371,8 +1339,8 @@ GET kibana_sample_data_ecommerce/_search
1371
1339
5. Use a 3-day window — use different window sizes to see trends at different time scales.
1372
1340
6. Use the built-in unweighted average functionin the `moving_fn` aggregation.
1373
1341
1374
-
1375
1342
::::{dropdown} Example response
1343
+
1376
1344
```console-result
1377
1345
{
1378
1346
"took": 13,
@@ -1747,17 +1715,13 @@ GET kibana_sample_data_ecommerce/_search
1747
1715
4. First day has no smoothed value as it needs previous days for the calculation
1748
1716
5. Moving average starts from second day, using a 3-day window
1749
1717
1750
-
1751
1718
::::
1752
1719
1753
-
1754
1720
::::{tip}
1755
1721
Notice how the smoothed values lag behind the actual values - this is because they need previous days' data to calculate. The first day will always be null when using moving averages.
Track running totals over time using the [`cumulative_sum`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-cumulative-sum-aggregation.html) aggregation.
@@ -1793,8 +1757,8 @@ GET kibana_sample_data_ecommerce/_search
1793
1757
2. `cumulative_sum` adds up values across buckets
1794
1758
3. Reference the revenue we want to accumulate
1795
1759
1796
-
1797
1760
::::{dropdown} Example response
1761
+
1798
1762
```console-result
1799
1763
{
1800
1764
"took": 4,
@@ -2169,11 +2133,8 @@ GET kibana_sample_data_ecommerce/_search
2169
2133
4. `revenue`: Daily revenue for this date
2170
2134
5. `cumulative_revenue`: Running total of revenue up to this date
2171
2135
2172
-
2173
2136
::::
2174
2137
2175
-
2176
-
2177
2138
## Next steps [aggregations-tutorial-next-steps]
2178
2139
2179
2140
Refer to the [aggregations reference](../aggregations.md) for more details on all available aggregation types.
0 commit comments