Skip to content

Commit 986a574

Browse files
committed
Improve docs on LIMIT
This adds a few extra details around how ESQL processes input docs and how it limits output results.
1 parent 1d6c6a5 commit 986a574

File tree

1 file changed

+36
-2
lines changed

1 file changed

+36
-2
lines changed

docs/reference/query-languages/esql/limitations.md

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,20 +10,54 @@ mapped_pages:
1010

1111
## Result set size limit [esql-max-rows]
1212

13-
By default, an {{esql}} query returns up to 1000 rows. You can increase the number of rows up to 10,000 using the [`LIMIT`](/reference/query-languages/esql/esql-commands.md#esql-limit) command. Queries do not return more than 10,000 rows, regardless of the `LIMIT` command’s value.
13+
By default, an {{esql}} query returns up to 1,000 rows. You can increase the number of rows up to 10,000 using the [`LIMIT`](/reference/query-languages/esql/esql-commands.md#esql-limit) command.
1414

15-
This limit only applies to the number of rows that are retrieved by the query. Queries and aggregations run on the full data set.
15+
For instance,
16+
```esql
17+
FROM index | WHERE field = "value"
18+
```
19+
is equivalent to:
20+
```esql
21+
FROM index | WHERE field = "value" | LIMIT 1000
22+
```
23+
24+
Queries do not return more than 10,000 rows, regardless of the `LIMIT` command’s value. This is a configurable upper limit.
1625

1726
To overcome this limitation:
1827

1928
* Reduce the result set size by modifying the query to only return relevant data. Use [`WHERE`](/reference/query-languages/esql/esql-commands.md#esql-where) to select a smaller subset of the data.
2029
* Shift any post-query processing to the query itself. You can use the {{esql}} [`STATS`](/reference/query-languages/esql/esql-commands.md#esql-stats-by) command to aggregate data in the query.
2130

31+
The upper limit only applies to the number of rows that are output by the query, not to the number of documents it processes: the query runs on the full data set.
32+
33+
Consider the following two queries:
34+
```esql
35+
FROM index | WHERE field0 == "value" | LIMIT 20000
36+
```
37+
and
38+
```esql
39+
FROM index | STATS AVG(field1) BY field2 | LIMIT 20000
40+
```
41+
42+
In both cases, the filtering by `field0` in the first query or the grouping by `field2` in the second is applied over all the documents present in the `index`, irrespective of their number or indexes size. However, both queries will return at most 10,000 rows, even if there were more rows available to return.
43+
2244
The default and maximum limits can be changed using these dynamic cluster settings:
2345

2446
* `esql.query.result_truncation_default_size`
2547
* `esql.query.result_truncation_max_size`
2648

49+
However, doing so involves trade-offs. A larger result-set involves a higher memory pressure and increased processing times; the internode traffic within and across clusters can also increase.
50+
51+
These limitations are akin those enforced by the [search API](/reference/elasticsearch/rest-apis/paginate-search-results.html#from-and-size-pagination).
52+
53+
| Functionality | Search | {{esql}} |
54+
|----------------------------------|-------------------------|-------------------------------------------|
55+
| Results returned by default | 10 | 1.000 |
56+
| Default upper limit | 10,000 | 10,000 |
57+
| Specify number of results | `size` | `LIMIT` |
58+
| Change default number of results | n/a | esql.query.result_truncation_default_size |
59+
| Change default upper limit | index-max-result-window | esql.query.result_truncation_max_size |
60+
2761

2862
## Field types [esql-supported-types]
2963

0 commit comments

Comments
 (0)