Skip to content

Commit 0acda3a

Browse files
authored
ESQL: Add documents_found and values_loaded (#125631) (#130029)
This adds `documents_found` and `values_loaded` to the to the ESQL response: ```json { "took" : 194, "is_partial" : false, "documents_found" : 100000, "values_loaded" : 200000, "columns" : [ { "name" : "a", "type" : "long" }, { "name" : "b", "type" : "long" } ], "values" : [[10, 1]] } ``` These are cheap enough to collect that we can do it for every query and return it with every response. It's small, but it still gives you a reasonable sense of how much work Elasticsearch had to go through to perform the query. I've also added these two fields to the driver profile and task status: ```json "drivers" : [ { "description" : "data", "cluster_name" : "runTask", "node_name" : "runTask-0", "start_millis" : 1742923173077, "stop_millis" : 1742923173087, "took_nanos" : 9557014, "cpu_nanos" : 9091340, "documents_found" : 5, <---- THESE "values_loaded" : 15, <---- THESE "iterations" : 6, ... ``` These are at a high level and should be easy to reason about. We'd like to extract this into a "show me how difficult this running query is" API one day. But today, just plumbing it into the debugging output is good. Any `Operator` can claim to "find documents" or "load values" by overriding a method on its `Operator.Status` implementation: ```java /** * The number of documents found by this operator. Most operators * don't find documents and will return {@code 0} here. */ default long documentsFound() { return 0; } /** * The number of values loaded by this operator. Most operators * don't load values and will return {@code 0} here. */ default long valuesLoaded() { return 0; } ``` In this PR all of the `LuceneOperator`s declare that each `position` they emit is a "document found" and the `ValuesSourceValuesSourceReaderOperator` says each value it makes is a "value loaded". That's pretty pretty much true. The `LuceneCountOperator` and `LuceneMinMaxOperator` sort of pretend that the count/min/max that they emit is a "document" - but that's good enough to give you a sense of what's going on. It's *like* document.
1 parent a0579d5 commit 0acda3a

File tree

57 files changed

+1168
-359
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+1168
-359
lines changed

docs/changelog/125631.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 125631
2+
summary: Add `documents_found` and `values_loaded`
3+
area: ES|QL
4+
type: enhancement
5+
issues: []

docs/reference/esql/esql-rest.asciidoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,8 @@ Which returns:
194194
{
195195
"took": 28,
196196
"is_partial": false,
197+
"documents_found": 5,
198+
"values_loaded": 20,
197199
"columns": [
198200
{"name": "author", "type": "text"},
199201
{"name": "name", "type": "text"},

docs/reference/esql/functions/description/knn.asciidoc

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/esql/functions/examples/knn.asciidoc

Lines changed: 21 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/esql/functions/functionNamedParams/knn.asciidoc

Lines changed: 13 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/esql/functions/functionNamedParams/to_ip.asciidoc

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/esql/functions/layout/knn.asciidoc

Lines changed: 18 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/esql/functions/parameters/knn.asciidoc

Lines changed: 12 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/esql/functions/signature/knn.svg

Lines changed: 1 addition & 0 deletions
Loading

docs/reference/esql/functions/types/knn.asciidoc

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)