From e8b83cb11222e127f2a975ef65a806f18a724b58 Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Tue, 25 Mar 2025 12:43:11 +0100 Subject: [PATCH 1/9] [DOCS][8.x] Add Search for ESQL landing page --- docs/reference/esql/esql-for-search.asciidoc | 191 +++++++++++++++++++ docs/reference/esql/esql-language.asciidoc | 7 +- 2 files changed, 196 insertions(+), 2 deletions(-) create mode 100644 docs/reference/esql/esql-for-search.asciidoc diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc new file mode 100644 index 0000000000000..3371e9e13c181 --- /dev/null +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -0,0 +1,191 @@ +[[esql-for-search]] +=== Search with {esql} + +This page provides an overview of how to use {esql} for search use cases. + +// [TIP] +// ==== +// Prefer to get started with a hands-on tutorial? Check out <>. +// ==== + +The following table summarizes the key search features available in {esql} and when they were introduced. + +[cols="1,1,2", options="header"] +|=== +|Feature |Available since |Description + +|<> +|8.17 +|Perform text searches with <> and <> + +|<> +|8.17 +|Execute complex queries with <> using Query String syntax + +|<> +|8.18/9.0 +|Calculate and sort by relevance with `METADATA _score` + +|Enhanced match options +|8.18/9.0 +|Configure text searches with additional parameters for the `match` function + +|<> +|8.18/9.0 +|Use Kibana Query Language with <> function + +|<> +|8.18/9.0 +|Perform semantic searches on `semantic_text` field types + +|<> +|8.18/9.0 +|Combine lexical and semantic search approaches with custom weights +|=== + +[[esql-for-search-full-text]] +==== Full text search + +[[esql-for-searc-match-functio-operator]] +===== Match function and operator + +ES|QL offers two syntax options for `match`, which replicate the functionality of <> queries in Query DSL. + +Use the compact operator syntax (`:`) for simple text matching with default parameters. + +[source,esql] +---- +FROM logs | WHERE match(message, "connection error") +---- + +Use the `match()` function syntax when you need to pass additional parameters: + +[source,esql] +---- +FROM products | WHERE match(name, "laptop", { "boost": 2.0 }) +---- + +[TIP] +==== +See <> for more advanced options using match. +==== + +[IMPORTANT] +==== +These queries match documents but don't automatically sort by relevance. To get the most relevant results first, you need to use `METADATA _score` and sort by score. See <> for more information. +==== + +[[esql-for-search-query-string]] +===== Query string function (`QSTR`) + +The <> provides the same functionality as the Query DSL's `query_string` query. This is for advanced use cases, such as wildcard searches, searches across multiple fields, and more. + +[source,esql] +---- +FROM articles METADATA _score +| WHERE QSTR("(new york city) OR (big apple)") +| SORT _score DESC +| LIMIT 10 +---- + +For complete details, refer to the <>. + +[[esql-for-search-scoring]] +==== Relevance scoring + +To get the most relevant results first, you need to retrieve and sort by relevance score: + +[source,esql] +---- +FROM books METADATA _score +| WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare") +| SORT _score DESC +---- + +When working with relevance scoring in ES|QL, it's important to understand how `_score` works: + +* If you don't include `METADATA _score` in your query, this only performs filtering operations with no relevance calculation. +* When you include `METADATA _score`, search functions included in `WHERE` conditions contribute to the relevance score. +* Filtering operations (like range conditions and exact matches) don't affect the score. +* Including `METADATA _score` doesn't automatically sort your results by relevance. You must explicitly use `SORT _score DESC` or `SORT _score ASC` to order your results by relevance. + +[[esql-for-search-semantic]] +==== Semantic search + +You can perform semantic searches over <> field types using the same match syntax as full-text search. + +This example uses the match operator `:``: + +[source,esql] +---- +FROM articles METADATA _score +| WHERE semantic_content:"What are the impacts of climate change on agriculture?" +| SORT _score DESC +---- + +This example uses the match function: + +[source,esql] +---- +FROM articles METADATA _score +| WHERE match(semantic_content, "What are the impacts of climate change on agriculture?") +| SORT _score DESC +---- + +[[esql-for-search-hybrid]] +==== Hybrid search + +Combine traditional and semantic search with custom weights: + +[source,esql] +---- +FROM books METADATA _score +| WHERE match(semantic_title, "fantasy adventure", { "boost": 0.75 }) + OR match(title, "fantasy adventure", { "boost": 0.25 }) +| SORT _score DESC +---- + +[[esql-for-search-kql]] +==== Kibana Query Language (KQL) + +Use the <> to use the {kibana-ref}/kuery-query.html[Kibana Query Language] in your ES|QL queries: + +[source,esql] +---- +FROM logs* +| WHERE KQL("http.request.method:GET AND agent.type:filebeat") +---- + +The `kql` function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to ES|QL without needing to rewrite them all at once. + +[[esql-for-search-limitations]] +==== Limitations + +Refer to <> for a list of known limitations. + +[[esql-for-search-next-steps]] +==== Next steps + +[[esql-for-search-tutorials]] +===== Tutorials and how-to guides + +// TODO * <>: Hands-on tutorial for getting started with search tools in {esql} +* <>: Learn how to use the `semantic_text` field type + +[[esql-for-search-reference]] +===== Technical reference + +* <>: Complete reference for all search functions +* <>: Current limitations for search in ES|QL + +[[esql-for-search-concepts]] +===== Background concepts + +* <>: How text is processed for full-text search +* <>: Overview of semantic search in Elasticsearch + +[[esql-for-search-blogs]] +===== Related blog posts + +// TODO* https://www.elastic.co/blog/esql-you-know-for-search-scoring-semantic-search[ES|QL, you know for Search] - Introducing scoring and semantic search +* https://www.elastic.co/blog/introducing-full-text-filtering-with-esql[Introducing full text filtering in ES|QL] - Overview of text filtering capabilities diff --git a/docs/reference/esql/esql-language.asciidoc b/docs/reference/esql/esql-language.asciidoc index cb2d8260469f6..b0590b76a9672 100644 --- a/docs/reference/esql/esql-language.asciidoc +++ b/docs/reference/esql/esql-language.asciidoc @@ -11,8 +11,10 @@ Detailed reference documentation for the {esql} language: * <> * <> * <> -* <> +* <> * <> + +* <> * <> * <> * <> @@ -22,8 +24,9 @@ include::esql-commands.asciidoc[] include::esql-functions-operators.asciidoc[] include::metadata-fields.asciidoc[] include::multivalued-fields.asciidoc[] +include::esql-for-search.asciidoc[] include::esql-process-data-with-dissect-grok.asciidoc[] -include::esql-enrich-data.asciidoc[] include::esql-lookup-join.asciidoc[] +include::esql-enrich-data.asciidoc[] include::implicit-casting.asciidoc[] include::time-spans.asciidoc[] From 2097198c497a42616972dce1790d11b94429b958 Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Tue, 25 Mar 2025 14:27:55 +0100 Subject: [PATCH 2/9] tweak wording --- docs/reference/esql/esql-for-search.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index 3371e9e13c181..441f577d8b450 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -16,7 +16,7 @@ The following table summarizes the key search features available in {esql} and w |<> |8.17 -|Perform text searches with <> and <> +|Perform basic text searches with <> and <> |<> |8.17 @@ -176,7 +176,7 @@ Refer to <> for a list of ===== Technical reference * <>: Complete reference for all search functions -* <>: Current limitations for search in ES|QL +* <>: Current limitations for search in ES|QL [[esql-for-search-concepts]] ===== Background concepts From 489e126e5de0dad18734992d9e738e57d9c0485b Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Tue, 25 Mar 2025 17:01:54 +0100 Subject: [PATCH 3/9] Revamp per Carlos' review --- docs/reference/esql/esql-for-search.asciidoc | 82 ++++++++++++-------- 1 file changed, 51 insertions(+), 31 deletions(-) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index 441f577d8b450..133e61a4352c3 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -43,10 +43,40 @@ The following table summarizes the key search features available in {esql} and w |Combine lexical and semantic search approaches with custom weights |=== +[[esql-filtering-vs-searching]] +==== Filtering vs. searching + +{esql} can be used for both simple filtering and relevance-based searching: + +* **Filtering** removes non-matching documents without calculating relevance scores +* **Searching** both filters documents and ranks them by how well they match the query + +[[esql-for-search-scoring]] +===== Relevance scoring + +To get the most relevant results first, you need to use `METADATA _score` and sort by score. For example: + +[source,esql] +---- +FROM books METADATA _score +| WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare") +| SORT _score DESC +---- + +[[esql-for-search-how-scoring-works]] +===== How `_score` works + +When working with relevance scoring in ES|QL: + +* If you don't include `METADATA _score` in your query, this only performs filtering operations with no relevance calculation. +* When you include `METADATA _score`, any search function included in `WHERE` conditions contribute to the relevance score. This means that every occurrence of `MATCH`, `QSTR` and `KQL` will affect the score. +* Filtering operations that are not search functions, like range conditions and exact matches, don't affect the score. +* Including `METADATA _score` doesn't automatically sort your results by relevance. You must explicitly use `SORT _score DESC` or `SORT _score ASC` to order your results by relevance. + [[esql-for-search-full-text]] ==== Full text search -[[esql-for-searc-match-functio-operator]] +[[esql-for-search-match-function-operator]] ===== Match function and operator ES|QL offers two syntax options for `match`, which replicate the functionality of <> queries in Query DSL. @@ -65,6 +95,14 @@ Use the `match()` function syntax when you need to pass additional parameters: FROM products | WHERE match(name, "laptop", { "boost": 2.0 }) ---- +These full-text functions address several key limitations that existed for text filtering in {esql}: + +* They work directly on multivalued fields, returning results when any value in a multivalued field matches the query +* They leverage analyzers, ensuring the query is analyzed with the same process as the indexed data (enabling case-insensitive matching, ASCII folding, stopword removal, and synonym support) +* They are highly performant, using Lucene index structures rather than pattern matching or regular expressions to locate terms in your data + +Refer to this blog for more context: https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL] for more information.. + [TIP] ==== See <> for more advanced options using match. @@ -90,31 +128,25 @@ FROM articles METADATA _score For complete details, refer to the <>. -[[esql-for-search-scoring]] -==== Relevance scoring +[[esql-for-search-kql]] +===== Kibana Query Language (KQL) function -To get the most relevant results first, you need to retrieve and sort by relevance score: +Use the <> to use the {kibana-ref}/kuery-query.html[Kibana Query Language] in your ES|QL queries: [source,esql] ---- -FROM books METADATA _score -| WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare") -| SORT _score DESC +FROM logs* +| WHERE KQL("http.request.method:GET AND agent.type:filebeat") ---- -When working with relevance scoring in ES|QL, it's important to understand how `_score` works: - -* If you don't include `METADATA _score` in your query, this only performs filtering operations with no relevance calculation. -* When you include `METADATA _score`, search functions included in `WHERE` conditions contribute to the relevance score. -* Filtering operations (like range conditions and exact matches) don't affect the score. -* Including `METADATA _score` doesn't automatically sort your results by relevance. You must explicitly use `SORT _score DESC` or `SORT _score ASC` to order your results by relevance. +The `kql` function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to ES|QL without needing to rewrite them all at once. [[esql-for-search-semantic]] ==== Semantic search You can perform semantic searches over <> field types using the same match syntax as full-text search. -This example uses the match operator `:``: +This example uses the match operator `:`: [source,esql] ---- @@ -145,19 +177,6 @@ FROM books METADATA _score | SORT _score DESC ---- -[[esql-for-search-kql]] -==== Kibana Query Language (KQL) - -Use the <> to use the {kibana-ref}/kuery-query.html[Kibana Query Language] in your ES|QL queries: - -[source,esql] ----- -FROM logs* -| WHERE KQL("http.request.method:GET AND agent.type:filebeat") ----- - -The `kql` function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to ES|QL without needing to rewrite them all at once. - [[esql-for-search-limitations]] ==== Limitations @@ -181,11 +200,12 @@ Refer to <> for a list of [[esql-for-search-concepts]] ===== Background concepts -* <>: How text is processed for full-text search -* <>: Overview of semantic search in Elasticsearch +* <>: Learn how text is processed for full-text search +* <>: Get an overview of semantic search in {es} +* <>: Understand the difference between query and filter contexts in {es} [[esql-for-search-blogs]] ===== Related blog posts -// TODO* https://www.elastic.co/blog/esql-you-know-for-search-scoring-semantic-search[ES|QL, you know for Search] - Introducing scoring and semantic search -* https://www.elastic.co/blog/introducing-full-text-filtering-with-esql[Introducing full text filtering in ES|QL] - Overview of text filtering capabilities +// TODO* https://www.elastic.co/blog/esql-you-know-for-search-scoring-semantic-search[ES|QL, you know for Search]: Introducing scoring and semantic search +* https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL]: Overview of text filtering capabilities From 2813ba758f67d313facef90dcf2724ae22e89e41 Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Tue, 25 Mar 2025 17:05:52 +0100 Subject: [PATCH 4/9] fix typo --- docs/reference/esql/esql-for-search.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index 133e61a4352c3..51e86e305f489 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -101,7 +101,7 @@ These full-text functions address several key limitations that existed for text * They leverage analyzers, ensuring the query is analyzed with the same process as the indexed data (enabling case-insensitive matching, ASCII folding, stopword removal, and synonym support) * They are highly performant, using Lucene index structures rather than pattern matching or regular expressions to locate terms in your data -Refer to this blog for more context: https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL] for more information.. +Refer to this blog for more context: https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL]. [TIP] ==== From 9b108a0ca86bef58c6a4bd4852f21390fc5fc25a Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Tue, 25 Mar 2025 17:07:25 +0100 Subject: [PATCH 5/9] make heading consistent --- docs/reference/esql/esql-for-search.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index 51e86e305f489..7bc51f42bfa18 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -129,7 +129,7 @@ FROM articles METADATA _score For complete details, refer to the <>. [[esql-for-search-kql]] -===== Kibana Query Language (KQL) function +===== Kibana Query Language function (`KQL`) Use the <> to use the {kibana-ref}/kuery-query.html[Kibana Query Language] in your ES|QL queries: From b6c22074a0b6173d090dd2da0845699b1848e91c Mon Sep 17 00:00:00 2001 From: Liam Thompson <32779855+leemthompo@users.noreply.github.com> Date: Tue, 25 Mar 2025 17:58:34 +0100 Subject: [PATCH 6/9] mention filtering is faster than searching --- docs/reference/esql/esql-for-search.asciidoc | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index 7bc51f42bfa18..b743bd4e948e6 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -51,6 +51,7 @@ The following table summarizes the key search features available in {esql} and w * **Filtering** removes non-matching documents without calculating relevance scores * **Searching** both filters documents and ranks them by how well they match the query +Note that filtering is faster than searching, because it doesn't require score calculations. [[esql-for-search-scoring]] ===== Relevance scoring From 1e9b31f0f72171ac30fe1efe61c527cf7486b1e5 Mon Sep 17 00:00:00 2001 From: Liam Thompson <32779855+leemthompo@users.noreply.github.com> Date: Tue, 25 Mar 2025 17:59:38 +0100 Subject: [PATCH 7/9] add space --- docs/reference/esql/esql-for-search.asciidoc | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index b743bd4e948e6..de5fdc6b218fc 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -52,6 +52,7 @@ The following table summarizes the key search features available in {esql} and w * **Searching** both filters documents and ranks them by how well they match the query Note that filtering is faster than searching, because it doesn't require score calculations. + [[esql-for-search-scoring]] ===== Relevance scoring From fa88d925d9204d1950d4fa121f9033330d88a94d Mon Sep 17 00:00:00 2001 From: Liam Thompson <32779855+leemthompo@users.noreply.github.com> Date: Wed, 26 Mar 2025 08:42:50 +0100 Subject: [PATCH 8/9] Add tech preview banner --- docs/reference/esql/esql-for-search.asciidoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index de5fdc6b218fc..85244187080f0 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -1,6 +1,8 @@ [[esql-for-search]] === Search with {esql} +preview[] + This page provides an overview of how to use {esql} for search use cases. // [TIP] From bdd1305c9f35e527e94f44527c5f18443cc2f3a0 Mon Sep 17 00:00:00 2001 From: Liam Thompson Date: Wed, 26 Mar 2025 09:54:19 +0100 Subject: [PATCH 9/9] move to "Using ESQL" section with improved title Renames "Search with ESQL" to "Using ESQL for search" and relocates the page from language reference to usage section for better discoverability. --- docs/reference/esql/esql-for-search.asciidoc | 4 +- docs/reference/esql/esql-get-started.asciidoc | 2 +- docs/reference/esql/esql-language.asciidoc | 3 - docs/reference/esql/esql-using.asciidoc | 18 ++++-- docs/reference/esql/index.asciidoc | 59 +++++++++++-------- 5 files changed, 51 insertions(+), 35 deletions(-) diff --git a/docs/reference/esql/esql-for-search.asciidoc b/docs/reference/esql/esql-for-search.asciidoc index 85244187080f0..3e6869298b691 100644 --- a/docs/reference/esql/esql-for-search.asciidoc +++ b/docs/reference/esql/esql-for-search.asciidoc @@ -1,7 +1,7 @@ [[esql-for-search]] -=== Search with {esql} +=== Using {esql} for search -preview[] +preview::[] This page provides an overview of how to use {esql} for search use cases. diff --git a/docs/reference/esql/esql-get-started.asciidoc b/docs/reference/esql/esql-get-started.asciidoc index b7928898a3bbb..6451ed1909201 100644 --- a/docs/reference/esql/esql-get-started.asciidoc +++ b/docs/reference/esql/esql-get-started.asciidoc @@ -4,7 +4,7 @@ Getting started ++++ -This guide shows how you can use {esql} to query and aggregate your data. +This guide shows how you can use {esql} to query and aggregate your data. Refer to <> if you'd like to learn more about using {esql} for search use cases. [TIP] ==== diff --git a/docs/reference/esql/esql-language.asciidoc b/docs/reference/esql/esql-language.asciidoc index b0590b76a9672..38960df404a37 100644 --- a/docs/reference/esql/esql-language.asciidoc +++ b/docs/reference/esql/esql-language.asciidoc @@ -11,9 +11,7 @@ Detailed reference documentation for the {esql} language: * <> * <> * <> -* <> * <> - * <> * <> * <> @@ -24,7 +22,6 @@ include::esql-commands.asciidoc[] include::esql-functions-operators.asciidoc[] include::metadata-fields.asciidoc[] include::multivalued-fields.asciidoc[] -include::esql-for-search.asciidoc[] include::esql-process-data-with-dissect-grok.asciidoc[] include::esql-lookup-join.asciidoc[] include::esql-enrich-data.asciidoc[] diff --git a/docs/reference/esql/esql-using.asciidoc b/docs/reference/esql/esql-using.asciidoc index d2e18bf1b91a3..a7b7ad1998ab9 100644 --- a/docs/reference/esql/esql-using.asciidoc +++ b/docs/reference/esql/esql-using.asciidoc @@ -1,27 +1,33 @@ [[esql-using]] == Using {esql} +This page is an overview of the various ways you can use {esql} across different Elastic interfaces and use cases. + <>:: -Information about using the <>. +Learn how to use the <>. + +<>:: +Learn how to use {esql} for search use cases. <>:: -Using {esql} in {kib} to query and aggregate your data, create visualizations, +Learn how to use {esql} in {kib} to query and aggregate your data, create visualizations, and set up alerts. <>:: -Using {esql} in {elastic-sec} to investigate events in Timeline, create +Learn how to use {esql} in {elastic-sec} to investigate events in Timeline, create detection rules, and build {esql} queries using Elastic AI Assistant. <>:: -Using {esql} to query multiple indexes and resolve field type mismatches. +Learn how to use {esql} to query multiple indexes and resolve field type mismatches. <>:: -Using {esql} to query across multiple clusters. +Learn how to use {esql} to query across multiple clusters. <>:: -Using the <> to list and cancel {esql} queries. +Learn how to use the <> to list and cancel {esql} queries. include::esql-rest.asciidoc[] +include::esql-for-search.asciidoc[] include::esql-kibana.asciidoc[] include::esql-security-solution.asciidoc[] include::esql-multi-index.asciidoc[] diff --git a/docs/reference/esql/index.asciidoc b/docs/reference/esql/index.asciidoc index 54627a6de3c62..7c8c793d483bb 100644 --- a/docs/reference/esql/index.asciidoc +++ b/docs/reference/esql/index.asciidoc @@ -22,21 +22,7 @@ a series of operations, where the output of one operation becomes the input for the next, enabling complex data transformations and analysis. [discrete] -=== The {esql} Compute Engine - -{esql} is more than a language: it represents a significant investment in new -compute capabilities within {es}. To achieve both the functional and performance -requirements for {esql}, it was necessary to build an entirely new compute -architecture. {esql} search, aggregation, and transformation functions are -directly executed within Elasticsearch itself. Query expressions are not -transpiled to Query DSL for execution. This approach allows {esql} to be -extremely performant and versatile. - -The new {esql} execution engine was designed with performance in mind — it -operates on blocks at a time instead of per row, targets vectorization and cache -locality, and embraces specialization and multi-threading. It is a separate -component from the existing Elasticsearch aggregation framework with different -performance characteristics. +=== Documentation organization The {esql} documentation is organized in these sections: @@ -45,16 +31,24 @@ A tutorial to help you get started with {esql}. <>:: -Reference documentation for the <>, -<>, and <>. Information about working with <> and <>. And guidance for -<> and <>. +Reference documentation for the <>: + +* Reference for <>, and <> +* How to work with <> and <> +* How to work with +<>, <>, and <> <>:: -An overview of using the <>, <>, -<>, <>, and <>. +An overview of: +* <> +* <> +* <> +* <> +* <> +* <> <>:: The current limitations of {esql}. @@ -62,6 +56,8 @@ The current limitations of {esql}. <>:: A few examples of what you can do with {esql}. + + include::esql-get-started.asciidoc[] include::esql-language.asciidoc[] @@ -74,3 +70,20 @@ include::esql-examples.asciidoc[] :esql-tests!: :esql-specs!: + +[discrete] +=== The {esql} Compute Engine + +{esql} is more than a language: it represents a significant investment in new +compute capabilities within {es}. To achieve both the functional and performance +requirements for {esql}, it was necessary to build an entirely new compute +architecture. {esql} search, aggregation, and transformation functions are +directly executed within Elasticsearch itself. Query expressions are not +transpiled to Query DSL for execution. This approach allows {esql} to be +extremely performant and versatile. + +The new {esql} execution engine was designed with performance in mind — it +operates on blocks at a time instead of per row, targets vectorization and cache +locality, and embraces specialization and multi-threading. It is a separate +component from the existing Elasticsearch aggregation framework with different +performance characteristics. \ No newline at end of file