Skip to content

Commit e8b83cb

Browse files
leemthompoleemthompo
committed
[DOCS][8.x] Add Search for ESQL landing page
1 parent d484828 commit e8b83cb

File tree

2 files changed

+196
-2
lines changed

2 files changed

+196
-2
lines changed

‎docs/reference/esql/esql-for-search.asciidoc‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
[[esql-for-search]]
2+
=== Search with {esql}
3+
4+
This page provides an overview of how to use {esql} for search use cases.
5+
6+
// [TIP]
7+
// ====
8+
// Prefer to get started with a hands-on tutorial? Check out <<esql-search-tutorial>>.
9+
// ====
10+
11+
The following table summarizes the key search features available in {esql} and when they were introduced.
12+
13+
[cols="1,1,2", options="header"]
14+
|===
15+
|Feature |Available since |Description
16+
17+
|<<esql-search-functions,Full text search functions>>
18+
|8.17
19+
|Perform text searches with <<esql-match, match function>> and <<esql-search-operators,match operator (`:`)>>
20+
21+
|<<esql-for-search-query-string,Query string function>>
22+
|8.17
23+
|Execute complex queries with <<esql-qstr,`qstr`>> using Query String syntax
24+
25+
|<<esql-for-search-scoring,Relevance scoring>>
26+
|8.18/9.0
27+
|Calculate and sort by relevance with `METADATA _score`
28+
29+
|Enhanced match options
30+
|8.18/9.0
31+
|Configure text searches with additional parameters for the `match` function
32+
33+
|<<esql-for-search-kql,Kibana Query Language>>
34+
|8.18/9.0
35+
|Use Kibana Query Language with <<esql-kql,`kql`>> function
36+
37+
|<<esql-for-search-semantic,Semantic search>>
38+
|8.18/9.0
39+
|Perform semantic searches on `semantic_text` field types
40+
41+
|<<esql-for-search-hybrid,Hybrid search>>
42+
|8.18/9.0
43+
|Combine lexical and semantic search approaches with custom weights
44+
|===
45+
46+
[[esql-for-search-full-text]]
47+
==== Full text search
48+
49+
[[esql-for-searc-match-functio-operator]]
50+
===== Match function and operator
51+
52+
ES|QL offers two syntax options for `match`, which replicate the functionality of <<query-dsl-match-query, `match`>> queries in Query DSL.
53+
54+
Use the compact operator syntax (`:`) for simple text matching with default parameters.
55+
56+
[source,esql]
57+
----
58+
FROM logs | WHERE match(message, "connection error")
59+
----
60+
61+
Use the `match()` function syntax when you need to pass additional parameters:
62+
63+
[source,esql]
64+
----
65+
FROM products | WHERE match(name, "laptop", { "boost": 2.0 })
66+
----
67+
68+
[TIP]
69+
====
70+
See <<match-field-params,Match field parameters>> for more advanced options using match.
71+
====
72+
73+
[IMPORTANT]
74+
====
75+
These queries match documents but don't automatically sort by relevance. To get the most relevant results first, you need to use `METADATA _score` and sort by score. See <<esql-for-search-scoring,Relevance scoring>> for more information.
76+
====
77+
78+
[[esql-for-search-query-string]]
79+
===== Query string function (`QSTR`)
80+
81+
The <<esql-qstr,`qstr` function>> provides the same functionality as the Query DSL's `query_string` query. This is for advanced use cases, such as wildcard searches, searches across multiple fields, and more.
82+
83+
[source,esql]
84+
----
85+
FROM articles METADATA _score
86+
| WHERE QSTR("(new york city) OR (big apple)")
87+
| SORT _score DESC
88+
| LIMIT 10
89+
----
90+
91+
For complete details, refer to the <<query-dsl-query-string-query, Query DSL `query_string` docs>>.
92+
93+
[[esql-for-search-scoring]]
94+
==== Relevance scoring
95+
96+
To get the most relevant results first, you need to retrieve and sort by relevance score:
97+
98+
[source,esql]
99+
----
100+
FROM books METADATA _score
101+
| WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare")
102+
| SORT _score DESC
103+
----
104+
105+
When working with relevance scoring in ES|QL, it's important to understand how `_score` works:
106+
107+
* If you don't include `METADATA _score` in your query, this only performs filtering operations with no relevance calculation.
108+
* When you include `METADATA _score`, search functions included in `WHERE` conditions contribute to the relevance score.
109+
* Filtering operations (like range conditions and exact matches) don't affect the score.
110+
* Including `METADATA _score` doesn't automatically sort your results by relevance. You must explicitly use `SORT _score DESC` or `SORT _score ASC` to order your results by relevance.
111+
112+
[[esql-for-search-semantic]]
113+
==== Semantic search
114+
115+
You can perform semantic searches over <<semantic-text, `semantic_text`>> field types using the same match syntax as full-text search.
116+
117+
This example uses the match operator `:``:
118+
119+
[source,esql]
120+
----
121+
FROM articles METADATA _score
122+
| WHERE semantic_content:"What are the impacts of climate change on agriculture?"
123+
| SORT _score DESC
124+
----
125+
126+
This example uses the match function:
127+
128+
[source,esql]
129+
----
130+
FROM articles METADATA _score
131+
| WHERE match(semantic_content, "What are the impacts of climate change on agriculture?")
132+
| SORT _score DESC
133+
----
134+
135+
[[esql-for-search-hybrid]]
136+
==== Hybrid search
137+
138+
Combine traditional and semantic search with custom weights:
139+
140+
[source,esql]
141+
----
142+
FROM books METADATA _score
143+
| WHERE match(semantic_title, "fantasy adventure", { "boost": 0.75 })
144+
OR match(title, "fantasy adventure", { "boost": 0.25 })
145+
| SORT _score DESC
146+
----
147+
148+
[[esql-for-search-kql]]
149+
==== Kibana Query Language (KQL)
150+
151+
Use the <<esql-kql,KQL function>> to use the {kibana-ref}/kuery-query.html[Kibana Query Language] in your ES|QL queries:
152+
153+
[source,esql]
154+
----
155+
FROM logs*
156+
| WHERE KQL("http.request.method:GET AND agent.type:filebeat")
157+
----
158+
159+
The `kql` function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to ES|QL without needing to rewrite them all at once.
160+
161+
[[esql-for-search-limitations]]
162+
==== Limitations
163+
164+
Refer to <<esql-limitations-full-text-search, {esql} limitations>> for a list of known limitations.
165+
166+
[[esql-for-search-next-steps]]
167+
==== Next steps
168+
169+
[[esql-for-search-tutorials]]
170+
===== Tutorials and how-to guides
171+
172+
// TODO * <<esql-search-tutorial>>: Hands-on tutorial for getting started with search tools in {esql}
173+
* <<semantic-search-semantic-text>>: Learn how to use the `semantic_text` field type
174+
175+
[[esql-for-search-reference]]
176+
===== Technical reference
177+
178+
* <<esql-search-functions>>: Complete reference for all search functions
179+
* <<esql-limitations-full-text-search>>: Current limitations for search in ES|QL
180+
181+
[[esql-for-search-concepts]]
182+
===== Background concepts
183+
184+
* <<analysis>>: How text is processed for full-text search
185+
* <<semantic-search>>: Overview of semantic search in Elasticsearch
186+
187+
[[esql-for-search-blogs]]
188+
===== Related blog posts
189+
190+
// TODO* https://www.elastic.co/blog/esql-you-know-for-search-scoring-semantic-search[ES|QL, you know for Search] - Introducing scoring and semantic search
191+
* https://www.elastic.co/blog/introducing-full-text-filtering-with-esql[Introducing full text filtering in ES|QL] - Overview of text filtering capabilities

‎docs/reference/esql/esql-language.asciidoc‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,10 @@ Detailed reference documentation for the {esql} language:
1111
* <<esql-functions>>
1212
* <<esql-metadata-fields>>
1313
* <<esql-multivalued-fields>>
14-
* <<esql-enrich-data>>
14+
* <<esql-for-search>>
1515
* <<esql-lookup-join>>
16+
17+
* <<esql-enrich-data>>
1618
* <<esql-process-data-with-dissect-and-grok>>
1719
* <<esql-implicit-casting>>
1820
* <<esql-time-spans>>
@@ -22,8 +24,9 @@ include::esql-commands.asciidoc[]
2224
include::esql-functions-operators.asciidoc[]
2325
include::metadata-fields.asciidoc[]
2426
include::multivalued-fields.asciidoc[]
27+
include::esql-for-search.asciidoc[]
2528
include::esql-process-data-with-dissect-grok.asciidoc[]
26-
include::esql-enrich-data.asciidoc[]
2729
include::esql-lookup-join.asciidoc[]
30+
include::esql-enrich-data.asciidoc[]
2831
include::implicit-casting.asciidoc[]
2932
include::time-spans.asciidoc[]

0 commit comments

Comments
 (0)