Skip to content

Commit cc0fd52

Browse files
committed
Explain how to use commands in dev tools, simplify examples accordingly
1 parent 73d3d6c commit cc0fd52

File tree

1 file changed

+89
-123
lines changed

1 file changed

+89
-123
lines changed

solutions/search/esql-full-text-filter-tutorial.md

Lines changed: 89 additions & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,31 @@ Want to get started quickly? Run the following command in your terminal to set u
2222
curl -fsSL https://elastic.co/start-local | sh
2323
```
2424

25+
## Running {{esql}} queries
26+
27+
In this tutorial, you'll see {{esql}} examples in the following format:
28+
29+
```esql
30+
FROM cooking_blog
31+
| WHERE description:"fluffy pancakes"
32+
| LIMIT 1000
33+
```
34+
35+
If you want to run these queries in the [Dev Tools Console](/explore-analyze/query-filter/languages/esql-rest.md#esql-kibana-console), you'll need use the following syntax:
36+
37+
```console
38+
POST /_query?format=txt
39+
{
40+
"query": """
41+
FROM cooking_blog
42+
| WHERE description:"fluffy pancakes"
43+
| LIMIT 1000
44+
"""
45+
}
46+
```
47+
48+
If you'd prefer to use your favorite programming language, refer to [Client libraries](/solutions/search/site-or-app/clients.md) for a list of official and community-supported clients.
49+
2550
## Step 1: Create an index
2651

2752
Create the `cooking_blog` index to get started:
@@ -133,14 +158,9 @@ Both are equivalent and can be used interchangeably. The compact syntax is more
133158
Here's how to search the `description` field for "fluffy pancakes":
134159

135160
```esql
136-
POST /_query?format=txt
137-
{
138-
"query": """
139-
FROM cooking_blog # Specify the index to search
140-
| WHERE description:"fluffy pancakes" # Full-text search with OR logic by default
141-
| LIMIT 1000 # Return up to 1000 results
142-
"""
143-
}
161+
FROM cooking_blog # Specify the index to search
162+
| WHERE description:"fluffy pancakes" # Full-text search with OR logic by default
163+
| LIMIT 1000 # Return up to 1000 results
144164
```
145165

146166
By default, like the Query DSL `match` query, {{esql}} uses `OR` logic between terms. This means it will match documents that contain either "fluffy" or "pancakes", or both, in the description field.
@@ -149,15 +169,10 @@ By default, like the Query DSL `match` query, {{esql}} uses `OR` logic between t
149169
You can control which fields to include in the response using the `KEEP` command:
150170

151171
```esql
152-
POST /_query?format=txt
153-
{
154-
"query": """
155-
FROM cooking_blog
156-
| WHERE description:"fluffy pancakes"
157-
| KEEP title, description, rating # Select only specific fields to include in response
158-
| LIMIT 1000
159-
"""
160-
}
172+
FROM cooking_blog
173+
| WHERE description:"fluffy pancakes"
174+
| KEEP title, description, rating # Select only specific fields to include in response
175+
| LIMIT 1000
161176
```
162177
:::
163178

@@ -166,14 +181,9 @@ POST /_query?format=txt
166181
Sometimes you need to require that all search terms appear in the matching documents. Here's how to do that using the function syntax with the `operator` parameter:
167182

168183
```esql
169-
POST /_query?format=txt
170-
{
171-
"query": """
172-
FROM cooking_blog
173-
| WHERE match(description, "fluffy pancakes", {"operator": "AND"}) # Require ALL terms to match instead of default OR
174-
| LIMIT 1000
175-
"""
176-
}
184+
FROM cooking_blog
185+
| WHERE match(description, "fluffy pancakes", {"operator": "AND"}) # Require ALL terms to match instead of default OR
186+
| LIMIT 1000
177187
```
178188

179189
This stricter search returns *zero hits* on our sample data, as no document contains both "fluffy" and "pancakes" in the description.
@@ -183,14 +193,9 @@ This stricter search returns *zero hits* on our sample data, as no document cont
183193
Sometimes requiring all terms is too strict, but the default OR behavior is too lenient. You can specify a minimum number of terms that must match:
184194

185195
```esql
186-
POST /_query?format=txt
187-
{
188-
"query": """
189-
FROM cooking_blog
190-
| WHERE match(title, "fluffy pancakes breakfast", {"minimum_should_match": 2}) # At least 2 of the 3 terms must match
191-
| LIMIT 1000
192-
"""
193-
}
196+
FROM cooking_blog
197+
| WHERE match(title, "fluffy pancakes breakfast", {"minimum_should_match": 2}) # At least 2 of the 3 terms must match
198+
| LIMIT 1000
194199
```
195200

196201
This query searches the title field to match at least 2 of the 3 terms: "fluffy", "pancakes", or "breakfast".
@@ -200,33 +205,23 @@ This query searches the title field to match at least 2 of the 3 terms: "fluffy"
200205
When users enter a search query, they often don't know (or care) whether their search terms appear in a specific field. {{esql}} provides ways to search across multiple fields simultaneously:
201206

202207
```esql
203-
POST /_query?format=txt
204-
{
205-
"query": """
206-
FROM cooking_blog
207-
| WHERE title:"vegetarian curry" OR description:"vegetarian curry" OR tags:"vegetarian curry" # Search across different fields with equal importance
208-
| LIMIT 1000
209-
"""
210-
}
208+
FROM cooking_blog
209+
| WHERE title:"vegetarian curry" OR description:"vegetarian curry" OR tags:"vegetarian curry" # Search across different fields with equal importance
210+
| LIMIT 1000
211211
```
212212

213213
This query searches for "vegetarian curry" across the title, description, and tags fields. Each field is treated with equal importance.
214214

215215
However, in many cases, matches in certain fields (like the title) might be more relevant than others. We can adjust the importance of each field using scoring:
216216

217217
```esql
218-
POST /_query?format=txt
219-
{
220-
"query": """
221-
FROM cooking_blog METADATA _score # Request _score metadata for relevance-based results
222-
| WHERE match(title, "vegetarian curry", {"boost": 2.0}) # Title matches are twice as important
223-
OR match(description, "vegetarian curry")
224-
OR match(tags, "vegetarian curry")
225-
| KEEP title, description, tags, _score # Include relevance score in results
226-
| SORT _score DESC # You must explicitly sort by `_score` to see relevance-based results
227-
| LIMIT 1000
228-
"""
229-
}
218+
FROM cooking_blog METADATA _score # Request _score metadata for relevance-based results
219+
| WHERE match(title, "vegetarian curry", {"boost": 2.0}) # Title matches are twice as important
220+
OR match(description, "vegetarian curry")
221+
OR match(tags, "vegetarian curry")
222+
| KEEP title, description, tags, _score # Include relevance score in results
223+
| SORT _score DESC # You must explicitly sort by `_score` to see relevance-based results
224+
| LIMIT 1000
230225
```
231226

232227
In this example, we're using the `boost` parameter to make matches in the title field twice as important as matches in other fields. We also request the `_score` metadata field to sort results by relevance.
@@ -244,16 +239,11 @@ Remember that including `METADATA _score` doesn't automatically sort your result
244239
Filtering allows you to narrow down your search results based on exact criteria. Unlike full-text searches, filters are binary (yes/no) and do not affect the relevance score. Filters execute faster than queries because excluded results don't need to be scored.
245240

246241
```esql
247-
POST /_query?format=txt
248-
{
249-
"query": """
250-
FROM cooking_blog
251-
| WHERE category.keyword == "Breakfast" # Exact match using keyword field (case-sensitive)
252-
| KEEP title, author, rating, tags
253-
| SORT rating DESC
254-
| LIMIT 1000
255-
"""
256-
}
242+
FROM cooking_blog
243+
| WHERE category.keyword == "Breakfast" # Exact match using keyword field(case-sensitive)
244+
| KEEP title, author, rating, tags
245+
| SORT rating DESC
246+
| LIMIT 1000
257247
```
258248

259249
Note the use of `category.keyword` here. This refers to the [`keyword`](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md) multi-field of the `category` field, ensuring an exact, case-sensitive match.
@@ -263,32 +253,22 @@ Note the use of `category.keyword` here. This refers to the [`keyword`](elastics
263253
Often users want to find content published within a specific time frame:
264254

265255
```esql
266-
POST /_query?format=txt
267-
{
268-
"query": """
269-
FROM cooking_blog
270-
| WHERE date >= "2023-05-01" AND date <= "2023-05-31" # Inclusive date range filter
271-
| KEEP title, author, date, rating
272-
| LIMIT 1000
273-
"""
274-
}
256+
FROM cooking_blog
257+
| WHERE date >= "2023-05-01" AND date <= "2023-05-31" # Inclusive date range filter
258+
| KEEP title, author, date, rating
259+
| LIMIT 1000
275260
```
276261

277262
### Find exact matches
278263

279264
Sometimes users want to search for exact terms to eliminate ambiguity in their search results:
280265

281266
```esql
282-
POST /_query?format=txt
283-
{
284-
"query": """
285-
FROM cooking_blog
286-
| WHERE author.keyword == "Maria Rodriguez" # Exact match on author
287-
| KEEP title, author, rating, tags
288-
| SORT rating DESC
289-
| LIMIT 1000
290-
"""
291-
}
267+
FROM cooking_blog
268+
| WHERE author.keyword == "Maria Rodriguez" # Exact match on author
269+
| KEEP title, author, rating, tags
270+
| SORT rating DESC
271+
| LIMIT 1000
292272
```
293273

294274
Like the `term` query in Query DSL, this has zero flexibility and is case-sensitive.
@@ -298,41 +278,32 @@ Like the `term` query in Query DSL, this has zero flexibility and is case-sensit
298278
Complex searches often require combining multiple search criteria:
299279

300280
```esql
301-
POST /_query?format=txt
302-
{
303-
"query": """
304-
FROM cooking_blog METADATA _score
305-
| WHERE rating >= 4.5 # Numerical filter
306-
AND NOT category.keyword == "Dessert" # Exclusion filter
307-
AND (title:"curry spicy" OR description:"curry spicy") # Full-text search in multiple fields
308-
| SORT _score DESC
309-
| KEEP title, author, rating, tags, description
310-
| LIMIT 1000
311-
"""
312-
}
281+
FROM cooking_blog METADATA _score
282+
| WHERE rating >= 4.5 # Numerical filter
283+
AND NOT category.keyword == "Dessert" # Exclusion filter
284+
AND (title:"curry spicy" OR description:"curry spicy") # Full-text search in multiple fields
285+
| SORT _score DESC
286+
| KEEP title, author, rating, tags, description
287+
| LIMIT 1000
313288
```
314289

315290
### Combine relevance scoring with custom criteria
316291

317292
For more complex relevance scoring with combined criteria, you can use the `EVAL` command to calculate custom scores:
318293

319294
```esql
320-
POST /_query?format=txt
321-
{
322-
"query": """
323-
FROM cooking_blog METADATA _score
324-
| EVAL tags_concat = MV_CONCAT(tags.keyword, ",") # Convert multi-value field to string
325-
| WHERE tags_concat LIKE "*vegetarian*" AND rating >= 4.5 # Wildcard pattern matching
326-
| WHERE match(title, "curry spicy", {"boost": 2.0}) OR match(description, "curry spicy")
327-
| EVAL category_boost = CASE(category.keyword == "Main Course", 1.0, 0.0) # Conditional boost
328-
| EVAL date_boost = CASE(DATE_DIFF("month", date, NOW()) <= 1, 0.5, 0.0) # Boost recent content
329-
| EVAL custom_score = _score + category_boost + date_boost # Combine scores
330-
| WHERE NOT category.keyword == "Dessert"
331-
| WHERE custom_score > 0 # Filter based on custom score
332-
| SORT custom_score DESC
333-
| LIMIT 1000
334-
"""
335-
}
295+
FROM cooking_blog METADATA _score
296+
| EVAL tags_concat = MV_CONCAT(tags.keyword, ",") # Convert multi-value field to string
297+
| WHERE tags_concat LIKE "*vegetarian*" AND rating >= 4.5 # Wildcard pattern matching
298+
| WHERE match(title, "curry spicy", {"boost": 2.0}) OR match(description, "curry spicy")
299+
| EVAL category_boost = CASE(category.keyword == "Main Course", 1.0, 0.0) # Conditional boost
300+
| EVAL date_boost = CASE(DATE_DIFF("month", date, NOW()) <= 1, 0.5, 0.0) # Boost recent content
301+
| EVAL custom_score = _score + category_boost + date_boost # Combine scores
302+
| WHERE NOT category.keyword == "Dessert"
303+
| WHERE custom_score > 0 # Filter based on custom score
304+
| SORT custom_score DESC
305+
| LIMIT 1000
306+
336307
```
337308

338309
## Bonus: Semantic search with ES|QL
@@ -367,29 +338,24 @@ POST /cooking_blog/_doc
367338

368339
Once the document has been processed by the underlying model running on the inference endpoint, you can perform semantic searches. Here's an example natural language query against the `semantic_description` field:
369340

370-
```console
371-
POST /_query?format=txt
372-
{
373-
"query": """
374-
FROM cooking_blog METADATA _score
375-
| WHERE semantic_description:"What are some easy to prepare but nutritious plant-based meals?"
376-
| SORT _score DESC
377-
| LIMIT 5
378-
"""
379-
}
341+
```esql
342+
FROM cooking_blog METADATA _score
343+
| WHERE semantic_description:"What are some easy to prepare but nutritious plant-based meals?"
344+
| SORT _score DESC
345+
| LIMIT 5
346+
380347
```
381348

382349
:::{tip}
383-
Follow this [tutorial](/solutions/search/semantic-search/semantic-search-semantic-text.md) if you'd like to test this workflow against a large dataset.
350+
Follow this [tutorial](/solutions/search/semantic-search/semantic-search-semantic-text.md) if you'd like to test out the semantic search workflow against a large dataset.
384351
:::
385352

386353
## Learn more
387354

388-
This tutorial introduced the basics of full-text search and filtering in {{esql}}. Building a real-world search experience requires understanding many more advanced concepts and techniques. Here are some resources once you're ready to dive deeper:
355+
This tutorial introduced the basics of search and filtering in {{esql}}. Building a real-world search experience requires understanding many more advanced concepts and techniques. Here are some resources once you're ready to dive deeper:
389356

390357
- [Full-text search](full-text.md): Learn about the core components of full-text search in Elasticsearch.
391358
- [Text analysis](full-text/text-analysis-during-search.md): Understand how text is processed for full-text search.
392359
- [{{esql}} search functions](elasticsearch://reference/query-languages/esql/esql-functions-operators.md#esql-search-functions): Explore the full list of search functions available in {{esql}}.
393-
% semantic search stuff
394360
- [Semantic search](/solutions/search/semantic-search.md): Understand your various options for semantic search in Elasticsearch.
395361
- [The `semantic_text` workflow](/solutions/search/semantic-search.md#_semantic_text_workflow): Learn how to use the `semantic_text` field type for semantic search. This is the recommended approach for must users looking to perform semantic search in {{es}}, because it abstracts away the complexity of setting up inference endpoints and models.

0 commit comments

Comments
 (0)