Skip to content

ESQL: Fix docs for RERANK #132534

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 7, 2025
Merged

ESQL: Fix docs for RERANK #132534

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
939ae8d
ESQL: Fix docs for RERANK
nik9000 Aug 7, 2025
c62f010
Merge branch 'main' into esql_rerank_docs
nik9000 Aug 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions ...nce/query-languages/esql/_snippets/commands/examples/rerank.csv-spec/combine.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
% This is generated by ESQL's AbstractFunctionTestCase. Do not edit it. See ../README.md for how to regenerate it.

```esql
FROM books METADATA _score
| WHERE MATCH(description, "hobbit") OR MATCH(author, "Tolkien")
| SORT _score DESC
| LIMIT 100
| RERANK rerank_score = "hobbit" ON description, author WITH { "inference_id" : "test_reranker" }
| EVAL original_score = _score, _score = rerank_score + original_score
| SORT _score
| LIMIT 3
| KEEP title, original_score, rerank_score, _score
```

| title:text | _score:double | rerank_score:double | rerank_score:double |
| --- | --- | --- | --- |
| Poems from the Hobbit | 4.012462615966797 | 0.001396648003719747 | 0.001396648003719747 |
| The Lord of the Rings - Boxed Set | 3.768855094909668 | 0.0010020040208473802 | 0.001396648003719747 |
| Return of the King Being the Third Part of The Lord of the Rings | 3.6248698234558105 | 9.000900317914784E-4 | 0.001396648003719747 |
17 changes: 17 additions & 0 deletions ...uery-languages/esql/_snippets/commands/examples/rerank.csv-spec/simple-query.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
% This is generated by ESQL's AbstractFunctionTestCase. Do not edit it. See ../README.md for how to regenerate it.

```esql
FROM books METADATA _score
| WHERE MATCH(description, "hobbit")
| SORT _score DESC
| LIMIT 100
| RERANK "hobbit" ON description WITH { "inference_id" : "test_reranker" }
| LIMIT 3
| KEEP title, _score
```

| title:text | _score:double |
| --- | --- |
| Poems from the Hobbit | 0.0015673980815336108 |
| A Tolkien Compass: Including J. R. R. Tolkien's Guide to the Names in The Lord of the Rings | 0.007936508394777775 |
| Return of the King Being the Third Part of The Lord of the Rings | 9.960159659385681E-4 |
18 changes: 18 additions & 0 deletions ...query-languages/esql/_snippets/commands/examples/rerank.csv-spec/two-queries.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
% This is generated by ESQL's AbstractFunctionTestCase. Do not edit it. See ../README.md for how to regenerate it.

```esql
FROM books METADATA _score
| WHERE MATCH(description, "hobbit") OR MATCH(author, "Tolkien")
| SORT _score DESC
| LIMIT 100
| RERANK rerank_score = "hobbit" ON description, author WITH { "inference_id" : "test_reranker" }
| SORT rerank_score
| LIMIT 3
| KEEP title, _score, rerank_score
```

| title:text | _score:double | rerank_score:double |
| --- | --- | --- |
| Return of the Shadow | 2.8181066513061523 | 5.740527994930744E-4 |
| Return of the King Being the Third Part of The Lord of the Rings | 3.6248698234558105 | 9.000900317914784E-4 |
| The Lays of Beleriand | 1.3002015352249146 | 9.36329597607255E-4 |
56 changes: 6 additions & 50 deletions docs/reference/query-languages/esql/_snippets/commands/layout/rerank.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,61 +100,17 @@ If you don't want to increase the timeout limit, try the following:

Rerank search results using a simple query and a single field:

```esql
FROM books
| WHERE MATCH(title, "science fiction")
| SORT _score DESC
| LIMIT 100
| RERANK "science fiction" ON (title) WITH { "inference_id" : "my_reranker" }
| LIMIT 3
| KEEP title, _score
```

| title:keyword | _score:double |
|---------------|---------------|
| Neuromancer | 0.98 |
| Dune | 0.95 |
| Foundation | 0.92 |
:::{include} ../examples/rerank.csv-spec/simple-query.md
:::

Rerank search results using a query and multiple fields, and store the new score
in a column named `rerank_score`:

```esql
FROM movies
| WHERE MATCH(title, "dystopian future") OR MATCH(synopsis, "dystopian future")
| SORT _score DESC
| LIMIT 100
| RERANK rerank_score = "dystopian future" ON (title, synopsis) WITH { "inference_id" : "my_reranker" }
| SORT rerank_score DESC
| LIMIT 5
| KEEP title, _score, rerank_score
```

| title:keyword | _score:double | rerank_score:double |
|-----------------|---------------|---------------------|
| Blade Runner | 8.75 | 0.99 |
| The Matrix | 9.12 | 0.97 |
| Children of Men | 8.50 | 0.96 |
| Akira | 8.99 | 0.94 |
| Gattaca | 8.65 | 0.91 |
:::{include} ../examples/rerank.csv-spec/two-queries.md
:::

Combine the original score with the reranked score:

```esql
FROM movies
| WHERE MATCH(title, "dystopian future") OR MATCH(synopsis, "dystopian future")
| SORT _score DESC
| LIMIT 100
| RERANK rerank_score = "dystopian future" ON (title, synopsis) WITH { "inference_id" : "my_reranker" }
| EVAL original_score = _score, _score = rerank_score + original_score
| SORT _score DESC
| LIMIT 2
| KEEP title, original_score, rerank_score, _score
```

| title:keyword | original_score:double | rerank_score:double | _score:double |
|---------------|-----------------------|---------------------|---------------|
| The Matrix | 9.12 | 0.97 | 10.09 |
| Akira | 8.99 | 0.94 | 9.93 |


:::{include} ../examples/rerank.csv-spec/combine.md
:::
71 changes: 71 additions & 0 deletions x-pack/plugin/esql/qa/testFixtures/src/main/resources/rerank.csv-spec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,3 +223,74 @@ book_no:keyword | title:keyword | author:keyword
5335 | Letters of J R R Tolkien | J.R.R. Tolkien | 0.04
2130 | The J. R. R. Tolkien Audio Collection | [Christopher Tolkien, John Ronald Reuel Tolkien] | 0.03
;


simple
required_capability: rerank

// tag::simple-query[]
FROM books METADATA _score
| WHERE MATCH(description, "hobbit")
| SORT _score DESC
| LIMIT 100
| RERANK "hobbit" ON description WITH { "inference_id" : "test_reranker" }
| LIMIT 3
| KEEP title, _score
// end::simple-query[]
;

// tag::simple-query-result[]
title:text | _score:double
Poems from the Hobbit | 0.0015673980815336108
A Tolkien Compass: Including J. R. R. Tolkien's Guide to the Names in The Lord of the Rings | 0.007936508394777775
Return of the King Being the Third Part of The Lord of the Rings | 9.960159659385681E-4
// end::simple-query-result[]
;

two_queries
required_capability: rerank

// tag::two-queries[]
FROM books METADATA _score
| WHERE MATCH(description, "hobbit") OR MATCH(author, "Tolkien")
| SORT _score DESC
| LIMIT 100
| RERANK rerank_score = "hobbit" ON description, author WITH { "inference_id" : "test_reranker" }
| SORT rerank_score
| LIMIT 3
| KEEP title, _score, rerank_score
// end::two-queries[]
;

// tag::two-queries-result[]
title:text | _score:double | rerank_score:double
Return of the Shadow | 2.8181066513061523 | 5.740527994930744E-4
Return of the King Being the Third Part of The Lord of the Rings | 3.6248698234558105 | 9.000900317914784E-4
The Lays of Beleriand | 1.3002015352249146 | 9.36329597607255E-4
// end::two-queries-result[]
;

combine
required_capability: rerank
required_capability: rerank_combine

// tag::combine[]
FROM books METADATA _score
| WHERE MATCH(description, "hobbit") OR MATCH(author, "Tolkien")
| SORT _score DESC
| LIMIT 100
| RERANK rerank_score = "hobbit" ON description, author WITH { "inference_id" : "test_reranker" }
| EVAL original_score = _score, _score = rerank_score + original_score
| SORT _score
| LIMIT 3
| KEEP title, original_score, rerank_score, _score
// end::combine[]
;

// tag::combine-result[]
title:text | _score:double | rerank_score:double | rerank_score:double
Poems from the Hobbit | 4.012462615966797 | 0.001396648003719747 | 0.001396648003719747
The Lord of the Rings - Boxed Set | 3.768855094909668 | 0.0010020040208473802 | 0.001396648003719747
Return of the King Being the Third Part of The Lord of the Rings | 3.6248698234558105 | 9.000900317914784E-4 | 0.001396648003719747
// end::combine-result[]
;
Loading