diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index 22b82aa65d..c3c34d2de4 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -634,7 +634,6 @@ toc: - file: elasticsearch/elasticsearch-reference/oidc-realm.md - file: elasticsearch/elasticsearch-reference/overview-index-lifecycle-management.md - file: elasticsearch/elasticsearch-reference/recovery-prioritization.md - - file: elasticsearch/elasticsearch-reference/retrievers-overview.md - file: elasticsearch/elasticsearch-reference/role-mapping-resources.md - file: elasticsearch/elasticsearch-reference/rollup-overview.md - file: elasticsearch/elasticsearch-reference/saml-guide-stack.md diff --git a/solutions/search/querying-for-search.md b/solutions/search/querying-for-search.md index a0ad1f9c30..259be30840 100644 --- a/solutions/search/querying-for-search.md +++ b/solutions/search/querying-for-search.md @@ -10,6 +10,20 @@ mapped_urls: # Query languages for search +::::{tip} +This page is focused on the search use case. For an overview of Elastic query languages for every use case, refer to the [complete overview](/explore-analyze/query-filter/languages.md). +:::: + +Once you know which [search approaches](search-approaches.md) you need to use, you can start building and testing your search queries. {{es}} provides several query languages to help you express your search logic. + +| Interface | Endpoint | Description | Best for | +|-----------|----------|-------------|----------| +| [**Query DSL**](/explore-analyze/query-filter/languages/querydsl.md) | `_search` | Original, JSON-based query language native to Elasticsearch. Powerful but complex syntax. | Full-text and semantic search queries | +| [**ESQL**](/explore-analyze/query-filter/languages/esql.md) | `_query` | Fast, SQL-like language with piped syntax, built on new compute architecture. | Filtering, analysis, aggregations | +| [**Retrievers**](retrievers-overview.md) | `_search` | Modern `_search` API syntax focused on composability. | Building complex search pipelines, especially those using semantic search. Required for [semantic reranking](ranking/semantic-reranking.md). | + +These query languages are complementary, not mutually exclusive. You can use different query languages for different parts of your application, based on your specific needs. This flexibility allows you to gradually adopt newer interfaces as your requirements evolve. + % What needs to be done: Lift-and-shift % Use migrated content from existing pages that map to this page: diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/_retrievers_examples.md b/solutions/search/retrievers-examples.md similarity index 97% rename from raw-migrated-files/elasticsearch/elasticsearch-reference/_retrievers_examples.md rename to solutions/search/retrievers-examples.md index 90f94d78b9..8f942900ae 100644 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/_retrievers_examples.md +++ b/solutions/search/retrievers-examples.md @@ -5,7 +5,7 @@ Learn how to combine different retrievers in these hands-on examples. ## Add example data [retrievers-examples-setup] -To begin with, lets create the `retrievers_example` index, and add some documents to it. We will set `number_of_shards=1` for our examples to ensure consistent and reproducible ordering. +To begin with, let's create the `retrievers_example` index, and add some documents to it. We will set `number_of_shards=1` for our examples to ensure consistent and reproducible ordering. ```console PUT retrievers_example @@ -392,7 +392,9 @@ Which would return the following results: ## Example: Grouping results by year with `collapse` [retrievers-examples-collapsing-retriever-results] -In our result set, we have many documents with the same `year` value. We can clean this up using the `collapse` parameter with our retriever. This, as with the standard [collapse](collapse-search-results.md) feature, enables grouping results by any field and returns only the highest-scoring document from each group. In this example we’ll collapse our results based on the `year` field. +In our result set, we have many documents with the same `year` value. We can clean this up using the `collapse` parameter with our retriever. This, as with the standard +% [collapse](/raw-migrated-files/elasticsearch/elasticsearch-reference/collapse-search-results.md) feature, +enables grouping results by any field and returns only the highest-scoring document from each group. In this example we’ll collapse our results based on the `year` field. ```console GET /retrievers_example/_search @@ -541,7 +543,9 @@ This returns the following response with collapsed results. ## Example: Highlighting results based on nested sub-retrievers [retrievers-examples-highlighting-retriever-results] -Highlighting is now also available for nested sub-retrievers matches. For example, consider the same `rrf` retriever as above, with a `knn` and `standard` retriever as its sub-retrievers. We can specify a `highlight` section, as defined in [highlighting](highlighting.md) documentation, and compute highlights for the top results. +Highlighting is now also available for nested sub-retrievers matches. For example, consider the same `rrf` retriever as above, with a `knn` and `standard` retriever as its sub-retrievers. We can specify a `highlight` section, +% TODO URL as defined in the [highlighting](/raw-migrated-files/elasticsearch/elasticsearch-reference/highlighting.md) documentation, +and compute highlights for the top results. ```console GET /retrievers_example/_search @@ -738,7 +742,9 @@ POST /retrievers_example_nested/_doc/3 POST /retrievers_example_nested/_refresh ``` -Now we can run an `rrf` retriever query and also compute [inner hits](inner-hits.md) for the `nested_field.nested_vector` field, based on the `knn` query specified. +Now we can run an `rrf` retriever query and also compute inner hits +% TODO URL [inner hits](/raw-migrated-files/elasticsearch/elasticsearch-reference/inner-hits.md) +for the `nested_field.nested_vector` field, based on the `knn` query specified. ```console GET /retrievers_example_nested/_search @@ -1261,7 +1267,10 @@ The output of which, albeit a bit verbose, will provide all the necessary info t ## Example: Rerank results of an RRF retriever [retrievers-examples-text-similarity-reranker-on-top-of-rrf] -To demonstrate the full functionality of retrievers, the following examples also require access to a [semantic reranking model](semantic-reranking.md#semantic-reranking-models) set up using the [Elastic inference APIs](inference-apis.md). +To demonstrate the full functionality of retrievers, the following examples also require access to a semantic reranking model +% TODO URL [semantic reranking model](/raw-migrated-files/elasticsearch/elasticsearch-reference/semantic-reranking.md#semantic-reranking-models) +set up using the inference APIs. +% [Elastic inference APIs](/raw-migrated-files/elasticsearch/elasticsearch-reference/inference-apis.md). In this example we’ll set up a reranking service and use it with the `text_similarity_reranker` retriever to rerank our top results. diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/retrievers-overview.md b/solutions/search/retrievers-overview.md similarity index 95% rename from raw-migrated-files/elasticsearch/elasticsearch-reference/retrievers-overview.md rename to solutions/search/retrievers-overview.md index 0b6652f007..7f4fc7cf69 100644 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/retrievers-overview.md +++ b/solutions/search/retrievers-overview.md @@ -14,7 +14,7 @@ Retrievers come in various types, each tailored for different search operations. * [**Linear Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#linear-retriever). Combines the top results from multiple sub-retrievers using a weighted sum of their scores. Allows to specify different weights for each retriever, as well as independently normalize the scores from each result set. * [**RRF Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#rrf-retriever). Combines and ranks multiple first-stage retrievers using the reciprocal rank fusion (RRF) algorithm. Allows you to combine multiple result sets with different relevance indicators into a single result set. An RRF retriever is a **compound retriever**, where its `filter` element is propagated to its sub retrievers. * [**Rule Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#rule-retriever). Applies [query rules](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-using-query-rules.html#query-rules) to the query before returning results. -* [**Text Similarity Re-ranker Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#text-similarity-reranker-retriever). Used for [semantic reranking](../../../solutions/search/ranking/semantic-reranking.md). Requires first creating a `rerank` task using the [{{es}} Inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html). +* [**Text Similarity Re-ranker Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#text-similarity-reranker-retriever). Used for [semantic reranking](ranking/semantic-reranking.md). Requires first creating a `rerank` task using the [{{es}} Inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html). ## What makes retrievers useful? [retrievers-overview-why-are-they-useful] @@ -126,7 +126,7 @@ The Search Playground builds Elasticsearch queries using the retriever abstracti You can use the Playground to experiment with different retriever configurations and see how they affect search results. -Refer to the [Playground documentation](../../../solutions/search/rag/playground.md) for more information. +Refer to the [Playground documentation](rag/playground.md) for more information. ## API reference [retrievers-overview-api-reference] diff --git a/solutions/toc.yml b/solutions/toc.yml index 237052b942..a3c7e84f57 100644 --- a/solutions/toc.yml +++ b/solutions/toc.yml @@ -597,6 +597,10 @@ toc: - file: search/search-approaches/long-running-searches.md - file: search/search-approaches/near-real-time-search.md - file: search/querying-for-search.md + children: + - file: search/retrievers-overview.md + children: + - file: search/retrievers-examples.md - file: search/inference-api.md children: - file: search/inference-api/elastic-inference-service-eis.md