Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -424,5 +424,5 @@ The `constant_keyword` is not strictly required for this optimization: it is als

## Default search timeout [_default_search_timeout]

By default, search requests don’t time out. You can set a timeout using the [`search.default_search_timeout`](../../../solutions/search/querying-for-search.md#search-timeout) setting.
By default, search requests don’t time out. You can set a timeout using the [`search.default_search_timeout`](../../../solutions/search/the-search-api.md#search-timeout) setting.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ To help you manage your data, {{es}} offers you the following options:
* Define [multiple tiers](../../../manage-data/lifecycle/data-tiers.md) of data nodes with different performance characteristics.
* Automatically transition indices through the data tiers according to your performance needs and retention policies.
* Leverage [searchable snapshots](../../../deploy-manage/tools/snapshot-and-restore/searchable-snapshots.md) stored in a remote repository to provide resiliency for your older indices while reducing operating costs and maintaining search performance.
* Perform [asynchronous searches](../../../solutions/search/search-approaches/long-running-searches.md) of data stored on less-performant hardware.
* Perform [asynchronous searches](../../../solutions/search/async-search-api.md) of data stored on less-performant hardware.

**Data stream lifecycle** is less feature rich but is focused on simplicity. It allows you to do the following:

Expand Down
24 changes: 0 additions & 24 deletions raw-migrated-files/kibana/kibana/inference-endpoints.md

This file was deleted.

4 changes: 0 additions & 4 deletions raw-migrated-files/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -270,8 +270,6 @@ toc:
- file: docs-content/serverless/elasticsearch-ingest-data-through-api.md
- file: docs-content/serverless/elasticsearch-ingest-your-data.md
- file: docs-content/serverless/elasticsearch-manage-project.md
- file: docs-content/serverless/elasticsearch-search-your-data-the-search-api.md
- file: docs-content/serverless/elasticsearch-search-your-data.md
- file: docs-content/serverless/endpoint-protection-rules.md
- file: docs-content/serverless/general-billing-stop-project.md
- file: docs-content/serverless/general-manage-organization.md
Expand Down Expand Up @@ -607,7 +605,6 @@ toc:
- file: elasticsearch/elasticsearch-reference/scalability.md
- file: elasticsearch/elasticsearch-reference/search-analyze.md
- file: elasticsearch/elasticsearch-reference/search-with-synonyms.md
- file: elasticsearch/elasticsearch-reference/search-your-data.md
- file: elasticsearch/elasticsearch-reference/secure-cluster.md
- file: elasticsearch/elasticsearch-reference/secure-monitoring.md
- file: elasticsearch/elasticsearch-reference/secure-settings.md
Expand Down Expand Up @@ -645,7 +642,6 @@ toc:
- file: kibana/kibana/elasticsearch-mutual-tls.md
- file: kibana/kibana/esql.md
- file: kibana/kibana/get-started.md
- file: kibana/kibana/inference-endpoints.md
- file: kibana/kibana/install.md
- file: kibana/kibana/kibana-authentication.md
- file: kibana/kibana/kibana-concepts-analysts.md
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search-intro.html
navigation_title: The `async-search` API
---

# Long-running searches [async-search-intro]
# Long running searches with the Async Search API [async-search-intro]

{{es}} generally allows you to quickly search across big amounts of data. There are situations where a search executes on many shards, possibly against large data sets or multiple [remote clusters](../../../deploy-manage/remote-clusters.md), for which results are not expected to be returned in milliseconds. When you need to execute long-running searches, synchronously waiting for its results to be returned is not ideal. Instead, Async search lets you submit a search request that gets executed *asynchronously*, monitor the progress of the request, and retrieve results at a later stage. You can also retrieve partial results as they become available but before the search has completed.
{{es}} generally allows you to quickly search across big amounts of data. There are situations where a search executes on many shards, possibly against large data sets or multiple [remote clusters](../../deploy-manage/remote-clusters.md), for which results are not expected to be returned in milliseconds. When you need to execute long-running searches, synchronously waiting for its results to be returned is not ideal. Instead, Async search lets you submit a search request that gets executed *asynchronously*, monitor the progress of the request, and retrieve results at a later stage. You can also retrieve partial results as they become available but before the search has completed.

You can submit an async search request using the [submit async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#submit-async-search) API. The [get async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#get-async-search) API allows you to monitor the progress of an async search request and retrieve its results. An ongoing async search can be deleted through the [delete async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#delete-async-search) API.

4 changes: 1 addition & 3 deletions solutions/search/elasticsearch-basics-quickstart.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# {{es}} API quickstart [getting-started]
# Basics quickstart [getting-started]


This quick start guide is a hands-on introduction to the fundamental concepts of Elasticsearch: [indices, documents and field type mappings](../../manage-data/data-store/index-basics.md).
Expand All @@ -10,8 +10,6 @@ The code examples in this tutorial are in [Console](../../explore-analyze/query-

::::



## Requirements [getting-started-requirements]

You’ll need a running {{es}} cluster, together with {{kib}} to use the Dev Tools API Console. Run the following command in your terminal to set up a [single-node local cluster in Docker](get-started.md):
Expand Down
16 changes: 0 additions & 16 deletions solutions/search/examples.md

This file was deleted.

38 changes: 3 additions & 35 deletions solutions/search/full-text.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,42 +15,10 @@ Built on decades of information retrieval research, full-text search delivers re

You can combine full-text search with [semantic search using vectors](semantic-search.md) to build modern hybrid search applications. While vector search may require additional GPU resources, the full-text component remains cost-effective by leveraging existing CPU infrastructure.


## How full-text search works [full-text-search-how-it-works]

The following diagram illustrates the components of full-text search.

:::{image} ../../images/elasticsearch-reference-full-text-search-overview.svg
:alt: Components of full-text search from analysis to relevance scoring
:::

At a high level, full-text search involves the following:

* [**Text analysis**](../../manage-data/data-store/text-analysis.md): Analysis consists of a pipeline of sequential transformations. Text is transformed into a format optimized for searching using techniques such as stemming, lowercasing, and stop word elimination. {{es}} contains a number of built-in [analyzers](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-analyzers.html) and tokenizers, including options to analyze specific language text. You can also create custom analyzers.

::::{tip}
Refer to [Test an analyzer](../../manage-data/data-store/text-analysis/test-an-analyzer.md) to learn how to test an analyzer and inspect the tokens and metadata it generates.

::::

* **Inverted index creation**: After analysis is complete, {{es}} builds an inverted index from the resulting tokens. An inverted index is a data structure that maps each token to the documents that contain it. It’s made up of two key components:

* **Dictionary**: A sorted list of all unique terms in the collection of documents in your index.
* **Posting list**: For each term, a list of document IDs where the term appears, along with optional metadata like term frequency and position.

* **Relevance scoring**: Results are ranked by how relevant they are to the given query. The relevance score of each document is represented by a positive floating-point number called the `_score`. The higher the `_score`, the more relevant the document.

The default [similarity algorithm](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html) {{es}} uses for calculating relevance scores is [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25), a variation of the [TF-IDF algorithm](https://en.wikipedia.org/wiki/Tf–idf). BM25 calculates relevance scores based on term frequency, document frequency, and document length. Refer to this [technical blog post](https://www.elastic.co/blog/practical-bm25-part-2-the-bm25-algorithm-and-its-variables) for a deep dive into BM25.

* **Full-text search query**: Query text is analyzed [the same way as the indexed text](../../manage-data/data-store/text-analysis/index-search-analysis.md), and the resulting tokens are used to search the inverted index.

Query DSL supports a number of [full-text queries](https://www.elastic.co/guide/en/elasticsearch/reference/current/full-text-queries.html).

As of 8.17, {{esql}} also supports [full-text search](https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-functions-operators.html#esql-search-functions) functions.

## Getting started [full-text-search-getting-started]


## Getting started [full-text-search-getting-started]
For a high-level overview of how full-text search works, refer to [How full-text search works](full-text/how-full-text-works.md).

For a hands-on introduction to full-text search, refer to the [full-text search tutorial](get-started.md).

Expand Down Expand Up @@ -80,5 +48,5 @@ Learn how to build full-text search queries using {{es}}'s query languages:

For a technical deep dive into {{es}}'s BM25 implementation read this blog post: [The BM25 Algorithm and its Variables](https://www.elastic.co/blog/practical-bm25-part-2-the-bm25-algorithm-and-its-variables).

To learn how to optimize the relevance of your search results, refer to [Search relevance optimizations](examples.md).
To learn how to optimize the relevance of your search results, refer to [Search relevance optimizations](full-text/search-relevance.md).

30 changes: 30 additions & 0 deletions solutions/search/full-text/how-full-text-works.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# How full-text search works [full-text-search-how-it-works]

The following diagram illustrates the components of full-text search.

:::{image} ../../../images/elasticsearch-reference-full-text-search-overview.svg
:alt: Components of full-text search from analysis to relevance scoring
:width: 550px
:::

At a high level, full-text search involves the following:

* [**Text analysis**](../../../manage-data/data-store/text-analysis.md): Analysis consists of a pipeline of sequential transformations. Text is transformed into a format optimized for searching using techniques such as stemming, lowercasing, and stop word elimination. {{es}} contains a number of built-in [analyzers](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-analyzers.html) and tokenizers, including options to analyze specific language text. You can also create custom analyzers.
::::{tip}
Refer to [Test an analyzer](../../../manage-data/data-store/text-analysis/test-an-analyzer.md) to learn how to test an analyzer and inspect the tokens and metadata it generates.
::::

* **Inverted index creation**: After analysis is complete, {{es}} builds an inverted index from the resulting tokens. An inverted index is a data structure that maps each token to the documents that contain it. It’s made up of two key components:

* **Dictionary**: A sorted list of all unique terms in the collection of documents in your index.
* **Posting list**: For each term, a list of document IDs where the term appears, along with optional metadata like term frequency and position.

* **Relevance scoring**: Results are ranked by how relevant they are to the given query. The relevance score of each document is represented by a positive floating-point number called the `_score`. The higher the `_score`, the more relevant the document.

The default [similarity algorithm](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html) {{es}} uses for calculating relevance scores is [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25), a variation of the [TF-IDF algorithm](https://en.wikipedia.org/wiki/Tf–idf). BM25 calculates relevance scores based on term frequency, document frequency, and document length. Refer to this [technical blog post](https://www.elastic.co/blog/practical-bm25-part-2-the-bm25-algorithm-and-its-variables) for a deep dive into BM25.

* **Full-text search query**: Query text is analyzed [the same way as the indexed text](../../../manage-data/data-store/text-analysis/index-search-analysis.md), and the resulting tokens are used to search the inverted index.

Query DSL supports a number of [full-text queries](https://www.elastic.co/guide/en/elasticsearch/reference/current/full-text-queries.html).

As of 8.17, {{esql}} also supports [full-text search](https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-functions-operators.html#esql-search-functions) functions.
15 changes: 15 additions & 0 deletions solutions/search/full-text/search-relevance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/elasticsearch/reference/current/recipes.html
---

# Search relevance optimizations

This section includes a few recipes to help with common search relevance issues:

* [Mixing exact search with stemming](search-relevance/mixing-exact-search-with-stemming.md)
* [Getting consistent scores](search-relevance/consistent-scoring.md)
* [Incorporating static relevance signals into the score](search-relevance/static-scoring-signals.md)



Loading
Loading