[DOCS] Fix reranking IA, move retrievers to search api overview

docs/reference/index.asciidoc

@@ -32,6 +32,8 @@ include::alias.asciidoc[]
 
 include::search/search-your-data/search-your-data.asciidoc[]
 
+include::reranking/index.asciidoc[]
+
 include::query-dsl.asciidoc[]
 
 include::aggregations.asciidoc[]

docs/reference/reranking/index.asciidoc

@@ -0,0 +1,71 @@
+[[re-ranking-overview]]
+= Re-ranking
+
+Many search systems are built on two-stage retrieval pipelines.
+
+The first stage uses cheap, fast algorithms to find a subset of possible matches.
+
+The second stage uses a more powerful model, often machine learning-based, to reorder the documents.
+This second step is called re-ranking.
+This approach balances computational costs, because the resource-intensive model is only applied to a smaller set of pre-filtered results.
+
+{es} supports various ranking and re-ranking techniques to optimize search relevance and performance.
+
+[float]
+[[re-ranking-two-stage-pipeline]]
+== Two-stage retrieval pipelines
+
+Learn about retrieval pipelines and how re-ranking can be applied to your existing search experience.
+
+[float]
+[[re-ranking-first-stage-pipeline]]
+=== First stage: initial retrieval
+
+[float]
+[[re-ranking-ranking-overview-bm25]]
+==== Full-text search: BM25 scoring
+
+{es} ranks documents based on term frequency and inverse document frequency, adjusted for document length.
+BM25 is the default statistical scoring algorithm in {es} and works out-of-the-box.
+
+[float]
+[[re-ranking-ranking-overview-vector]]
+==== Vector search: similarity scoring
+
+Vector search involves transforming data into dense or sparse vector embeddings to capture semantic meanings, and computing similarity scores for query vectors.
+Store vectors using `semantic` fields for automatic vectorization or `dense_vector` and `sparse_vector` fields when you need more control over the underlying embedding model.
+Query vectors with `semantic` queries or `knn` and `sparse_vector` queries to compute similarity scores.
+Refer to <<semantic-search,semantic search>> for more information.
+
+[float]
+[[re-ranking-ranking-overview-hybrid]]
+==== Hybrid techniques
+
+Hybrid search techniques combine results from full-text and vector search pipelines.
+{es} enables combining lexical matching (BM25) and vector search scores using the advanced <<rrf,Reciprocal Rank Fusion (RRF)>> algorithm.
+
+[float]
+[[re-ranking-overview-second-stage]]
+=== Second Stage: Re-ranking
+
+When using the following advanced re-ranking pipelines, first-stage retrieval mechanisms effectively generate a set of candidates.
+These candidates are funneled into the re-ranker to perform more computationally expensive re-ranking tasks.
+
+[float]
+[[re-ranking-overview-semantic]]
+==== Semantic re-ranking
+
+<<semantic-reranking>> uses machine learning models to reorder search results based on their semantic similarity to a query.
+Models can be hosted directly in your {es} cluster, or you can use <<inference-apis,inference endpoints>> to call models provided by third-party services.
+Enables out-of-the-box semantic search capabilities on existing full-text search indices.
+
+[float]
+[[re-ranking-overview-ltr]]
+==== Learning to Rank (LTR)
+
+<<learning-to-rank>> is for advanced users.
+Train a machine learning model to build a ranking function for your search experience that updates over time.
+Best suited for when you have ample training data and need highly customized relevance tuning.
+
+include::semantic-reranking.asciidoc[]
+include::learning-to-rank.asciidoc[]

docs/reference/search/search-your-data/learning-to-rank.asciidoc → docs/reference/reranking/learning-to-rank.asciidoc

@@ -131,4 +131,4 @@ In the next pages of this guide you will learn to:
 * <<learning-to-rank-search-usage, Search using LTR model as a rescorer>>
 
 include::learning-to-rank-model-training.asciidoc[]
-include::learning-to-rank-search-usage.asciidoc[]
+include::learning-to-rank-search-usage.asciidoc[]

docs/reference/search/search-your-data/retrievers-reranking/semantic-reranking.asciidoc → docs/reference/reranking/semantic-reranking.asciidoc

@@ -1,5 +1,5 @@
 [[semantic-reranking]]
-=== Semantic reranking
+== Semantic reranking
 
 preview::[]
 
@@ -11,10 +11,6 @@ This overview focuses more on the high-level concepts and use cases for semantic
 Rerankers improve the relevance of results from earlier-stage retrieval mechanisms.
 _Semantic_ rerankers use machine learning models to reorder search results based on their semantic similarity to a query.
 
-First-stage retrievers and rankers must be very fast and efficient because they process either the entire corpus, or all matching documents.
-In a multi-stage pipeline, you can progressively use more computationally intensive ranking functions and techniques, as they will operate on smaller result sets at each step.
-This helps avoid query latency degradation and keeps costs manageable.
-
 Semantic reranking requires relatively large and complex machine learning models and operates in real-time in response to queries.
 This technique makes sense on a small _top-k_ result set, as one the of the final steps in a pipeline.
 This is a powerful technique for improving search relevance that works equally well with keyword, semantic, or hybrid retrieval algorithms.
@@ -24,7 +20,7 @@ The final sections include a practical, high-level overview of how to implement
 
 [discrete]
 [[semantic-reranking-use-cases]]
-==== Use cases
+=== Use cases
 
 Semantic reranking enables a variety of use cases:
 
@@ -45,7 +41,7 @@ Now that we've outlined the value of semantic reranking, we'll explore the speci
 
 [discrete]
 [[semantic-reranking-models]]
-==== Cross-encoder and bi-encoder models
+=== Cross-encoder and bi-encoder models
 
 At a high level, two model types are used for semantic reranking: cross-encoders and bi-encoders.
 
@@ -83,7 +79,7 @@ Note that similarity scores from bi-encoders/embedding similarities are _query-d
 
 [discrete]
 [[semantic-reranking-in-es]]
-==== Semantic reranking in {es}
+=== Semantic reranking in {es}
 
 In {es}, semantic rerankers are implemented using the {es} <<inference-apis,Inference API>> and a <<retriever,retriever>>.
 
@@ -134,7 +130,7 @@ POST _search
 
 [discrete]
 [[semantic-reranking-learn-more]]
-==== Learn more
+=== Learn more
 
 * Read the <<retriever,retriever reference documentation>> for syntax and implementation details
 * Learn more about the <<retrievers-overview,retrievers>> abstraction

docs/reference/search/search-your-data/search-api.asciidoc

@@ -530,3 +530,5 @@ include::retrieve-inner-hits.asciidoc[]
 include::search-shard-routing.asciidoc[]
 include::search-using-query-rules.asciidoc[]
 include::search-template.asciidoc[]
+include::retrievers-overview.asciidoc[]
+

docs/reference/search/search-your-data/search-your-data.asciidoc

@@ -45,8 +45,6 @@ results directly in the Kibana Search UI.
 include::search-api.asciidoc[]
 include::knn-search.asciidoc[]
 include::semantic-search.asciidoc[]
-include::retrievers-reranking/index.asciidoc[]
-include::learning-to-rank.asciidoc[]
 include::search-across-clusters.asciidoc[]
 include::search-with-synonyms.asciidoc[]
 include::search-application-overview.asciidoc[]