Resolving bugs

Author: mridula-s109

docs/reference/elasticsearch/rest-apis/retrievers.md

@@ -60,7 +60,7 @@ A standard retriever returns top documents from a traditional [query](/reference
 `filter`
 :   (Optional, [query object or list of query objects](/reference/query-languages/querydsl.md))
 
-    Applies a [boolean query filter](/reference/query-languages/query-dsl-bool-query.md) to this retriever, where all documents must match this query but do not contribute to the score.
+    Applies a [boolean query filter](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to this retriever, where all documents must match this query but do not contribute to the score.
 
 
 `search_after`
@@ -84,7 +84,7 @@ A standard retriever returns top documents from a traditional [query](/reference
 `min_score`
 :   (Optional, `float`)
 
-    Minimum [`_score`](/reference/query-languages/query-filter-context.md#relevance-scores) for matching documents. Documents with a lower `_score` are not included in the top documents.
+    Minimum [`_score`](/reference/query-languages/query-dsl/query-filter-context.md#relevance-scores) for matching documents. Documents with a lower `_score` are not included in the top documents.
 
 
 `collapse`
@@ -271,12 +271,12 @@ Each entry specifies the following parameters:
 `weight`
 :   (Optional, float)
 
-    The weight that each score of this retriever's top docs will be multiplied with. Must be greater or equal to 0. Defaults to 1.0.
+    The weight that each score of this retriever’s top docs will be multiplied with. Must be greater or equal to 0. Defaults to 1.0.
 
 `normalizer`
 :   (Optional, String)
 
-    Specifies how we will normalize the retriever's scores, before applying the specified `weight`. Available values are: `minmax`, and `none`. Defaults to `none`.
+    Specifies how we will normalize the retriever’s scores, before applying the specified `weight`. Available values are: `minmax`, and `none`. Defaults to `none`.
 
     * `none`
     * `minmax` : A `MinMaxScoreNormalizer` that normalizes scores based on the following formula
@@ -291,24 +291,17 @@ See also [this hybrid search example](docs-content://solutions/search/retrievers
 `rank_window_size`
 :   (Optional, integer)
 
-    This value determines the size of the individual result sets per query. A higher value will improve result relevance at the cost of performance. The final ranked result set is pruned down to the search request's [size](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#search-size-param). `rank_window_size` must be greater than or equal to `size` and greater than or equal to `1`. Defaults to the `size` parameter.
-
+    This value determines the size of the individual result sets per query. A higher value will improve result relevance at the cost of performance. The final ranked result set is pruned down to the search request’s [size](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#search-size-param). `rank_window_size` must be greater than or equal to `size` and greater than or equal to `1`. Defaults to the `size` parameter.
 
 `min_score`
 :   (Optional, float)
 
-    Minimum score threshold for documents to be included in the final result set. Documents with scores below this threshold will be filtered out. Must be greater than or equal to 0. Defaults to 0.
-
+    Minimum score threshold for documents to be included in the final result set. Documents with scores below this threshold will be filtered out. Must be greater than or equal to 0 if explicitly set. If not set, defaults to minimum float value, meaning no documents are filtered based on score  .
 
 `filter`
 :   (Optional, [query object or list of query objects](/reference/query-languages/querydsl.md))
 
-    Applies the specified [boolean query filter](/reference/query-languages/query-dsl-bool-query.md) to all of the specified sub-retrievers, according to each retriever's specifications.
-
-
-### Example: Hybrid search with min_score [linear-retriever-example]
-
-This example demonstrates how to use the Linear retriever to combine a standard retriever with a kNN retriever, applying weights, normalization, and a minimum score threshold:
+    Applies the specified [boolean query filter](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to all of the specified sub-retrievers, according to each retriever’s specifications.
 
 ```console
 GET /restaurants/_search
@@ -353,23 +346,24 @@ GET /restaurants/_search
 }
 ```
 
-1. Defines a retriever tree with a Linear retriever.
-2. The sub-retrievers array.
-3. The first sub-retriever is a `standard` retriever.
-4. The weight applied to the scores from the standard retriever (2.0).
-5. The normalization method applied to the standard retriever's scores.
-6. The second sub-retriever is a `knn` retriever.
-7. The weight applied to the scores from the kNN retriever (1.0).
-8. The normalization method applied to the kNN retriever's scores.
-9. The rank window size for the Linear retriever.
-10. The minimum score threshold - documents with a combined score below 1.5 will be filtered out from the final result set.
+1. Defines a retriever tree using the `linear` retriever type.
+2. The array of retrievers to be combined.
+3. A `standard` retriever used for traditional full-text search.
+4. Weight applied to the score from the `standard` retriever.
+5. Normalization method (`minmax`) applied to the `standard` retriever score.
+6. A `knn` retriever used for vector-based similarity search.
+7. Weight applied to the score from the `knn` retriever.
+8. Normalization method (`minmax`) applied to the `knn` retriever score.
+9. The number of top documents considered for scoring in the linear combination.
+10. Minimum score threshold for the final result set — documents below this combined score will be excluded.
 
 
 ## RRF Retriever [rrf-retriever]
 
 An [RRF](/reference/elasticsearch/rest-apis/reciprocal-rank-fusion.md) retriever returns top documents based on the RRF formula, equally weighting two or more child retrievers. Reciprocal rank fusion (RRF) is a method for combining multiple result sets with different relevance indicators into a single result set.
 
 
+
 #### Parameters [rrf-retriever-parameters]
 
 `retrievers`
@@ -387,13 +381,13 @@ An [RRF](/reference/elasticsearch/rest-apis/reciprocal-rank-fusion.md) retriever
 `rank_window_size`
 :   (Optional, integer)
 
-    This value determines the size of the individual result sets per query. A higher value will improve result relevance at the cost of performance. The final ranked result set is pruned down to the search request's [size](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#search-size-param). `rank_window_size` must be greater than or equal to `size` and greater than or equal to `1`. Defaults to the `size` parameter.
+    This value determines the size of the individual result sets per query. A higher value will improve result relevance at the cost of performance. The final ranked result set is pruned down to the search request’s [size](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#search-size-param). `rank_window_size` must be greater than or equal to `size` and greater than or equal to `1`. Defaults to the `size` parameter.
 
 
 `filter`
 :   (Optional, [query object or list of query objects](/reference/query-languages/querydsl.md))
 
-    Applies the specified [boolean query filter](/reference/query-languages/query-dsl-bool-query.md) to all of the specified sub-retrievers, according to each retriever's specifications.
+    Applies the specified [boolean query filter](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to all of the specified sub-retrievers, according to each retriever’s specifications.
 
 
 
@@ -502,12 +496,12 @@ For compound retrievers like `rrf`, the `window_size` parameter defines the tota
 
 When using the `rescorer`, an error is returned if the following conditions are not met:
 
-* The minimum configured rescore's `window_size` is:
+* The minimum configured rescore’s `window_size` is:
 
     * Greater than or equal to the `size` of the parent retriever for nested `rescorer` setups.
     * Greater than or equal to the `size` of the search request when used as the primary retriever in the tree.
 
-* And the maximum rescore's `window_size` is:
+* And the maximum rescore’s `window_size` is:
 
     * Smaller than or equal to the `size` or `rank_window_size` of the child retriever.
 
@@ -530,7 +524,7 @@ When using the `rescorer`, an error is returned if the following conditions are
 `filter`
 :   (Optional. [query object or list of query objects](/reference/query-languages/querydsl.md))
 
-    Applies a [boolean query filter](/reference/query-languages/query-dsl-bool-query.md) to the retriever, ensuring that all documents match the filter criteria without affecting their scores.
+    Applies a [boolean query filter](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to the retriever, ensuring that all documents match the filter criteria without affecting their scores.
 
 
 
@@ -699,7 +693,7 @@ score = ln(score), if score < 0
 `filter`
 :   (Optional, [query object or list of query objects](/reference/query-languages/querydsl.md))
 
-    Applies the specified [boolean query filter](/reference/query-languages/query-dsl-bool-query.md) to the child  `retriever`. If the child retriever already specifies any filters, then this top-level filter is applied in conjuction with the filter defined in the child retriever.
+    Applies the specified [boolean query filter](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to the child  `retriever`. If the child retriever already specifies any filters, then this top-level filter is applied in conjuction with the filter defined in the child retriever.
 
 
 
@@ -733,7 +727,7 @@ Follow these steps:
     }
     ```
 
-    1. [Adaptive allocations](docs-content://explore-analyze/machine-learning/nlp/ml-nlp-auto-scale.md#nlp-model-adaptive-allocations) will be enabled with the minimum of 1 and the maximum of 10 allocations.
+    1. [Adaptive allocations](docs-content://deploy-manage/autoscaling/trained-model-autoscaling.md#enabling-autoscaling-through-apis-adaptive-allocations) will be enabled with the minimum of 1 and the maximum of 10 allocations.
 
 2. Define a `text_similarity_rerank` retriever:
 
@@ -867,7 +861,7 @@ Follow these steps to load the model and create a semantic re-ranker.
 
 ## Query Rules Retriever [rule-retriever]
 
-The `rule` retriever enables fine-grained control over search results by applying contextual [query rules](/reference/elasticsearch/rest-apis/searching-with-query-rules.md#query-rules) to pin or exclude documents for specific queries. This retriever has similar functionality to the [rule query](/reference/query-languages/query-dsl-rule-query.md), but works out of the box with other retrievers.
+The `rule` retriever enables fine-grained control over search results by applying contextual [query rules](/reference/elasticsearch/rest-apis/searching-with-query-rules.md#query-rules) to pin or exclude documents for specific queries. This retriever has similar functionality to the [rule query](/reference/query-languages/query-dsl/query-dsl-rule-query.md), but works out of the box with other retrievers.
 
 ### Prerequisites [_prerequisites_16]
 
@@ -996,7 +990,7 @@ The [`from`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operati
 
 ### Using aggregations with a retriever tree [retriever-aggregations]
 
-[Aggregations](/reference/data-analysis/aggregations/index.md) are globally specified as part of a search request. The query used for an aggregation is the combination of all leaf retrievers as `should` clauses in a [boolean query](/reference/query-languages/query-dsl-bool-query.md).
+[Aggregations](/reference/aggregations/index.md) are globally specified as part of a search request. The query used for an aggregation is the combination of all leaf retrievers as `should` clauses in a [boolean query](/reference/query-languages/query-dsl/query-dsl-bool-query.md).
 
 
 ### Restrictions on search parameters when specifying a retriever [retriever-restrictions]

x-pack/plugin/rank-rrf/src/internalClusterTest/java/org/elasticsearch/xpack/rank/linear/LinearRetrieverIT.java

@@ -913,13 +913,9 @@ public void testLinearWithMinScore() {
         SearchRequestBuilder req = prepareSearchWithPIT(source);
         ElasticsearchAssertions.assertResponse(req, resp -> {
             assertNotNull(resp.pointInTimeId());
-            // TotalHits reflects the original query scope before compound minScore filtering.
-            // Asserting on hits.length verifies the retriever's minScore correctly filtered the returned hits.
-            // assertNotNull(resp.getHits().getTotalHits()); // getTotalHits() might still be non-null
-            // assertThat(resp.getHits().getTotalHits().value(), equalTo(1L)); // This assertion is incorrect based on expected behavior
-            // assertThat(resp.getHits().getTotalHits().relation(), equalTo(TotalHits.Relation.EQUAL_TO)); // Relation also reflects
-            // pre-filtering count
             assertThat(resp.getHits().getHits().length, equalTo(1)); // Verify actual returned hits count
+            // The total hits count reflects matches before min_score filtering.
+            assertThat(resp.getHits().getTotalHits().value(), equalTo(2L)); 
             assertThat(resp.getHits().getAt(0).getId(), equalTo("doc_2"));
             assertThat((double) resp.getHits().getAt(0).getScore(), closeTo(30.0f, 0.001f));
         });
@@ -944,7 +940,8 @@ public void testLinearWithMinScore() {
         ElasticsearchAssertions.assertResponse(req, resp -> {
             assertNotNull(resp.pointInTimeId());
             assertNotNull(resp.getHits().getTotalHits());
-            assertThat(resp.getHits().getTotalHits().value(), equalTo(3L));
+            // The total hits count reflects matches before min_score filtering.
+            assertThat(resp.getHits().getTotalHits().value(), equalTo(6L)); 
             assertThat(resp.getHits().getTotalHits().relation(), equalTo(TotalHits.Relation.EQUAL_TO));
             assertThat(resp.getHits().getHits().length, equalTo(3));
             assertThat(resp.getHits().getAt(0).getScore(), equalTo(30.0f));
@@ -992,7 +989,8 @@ public void testLinearWithMinScoreAndNormalization() {
         ElasticsearchAssertions.assertResponse(req, resp -> {
             assertNull(resp.pointInTimeId());
             assertNotNull(resp.getHits().getTotalHits());
-            assertThat(resp.getHits().getTotalHits().value(), equalTo(4L));
+            // The total hits count reflects matches before min_score filtering.
+            assertThat(resp.getHits().getTotalHits().value(), equalTo(6L)); 
             assertThat(resp.getHits().getTotalHits().relation(), equalTo(TotalHits.Relation.EQUAL_TO));
             assertThat(resp.getHits().getHits().length, equalTo(4));
             assertThat(resp.getHits().getAt(0).getId(), equalTo("doc_2"));
@@ -1023,7 +1021,8 @@ public void testLinearWithMinScoreAndNormalization() {
         ElasticsearchAssertions.assertResponse(req, resp -> {
             assertNotNull(resp.pointInTimeId());
             assertNotNull(resp.getHits().getTotalHits());
-            assertThat(resp.getHits().getTotalHits().value(), equalTo(3L));
+            // The total hits count reflects matches before min_score filtering.
+            assertThat(resp.getHits().getTotalHits().value(), equalTo(6L));
             assertThat(resp.getHits().getHits().length, equalTo(3));
             assertThat(resp.getHits().getAt(0).getId(), equalTo("doc_2"));
             assertThat((double) resp.getHits().getAt(0).getScore(), closeTo(1.9f, 0.1f));

x-pack/plugin/rank-rrf/src/main/java/org/elasticsearch/xpack/rank/linear/LinearRetrieverBuilder.java

@@ -191,9 +191,8 @@ protected RankDoc[] combineInnerRetrieverResults(List<ScoreDoc[]> rankResults, b
         }
         // sort the results based on the final score, tiebreaker based on smaller doc id
         LinearRankDoc[] sortedResults = docsToRankResults.values().toArray(LinearRankDoc[]::new);
-        Arrays.sort(sortedResults); // Sorts descending by score (highest first)
+        Arrays.sort(sortedResults);
 
-        // Find the number of results that meet the minScore threshold
         int validCount = 0;
         while (validCount < sortedResults.length && sortedResults[validCount].score >= minScore) {
             validCount++;
@@ -207,7 +206,6 @@ protected RankDoc[] combineInnerRetrieverResults(List<ScoreDoc[]> rankResults, b
             topResults[rank].rank = rank + 1;
         }
 
-        System.out.println("topResults: " + topResults.length);
         return topResults;
     }
 

x-pack/plugin/rank-rrf/src/main/java/org/elasticsearch/xpack/rank/linear/MinMaxScoreNormalizer.java

@@ -9,8 +9,6 @@
 
 import org.apache.lucene.search.ScoreDoc;
 
-import static org.elasticsearch.xpack.rank.linear.LinearRetrieverBuilder.DEFAULT_SCORE;
-
 public class MinMaxScoreNormalizer extends ScoreNormalizer {
 
     public static final MinMaxScoreNormalizer INSTANCE = new MinMaxScoreNormalizer();
@@ -55,10 +53,8 @@ public ScoreDoc[] normalizeScores(ScoreDoc[] docs) {
         boolean minEqualsMax = Math.abs(min - max) < EPSILON;
         for (int i = 0; i < docs.length; i++) {
             float score;
-            if (Float.isNaN(docs[i].score)) {
-                score = DEFAULT_SCORE;
-            } else if (minEqualsMax) {
-                score = docs[i].score;
+            if (minEqualsMax) {
+                score = min;
             } else {
                 score = (docs[i].score - min) / (max - min);
             }