Add docs for chunk_rescorer

Author: kderusso

docs/reference/elasticsearch/rest-apis/retrievers/retrievers-examples.md

@@ -440,7 +440,7 @@ GET /retrievers_example/_search
             "query": "artificial intelligence"
         }
     }
-}     
+}
 ```
 
 This returns the following response based on the final rrf score for each result.
@@ -497,7 +497,7 @@ GET /retrievers_example/_search
             "fields": ["text", "text_semantic"]
         }
     }
-}     
+}
 ```
 
 ::::{note}
@@ -570,7 +570,7 @@ GET /retrievers_example/_search
             "normalizer": "minmax"
         }
     }
-}     
+}
 ```
 
 This returns the following response based on the normalized score for each result:
@@ -1503,6 +1503,7 @@ PUT _inference/rerank/my-rerank-model
 ```
 
 Let’s start by reranking the results of the `rrf` retriever in our previous example.
+We'll also apply a `chunk_rescorer` to ensure that we only consider the best scoring chunks when sending information to the reranker.
 
 ```console
 GET retrievers_example/_search
@@ -1541,7 +1542,15 @@ GET retrievers_example/_search
             },
             "field": "text",
             "inference_id": "my-rerank-model",
-            "inference_text": "What are the state of the art applications of AI in information retrieval?"
+            "inference_text": "What are the state of the art applications of AI in information retrieval?",
+            "chunk_rescorer": {
+                "size": 1,
+                "chunking_settings": {
+                    "strategy": "sentence",
+                    "max_chunk_size": 300,
+                    "sentence_overlap": 0
+                }
+            },
         }
     },
     "_source": false

docs/reference/elasticsearch/rest-apis/retrievers/text-similarity-reranker-retriever.md

@@ -86,6 +86,22 @@ score = ln(score), if score < 0
 
     Applies the specified [boolean query filter](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to the child  `retriever`. If the child retriever already specifies any filters, then this top-level filter is applied in conjuction with the filter defined in the child retriever.
 
+`chunk_rescorer` {applies_to}`stack: beta 9.2`
+:   (Optional, `object`)
+
+    When specified, chunks and scores documents based on configured chunking settings, and only sends the best scoring chunks to the reranking model as input. This helps improve relevance when reranking long documents that would otherwise be truncated by the reranking model's token limit.
+
+    Parameters for `chunk_rescorer`:
+
+    `size`
+    :   (Optional, `int`)
+
+    The number of chunks to pass to the reranker for consideration. Defaults to `1`.
+
+    `chunking_settings`
+    :   (Optional, `object`)
+
+    Settings for chunking text into smaller passages for scoring and reranking. Defaults to the optimal chunk size for the Elastic Reranker. Note that if chunking settings are specified that chunk content into larger chunks than the reranker's token limit, it may result in truncation and negatively impact relevance.
 
 
 ## Example: Elastic Rerank [text-similarity-reranker-retriever-example-elastic-rerank]