Issue - #121567

This change adds the ability to use chunking for the elastic reranker as an alternative long document handling strategy to the existing truncation method. To enable chunking you must include the long_document_strategy (with the value set to chunk) in the service_settings of the rerank inference endpoint being used to perform inference. The value can also be set manually to truncate to force chunking but this is currently the default behavior. The max_chunks_per_doc value can optionally be included to limit the number of chunks that are sent for inference per document. If this value is not set then all chunks generated for the document will be sent. For example:

PUT _inference/rerank/my-elasticrerank-endpoint
{
  "service": "elasticsearch",
  "service_settings": {
    "model_id": ".rerank-v1", 
    "num_threads": 1,
    "num_allocations": 1,
    "long_document_strategy": "chunk",
    "max_chunks_per_doc": 2
  }
}

When using chunking, documents will be chunked before inference and the chunks (either all or some depending on whether max_chunks_per_doc is set) will be sent for inference. For each document, the relevance score returned to the user will be the maximum score for any given chunk within the document.

Testing

• Unit tests + integration tests
• Created an elastic reranker endpoint with no chunking configuration values and ensured that truncation worked as expected.
• Created an elastic reranker endpoint with truncate selected for long document strategy and ensured that truncation worked as expected.
• Created an elastic reranker endpoint with chunk selected for long document strategy and ensured that documents were chunked and all chunks were sent for inference.
• Created an elastic reranker endpoint with chunk selected for long document strategy and max_chunks_per_doc set and ensured that subset of chunks were sent for inference.
• (TODO) Created an non-elastic reranker elasticsearch service endpoint and ensured that inference is still working.