Merge branch 'main' into indexing_pressure_bulk_inference

Author: jimczi

CONTRIBUTING.md

@@ -168,16 +168,13 @@ You can import the Elasticsearch project into IntelliJ IDEA via:
 
 #### Checkstyle
 
-If you have the [Checkstyle] plugin installed, you can configure IntelliJ to
-check the Elasticsearch code. However, the Checkstyle configuration file does
-not work by default with the IntelliJ plugin, so instead an IDE-specific config
-file is generated automatically after IntelliJ finishes syncing. You can
-manually generate the file with `./gradlew configureIdeCheckstyle` in case
-it is removed due to a `./gradlew clean` or other action.
-
-IntelliJ should be automatically configured to use the generated rules after
-import via the `.idea/checkstyle-idea.xml` configuration file. No further
-action is required.
+IntelliJ should automatically configure checkstyle. It does so by running
+`configureIdeCheckstyle` on import. That makes `.idea/checkstyle-idea.xml`
+configuration file. IntelliJ points checkstyle at that.
+
+Things like `./gradlew clean` or `git clean -xdf` can nuke the file. You can
+regenerate it by running `./gradlew -Didea.active=true configureIdeCheckstyle`,
+but generally shouldn't have to.
 
 #### Formatting
 

docs/changelog/127472.yaml

@@ -0,0 +1,6 @@
+pr: 127472
+summary: Change queries ID to be the same as the async
+area: ES|QL
+type: feature
+issues:
+ - 127187

docs/changelog/129150.yaml

@@ -0,0 +1,6 @@
+pr: 129150
+summary: Add `none` chunking strategy to disable automatic chunking for inference
+  endpoints
+area: Machine Learning
+type: feature
+issues: []

docs/changelog/129176.yaml

@@ -0,0 +1,6 @@
+pr: 129176
+summary: Adjust unpromotable shard refresh request validation to allow `RefreshResult.NO_REFRESH`
+area: Searchable Snapshots
+type: bug
+issues:
+ - 129036

docs/changelog/129223.yaml

@@ -0,0 +1,5 @@
+pr: 129223
+summary: Fix text similarity reranker does not propagate min score correctly
+area: Search
+type: bug
+issues: []

docs/changelog/129278.yaml

@@ -0,0 +1,5 @@
+pr: 129278
+summary: Fix constant keyword optimization
+area: ES|QL
+type: bug
+issues: []

docs/changelog/129326.yaml

@@ -0,0 +1,6 @@
+pr: 129326
+summary: Check positions on `MultiPhraseQueries` as well as phrase queries
+area: Search
+type: bug
+issues:
+ - 123871

docs/changelog/129359.yaml

@@ -0,0 +1,5 @@
+pr: 129359
+summary: Add min score linear retriever
+area: Search
+type: enhancement
+issues: []

docs/reference/elasticsearch/mapping-reference/semantic-text.md

@@ -117,15 +117,16 @@ If specified, these will override the chunking settings set in the {{infer-cap}}
 endpoint associated with `inference_id`.
 If chunking settings are updated, they will not be applied to existing documents
 until they are reindexed.
+To completely disable chunking, use the `none` chunking strategy.
 
     **Valid values for `chunking_settings`**:
 
     `type`
-    :   Indicates the type of chunking strategy to use. Valid values are `word` or
+    :   Indicates the type of chunking strategy to use. Valid values are `none`, `word` or
     `sentence`. Required.
 
     `max_chunk_size`
-    :   The maximum number of works in a chunk. Required.
+    :   The maximum number of words in a chunk. Required for `word` and `sentence` strategies.
 
     `overlap`
     :   The number of overlapping words allowed in chunks. This cannot be defined as
@@ -136,6 +137,12 @@ until they are reindexed.
     :   The number of overlapping sentences allowed in chunks. Valid values are `0`
     or `1`. Required for `sentence` type chunking settings
 
+::::{warning}
+If the input exceeds the maximum token limit of the underlying model,  some services (such as OpenAI) may return an 
+error. In contrast, the `elastic` and `elasticsearch` services  will automatically truncate the input to fit within the 
+model's limit.
+::::
+
 ## {{infer-cap}} endpoint validation [infer-endpoint-validation]
 
 The `inference_id` will not be validated when the mapping is created, but when
@@ -166,10 +173,49 @@ For more details on chunking and how to configure chunking settings,
 see [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference)
 in the Inference API documentation.
 
+You can pre-chunk the input by sending it to Elasticsearch as an array of strings.
+Example:
+
+```console
+PUT test-index
+{
+  "mappings": {
+    "properties": {
+      "my_semantic_field": {
+        "type": "semantic_text",
+        "chunking_settings": {
+          "strategy": "none"    <1>
+        }
+      }
+    }
+  }
+}
+```
+
+1. Disable chunking on `my_semantic_field`.
+
+```console
+PUT test-index/_doc/1
+{
+    "my_semantic_field": ["my first chunk", "my second chunk", ...]    <1>
+    ...
+}
+```
+
+1. The text is pre-chunked and provided as an array of strings.
+   Each element in the array represents a single chunk that will be sent directly to the inference service without further chunking.
+
+**Important considerations**:
+
+* When providing pre-chunked input, ensure that you set the chunking strategy to `none` to avoid additional processing.
+* Each chunk should be sized carefully, staying within the token limit of the inference service and the underlying model.
+* If a chunk exceeds the model's token limit, the behavior depends on the service:
+  * Some services (such as OpenAI) will return an error.
+  * Others (such as `elastic` and `elasticsearch`) will automatically truncate the input.
+
 Refer
 to [this tutorial](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md)
-to learn more about semantic search using `semantic_text` and the `semantic`
-query.
+to learn more about semantic search using `semantic_text`.
 
 ## Extracting Relevant Fragments from Semantic Text [semantic-text-highlighting]
 

docs/reference/elasticsearch/mapping-reference/term-vector.md

@@ -14,6 +14,8 @@ Term vectors contain information about the terms produced by the [analysis](docs
 
 These term vectors can be stored so that they can be retrieved for a particular document.
 
+Refer to the [term vectors API examples](../rest-apis/term-vectors-examples.md) page for usage examples.
+
 The `term_vector` setting accepts:
 
 `no`