Merge branch 'main' into shard-logging

Author: smalyshev

build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/precommit/ThirdPartyAuditPrecommitPlugin.java

@@ -38,7 +38,7 @@ public TaskProvider<? extends Task> createTask(Project project) {
 
         project.getPlugins().apply(CompileOnlyResolvePlugin.class);
         project.getConfigurations().create("forbiddenApisCliJar");
-        project.getDependencies().add("forbiddenApisCliJar", "de.thetaphi:forbiddenapis:3.6");
+        project.getDependencies().add("forbiddenApisCliJar", "de.thetaphi:forbiddenapis:3.9");
         Configuration jdkJarHellConfig = project.getConfigurations().create(JDK_JAR_HELL_CONFIG_NAME);
 
         if (project.getPath().equals(LIBS_ELASTICSEARCH_CORE_PROJECT_PATH) == false) {

docs/changelog/143377.yaml

@@ -0,0 +1,5 @@
+area: Mapping
+issues: []
+pr: 143377
+summary: Re-use previously seen dynamically-created Mappers during document parsing
+type: bug

docs/changelog/143433.yaml

@@ -0,0 +1,6 @@
+area: Codec
+issues: []
+pr: 143433
+summary: Rewrite `*substring*` wildcard queries to contains term queries for binary
+  doc values keywords
+type: enhancement

docs/reference/elasticsearch/index-lifecycle-actions/ilm-rollover.md

@@ -57,9 +57,13 @@ The index will roll over once any `max_*` condition is satisfied and all `min_*`
 `max_docs`
 :   (Optional, integer) Triggers rollover after the specified maximum number of documents is reached. Documents added since the last refresh are not included in the document count. The document count does **not** include documents in replica shards.
 
-`max_size`
+`max_size` {applies_to}`stack: deprecated 9.3.0+`
 :   (Optional, [byte units](/reference/elasticsearch/rest-apis/api-conventions.md#byte-units)) Triggers rollover when the index reaches a certain size. This is the total size of all primary shards in the index. Replicas are not counted toward the maximum index size.
 
+    :::{admonition} Deprecated
+    The `max_size` rollover attribute will be removed in a future version. Use `max_primary_shard_size` instead. 
+    :::
+
     ::::{tip}
     To see the current index size, use the [_cat indices](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-indices) API. The `pri.store.size` value shows the combined size of all primary shards.
     ::::

docs/reference/enrich-processor/geoip-processor.md

@@ -214,8 +214,7 @@ If you can’t [automatically update](#geoip-automatic-updates) your IP geolocat
 * [Use a custom endpoint](#use-custom-geoip-endpoint)
 * [Manually update your IP geolocation databases](#manually-update-geoip-databases)
 
-$$$use-proxy-geoip-endpoint$$$
-**Use a reverse proxy endpoint**
+### Use a reverse proxy endpoint [use-proxy-geoip-endpoint]
 
 If you can’t connect directly to the Elastic GeoIP endpoint, consider setting up a secure reverse proxy. You can then specify the reverse proxy endpoint URL in the [`ingest.geoip.downloader.endpoint`](#ingest-geoip-downloader-endpoint) setting of each node’s `elasticsearch.yml` file.
 
@@ -228,8 +227,7 @@ In a strict setup the following domains may need to be added to the allowed doma
 * `geoip.elastic.co`
 * `storage.googleapis.com`
 
-$$$use-custom-geoip-endpoint$$$
-**Use a custom endpoint**
+### Use a custom endpoint [use-custom-geoip-endpoint]
 
 You can create a service that mimics the Elastic GeoIP endpoint. You can then get automatic updates from this service.
 
@@ -252,8 +250,7 @@ You can create a service that mimics the Elastic GeoIP endpoint. You can then ge
     By default, {{es}} checks the endpoint for updates every three days. To use another polling interval, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings) to set [`ingest.geoip.downloader.poll.interval`](#ingest-geoip-downloader-poll-interval).
 
 
-$$$manually-update-geoip-databases$$$
-**Manually update your IP geolocation databases**
+### Manually update your IP geolocation databases [manually-update-geoip-databases]
 
 1. Use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings) to set `ingest.geoip.downloader.enabled` to `false`. This disables automatic updates that may overwrite your database changes. This also deletes all downloaded databases.
 2. Download your `.mmdb` database files from the [MaxMind site](http://dev.maxmind.com/geoip/geoip2/geolite2).
@@ -264,7 +261,11 @@ $$$manually-update-geoip-databases$$$
 4. On self-managed deployments copy the database files to `$ES_CONFIG/ingest-geoip`.
 5. In your `geoip` processors, configure the `database_file` parameter to use a custom database file.
 
-### Node Settings [ingest-geoip-settings]
+### Troubleshooting geolocation database updates
+
+{{es}} ensures that only one node in the cluster downloads geolocation database updates at once. The node responsible for each download will record messages in its logs if it encounters any problems during the download. When troubleshooting problems with geolocation database updates, consult the logs from all the nodes in the cluster to gather accurate troubleshooting information.
+
+## Node Settings [ingest-geoip-settings]
 
 The `geoip` processor supports the following setting:
 
@@ -274,7 +275,7 @@ The `geoip` processor supports the following setting:
 Note that these settings are node settings and apply to all `geoip` and `ip_location` processors, i.e. there is a single cache for all such processors.
 
 
-### Cluster settings [geoip-cluster-settings]
+## Cluster settings [geoip-cluster-settings]
 
 $$$ingest-geoip-downloader-enabled$$$
 

docs/reference/query-languages/esql/_snippets/lists/aggregation-functions.md

@@ -2,8 +2,8 @@
 * [`AVG`](../../functions-operators/aggregation-functions/avg.md)
 * [`COUNT`](../../functions-operators/aggregation-functions/count.md)
 * [`COUNT_DISTINCT`](../../functions-operators/aggregation-functions/count_distinct.md)
-* [`FIRST`](../../functions-operators/aggregation-functions/first.md) {applies_to}`stack: preview` {applies_to}`serverless: preview`
-* [`LAST`](../../functions-operators/aggregation-functions/last.md) {applies_to}`stack: preview` {applies_to}`serverless: preview`
+% * [`FIRST`](../../functions-operators/aggregation-functions/first.md) {applies_to}`stack: preview` {applies_to}`serverless: preview`
+% * [`LAST`](../../functions-operators/aggregation-functions/last.md) {applies_to}`stack: preview` {applies_to}`serverless: preview`
 * [`MAX`](../../functions-operators/aggregation-functions/max.md)
 * [`MEDIAN`](../../functions-operators/aggregation-functions/median.md)
 * [`MEDIAN_ABSOLUTE_DEVIATION`](../../functions-operators/aggregation-functions/median_absolute_deviation.md)

docs/reference/query-languages/toc.yml

@@ -155,8 +155,8 @@ toc:
                   - file: esql/functions-operators/aggregation-functions/values.md
                   - file: esql/functions-operators/aggregation-functions/variance.md
                   - file: esql/functions-operators/aggregation-functions/weighted_avg.md
-                  - file: esql/functions-operators/aggregation-functions/first.md
-                  - file: esql/functions-operators/aggregation-functions/last.md
+                  - hidden: esql/functions-operators/aggregation-functions/first.md
+                  - hidden: esql/functions-operators/aggregation-functions/last.md
               - file: esql/functions-operators/time-series-aggregation-functions.md
                 children:
                   - file: esql/functions-operators/time-series-aggregation-functions/absent_over_time.md

docs/release-notes/known-issues.md

@@ -8,6 +8,30 @@ mapped_pages:
 
 Known issues are significant defects or limitations that may impact your implementation. These issues are actively being worked on and will be addressed in a future release. Review the Elasticsearch known issues to help you make informed decisions, such as upgrading to a new version.
 
+## 9.3.1 [elasticsearch-9.3.1-known-issues]
+
+* On multi-node clusters where one or more nodes do not have a GPU, the GPU stats collection for `_xpack/usage` triggers repeated WARN-level log messages from `OutboundHandler`:
+
+  ```
+  [WARN ][o.e.t.OutboundHandler] failed to serialize outbound message [org.elasticsearch.xpack.gpu.NodeGpuStatsResponse@...] java.lang.IllegalStateException: Negative longs unsupported, use writeLong or writeZLong for negative numbers [-1]
+  ```
+
+  The GPU stats for affected nodes are not collected, but all other `_xpack/usage` features continue to work normally. Single-node clusters are not affected because the response does not need to be serialized over the network.
+
+  To mitigate the log flooding, temporarily raise the log level for `OutboundHandler` to `ERROR`:
+
+  ```
+  PUT /_cluster/settings
+  {
+    "persistent": {
+      "logger.org.elasticsearch.transport.OutboundHandler": "ERROR"
+    }
+  }
+  ```
+
+  This bug is fixed in version 9.3.2.
+
+
 ## 9.2.4 [elasticsearch-9.2.4-known-issues]
 
 * Upgrading from 9.1.10 to 9.2.4 may cause the following error:
@@ -17,7 +41,7 @@ Known issues are significant defects or limitations that may impact your impleme
   ...
   org.elasticsearch.xcontent.XContentParseException: [-1:107008] [node_shutdown_info] unknown field [shutdown_started_millis] did you mean [shutdown_startedmillis]
   ```
-  
+
   This bug is addressed in version 9.2.5.
 
 ## 9.2.0 [elasticsearch-9.2.0-known-issues]

gradle/build.versions.toml

@@ -16,7 +16,7 @@ bytebuddy = "net.bytebuddy:byte-buddy:1.15.11"
 checkstyle = "com.puppycrawl.tools:checkstyle:11.0.1"
 commmons-io = "commons-io:commons-io:2.2"
 docker-compose = "com.avast.gradle:gradle-docker-compose-plugin:0.17.5"
-forbiddenApis = "de.thetaphi:forbiddenapis:3.8"
+forbiddenApis = "de.thetaphi:forbiddenapis:3.9"
 gradle-enterprise = "com.gradle:develocity-gradle-plugin:4.2.2"
 hamcrest = "org.hamcrest:hamcrest:3.0"
 httpcore5 = "org.apache.httpcomponents.core5:httpcore5:5.3.3"

gradle/verification-metadata.xml

@@ -1754,14 +1754,9 @@
             <sha256 value="b6ae5b90d5825070d2f82ac61417090fce8ed7e56348a3e86a2026774e8ba56a" origin="Generated by Gradle"/>
          </artifact>
       </component>
-      <component group="de.thetaphi" name="forbiddenapis" version="3.6">
-         <artifact name="forbiddenapis-3.6.jar">
-            <sha256 value="7b478e69fb613a05c2e1cb0ecfd257618cfb477333660914440a30b5afc712ff" origin="Generated by Gradle"/>
-         </artifact>
-      </component>
-      <component group="de.thetaphi" name="forbiddenapis" version="3.8">
-         <artifact name="forbiddenapis-3.8.jar">
-            <sha256 value="c8f7d13e5ac1cbbd605616b8882bf1693047a0351ac9e70c7b42fb6d43b3240d" origin="Generated by Gradle"/>
+      <component group="de.thetaphi" name="forbiddenapis" version="3.9">
+         <artifact name="forbiddenapis-3.9.jar">
+            <sha256 value="f44029523a5d8d827adfb5139479cbf5825d5521b132615056463131e371bb4f" origin="Generated by Gradle"/>
          </artifact>
       </component>
       <component group="dev.equo.ide" name="solstice" version="1.7.4">