Merge remote-tracking branch 'origin/main' into lucene_snapshot

Author: elasticsearchmachine

docs/reference/search-connectors/es-connectors-gmail.md

@@ -132,6 +132,8 @@ To get started, log into [Google Cloud Platform](https://cloud.google.com) and g
     You need to grant the following **OAuth Scopes** to your service account:
 
     * `https://www.googleapis.com/auth/admin.directory.user.readonly`
+    * `https://www.googleapis.com/auth/admin.directory.group.readonly`
+    * `https://www.googleapis.com/auth/gmail.readonly`
 
     This step allows the connector to access user data and their group memberships in your Google Workspace organization.
 
@@ -296,4 +298,4 @@ See [Security](/reference/search-connectors/es-connectors-security.md).
 
 This connector is built in Python with the [Elastic connector framework](https://github.com/elastic/connectors/tree/main).
 
-View the [source code for this connector](https://github.com/elastic/connectors/tree/main/connectors/sources/gmail.py) (branch *main*, compatible with Elastic *9.0*).
+View the [source code for this connector](https://github.com/elastic/connectors/tree/main/connectors/sources/gmail.py) (branch *main*, compatible with Elastic *9.0*).

docs/release-notes/breaking-changes.md

@@ -12,6 +12,13 @@ If you are migrating from a version prior to version 9.0, you must first upgrade
 
 % ## Next version [elasticsearch-nextversion-breaking-changes]
 
+## 9.0.8 [elasticsearch-9.0.8-breaking-changes]
+```{applies_to}
+stack: ga 9.0.8
+```
+
+There are no breaking changes associated with this release.
+
 ## 9.1.4 [elasticsearch-9.1.4-breaking-changes]
 
 There are no breaking changes associated with this release.

docs/release-notes/changelog-bundles/9.0.8.yml

@@ -0,0 +1,103 @@
+version: 9.0.8
+released: false
+generated: 2025-10-03T13:09:24.707151330Z
+changelogs:
+  - pr: 134134
+    summary: Prevent field caps from failing due to can match failure
+    area: Search
+    type: bug
+    issues:
+      - 116106
+  - pr: 134461
+    summary: Propagates filter() to aggregation functions' surrogates
+    area: Aggregations
+    type: bug
+    issues:
+      - 134380
+  - pr: 134636
+    summary: "[Island Browser] Add `manage`, `create_index`, `read`, `index`, `write`, `delete`, permission for third party agent indices `kibana_system`"
+    area: Authorization
+    type: enhancement
+    issues:
+      - 134136
+  - pr: 134656
+    summary: Fix deadlock in `ThreadPoolMergeScheduler` when a failing merge closes the `IndexWriter`
+    area: Engine
+    type: bug
+    issues: []
+  - pr: 134673
+    summary: Gracefully shutdown model deployment when node is removed from assignment routing
+    area: Machine Learning
+    type: bug
+    issues: []
+  - pr: 134711
+    summary: Add Reason field to elastic-agent upgrade details metadata
+    area: Infra/Plugins
+    type: enhancement
+    issues: []
+  - pr: 134955
+    summary: Prevent Transform from queuing too many PIT close requests by waiting for PIT to close before finishing the checkpoint
+    area: Transform
+    type: bug
+    issues:
+      - 134925
+  - pr: 134963
+    summary: Fix a bug in the GET _transform API that incorrectly claims some Transform configurations are missing
+    area: Transform
+    type: bug
+    issues:
+      - 134263
+  - pr: 135012
+    summary: Bypass MMap arena grouping as this has caused issues with too many regions being mapped
+    area: Engine
+    type: bug
+    issues: []
+  - pr: 135078
+    summary: Fix async get results with inconsistent headers
+    area: ES|QL
+    type: bug
+    issues:
+      - 135042
+  - pr: 135176
+    summary: '`CentroidCalculator` does not return negative summation weights'
+    area: Geo
+    type: bug
+    issues:
+      - 131861
+  - pr: 135235
+    summary: Fix systemd notify to use a shared arena
+    area: Infra/Node Lifecycle
+    type: bug
+    issues: []
+  - pr: 135270
+    summary: Add .reindexed-v7-ml-anomalies-* to anomaly results template index pattern
+    area: Machine Learning
+    type: bug
+    issues: []
+  - pr: 135414
+    summary: "Change reindex to use ::es-redacted:: filtering"
+    area: Audit
+    type: enhancement
+    issues: []
+  - pr: 135479
+    summary: Correctly apply field path to JSON processor when adding contents to document root
+    area: Ingest Node
+    type: bug
+    issues: []
+  - pr: 135653
+    summary: Reset health status on successful empty checkpoint
+    area: Machine Learning
+    type: bug
+    issues:
+      - 135650
+  - pr: 135776
+    summary: Fix KQL case-sensitivity for keyword fields in ES|QL
+    area: Search
+    type: bug
+    issues:
+      - 135772
+  - pr: 135845
+    summary: Fix for creating semantic_text fields on pre-8.11 indices crashing Elasticsearch
+    area: Mapping
+    type: bug
+    issues: []

docs/release-notes/deprecations.md

@@ -16,6 +16,13 @@ To give you insight into what deprecated features you’re using, {{es}}:
 
 % ## Next version [elasticsearch-nextversion-deprecations]
 
+## 9.0.8 [elasticsearch-9.0.8-deprecations]
+```{applies_to}
+stack: ga 9.0.8
+```
+
+There are no deprecations associated with this release.
+
 ## 9.1.4 [elasticsearch-9.1.4-deprecations]
 
 There are no deprecations associated with this release.

docs/release-notes/index.md

@@ -20,6 +20,62 @@ To check for security updates, go to [Security announcements for the Elastic sta
 % ### Fixes [elasticsearch-next-fixes]
 % *
 
+## 9.0.8 [elasticsearch-9.0.8-release-notes]
+```{applies_to}
+stack: ga 9.0.8
+```
+
+### Features and enhancements [elasticsearch-9.0.8-features-enhancements]
+
+Audit:
+* Change reindex to use ::es-redacted:: filtering [#135414](https://github.com/elastic/elasticsearch/pull/135414)
+
+Authorization:
+* [Island Browser] Add `manage`, `create_index`, `read`, `index`, `write`, `delete`, permission for third party agent indices `kibana_system` [#134636](https://github.com/elastic/elasticsearch/pull/134636) (issue: [#134136](https://github.com/elastic/elasticsearch/issues/134136))
+
+Infra/Plugins:
+* Add Reason field to elastic-agent upgrade details metadata [#134711](https://github.com/elastic/elasticsearch/pull/134711)
+
+
+### Fixes [elasticsearch-9.0.8-fixes]
+
+Aggregations:
+* Propagates filter() to aggregation functions' surrogates [#134461](https://github.com/elastic/elasticsearch/pull/134461) (issue: [#134380](https://github.com/elastic/elasticsearch/issues/134380))
+
+ES|QL:
+* Fix async get results with inconsistent headers [#135078](https://github.com/elastic/elasticsearch/pull/135078) (issue: [#135042](https://github.com/elastic/elasticsearch/issues/135042))
+
+Engine:
+* Bypass MMap arena grouping as this has caused issues with too many regions being mapped [#135012](https://github.com/elastic/elasticsearch/pull/135012)
+* Fix deadlock in `ThreadPoolMergeScheduler` when a failing merge closes the `IndexWriter` [#134656](https://github.com/elastic/elasticsearch/pull/134656)
+
+Geo:
+* `CentroidCalculator` does not return negative summation weights [#135176](https://github.com/elastic/elasticsearch/pull/135176) (issue: [#131861](https://github.com/elastic/elasticsearch/issues/131861))
+
+Infra/Node Lifecycle:
+* Fix systemd notify to use a shared arena [#135235](https://github.com/elastic/elasticsearch/pull/135235)
+
+Ingest Node:
+* Correctly apply field path to JSON processor when adding contents to document root [#135479](https://github.com/elastic/elasticsearch/pull/135479)
+
+Machine Learning:
+* Add .reindexed-v7-ml-anomalies-* to anomaly results template index pattern [#135270](https://github.com/elastic/elasticsearch/pull/135270)
+* Gracefully shutdown model deployment when node is removed from assignment routing [#134673](https://github.com/elastic/elasticsearch/pull/134673)
+* Reset health status on successful empty checkpoint [#135653](https://github.com/elastic/elasticsearch/pull/135653) (issue: [#135650](https://github.com/elastic/elasticsearch/issues/135650))
+
+Mapping:
+* Fix for creating semantic_text fields on pre-8.11 indices crashing Elasticsearch [#135845](https://github.com/elastic/elasticsearch/pull/135845)
+
+Search:
+* Fix KQL case-sensitivity for keyword fields in ES|QL [#135776](https://github.com/elastic/elasticsearch/pull/135776) (issue: [#135772](https://github.com/elastic/elasticsearch/issues/135772))
+* Prevent field caps from failing due to can match failure [#134134](https://github.com/elastic/elasticsearch/pull/134134) (issue: [#116106](https://github.com/elastic/elasticsearch/issues/116106))
+
+Transform:
+* Fix a bug in the GET _transform API that incorrectly claims some Transform configurations are missing [#134963](https://github.com/elastic/elasticsearch/pull/134963) (issue: [#134263](https://github.com/elastic/elasticsearch/issues/134263))
+* Prevent Transform from queuing too many PIT close requests by waiting for PIT to close before finishing the checkpoint [#134955](https://github.com/elastic/elasticsearch/pull/134955) (issue: [#134925](https://github.com/elastic/elasticsearch/issues/134925))
+
+
+
 ## 9.1.4 [elasticsearch-9.1.4-release-notes]
 
 ### Features and enhancements [elasticsearch-9.1.4-features-enhancements]

libs/entitlement/tools/common/src/main/java/org/elasticsearch/entitlement/tools/ExternalAccess.java

@@ -56,6 +56,11 @@ public static EnumSet<ExternalAccess> fromString(String accessAsString) {
         if ("PUBLIC".equals(accessAsString)) {
             return EnumSet.of(ExternalAccess.PUBLIC_CLASS, ExternalAccess.PUBLIC_METHOD);
         }
+        // used by JDK public API extractor (only), describing protected method access
+        // in this case public class access can be implied
+        if ("PROTECTED".equals(accessAsString)) {
+            return EnumSet.of(ExternalAccess.PUBLIC_CLASS, ExternalAccess.PROTECTED_METHOD);
+        }
         if ("PUBLIC-METHOD".equals(accessAsString)) {
             return EnumSet.of(ExternalAccess.PUBLIC_METHOD);
         }

libs/entitlement/tools/jdk-api-extractor/README.md

@@ -1,28 +1,39 @@
 This tool scans the JDK on which it is running to extract its public accessible API.
 That is:
-- public methods (including constructors) of public, exported classes as well as protected methods of these if not final.
-- internal implementations (overwrites) of above.
+- public methods (including constructors) of public, exported classes as well as protected methods of non-final classes.
+- any overwrites of public methods of public, exported super classes and interfaces.
 
 The output of this tool is meant to be diffed against the output for another JDK
 version to identify changes that need to be reviewed for entitlements.
+The output is compatible with the `public-callers-finder` tool to calculate the
+public transitive surface of new additions. See the example below.
 
 The following `TAB`-separated columns are written:
 1. module name
-2. fully qualified class name (ASM style, with `/` separators)
-3. method name
-4. method descriptor (ASM signature)
-5. visibility (`PUBLIC` / `PROTECTED`)
-6. `STATIC` modifier or empty
-7. `FINAL` modifier or empty
+2. unused / empty (for compatibility with `public-callers-finder`)
+3. unused / empty (for compatibility with `public-callers-finder`)
+4. fully qualified class name (ASM style, with `/` separators)
+5. method name
+6. method descriptor (ASM signature)
+7. visibility (`PUBLIC` / `PROTECTED`)
+8. `STATIC` modifier or empty
+9. `FINAL` modifier or empty
 
 Usage example:
 ```bash
 ./gradlew :libs:entitlement:tools:jdk-api-extractor:run -Druntime.java=24 --args="api-jdk24.tsv"
 ./gradlew :libs:entitlement:tools:jdk-api-extractor:run -Druntime.java=25 --args="api-jdk25.tsv"
+
 # diff the public apis
 diff -u libs/entitlement/tools/jdk-api-extractor/api-jdk24.tsv libs/entitlement/tools/jdk-api-extractor/api-jdk25.tsv > libs/entitlement/tools/jdk-api-extractor/api.diff
+
 # extract additions in the new JDK, these require the most careful review
 cat libs/entitlement/tools/jdk-api-extractor/api.diff | grep '^+[^+]' | sed 's/^+//' > api-jdk25-additions.tsv
+
+# review new additions next for critical ones that should require entitlements
+# once done, remove all lines that are not considered critical and run the public-callers-finder to report
+# the transitive public surface for these additions
+./gradlew :libs:entitlement:tools:public-callers-finder:run -Druntime.java=25 --args="api-jdk25-additions.tsv true"
 ```
 
 ### Optional arguments:

libs/entitlement/tools/jdk-api-extractor/src/main/java/org/elasticsearch/entitlement/tools/jdkapi/JdkApiExtractor.java

@@ -188,6 +188,8 @@ CharSequence toLine(ModuleClass moduleClass) {
             return String.join(
                 SEPARATOR,
                 moduleClass.module,
+                "", // compatibility with public-callers-finder
+                "", // compatibility with public-callers-finder
                 moduleClass.clazz,
                 method,
                 descriptor,

libs/entitlement/tools/public-callers-finder/README.md

@@ -1,4 +1,5 @@
-This tool scans the JDK on which it is running. It takes a list of methods (compatible with the output of the `securitymanager-scanner` tool), and looks for the "public surface" of these methods (i.e. any class/method accessible from regular Java code that calls into the original list, directly or transitively).
+This tool scans the JDK on which it is running. It takes a list of methods (compatible with the output of the `securitymanager-scanner` and `jdk-api-extractor` tools),
+and looks for the "public surface" of these methods (i.e. any class/method accessible from regular Java code that calls into the original list, directly or transitively).
 
 It acts basically as a recursive "Find Usages" in Intellij, stopping at the first fully accessible point (public method on a public class).
 The tool scans every method in every class inside the same java module; e.g.
@@ -14,19 +15,18 @@ it treats calls to `super` in `S.m` as regular calls (e.g. `example() -> S.m() -
 
 In order to run the tool, use:
 ```shell
-./gradlew :libs:entitlement:tools:public-callers-finder:run <input-file> [<bubble-up-from-public>]
+./gradlew :libs:entitlement:tools:public-callers-finder:run -Druntime.java=25 --args="<input-file> [<bubble-up-from-public>]"
 ```
 Where `input-file` is a CSV file (columns separated by `TAB`) that contains the following columns:
-Module name
-1. unused
+1. Module name
 2. unused
 3. unused
 4. Fully qualified class name (ASM style, with `/` separators)
 5. Method name
 6. Method descriptor (ASM signature)
 7. Visibility (PUBLIC/PUBLIC-METHOD/PRIVATE)
 
-And `bubble-up-from-public` is a boolean (`true|false`) indicating if the code should stop at the first public method (`false`: default, recommended) or continue to find usages recursively even after reaching the "public surface".
+And `bubble-up-from-public` is a boolean (`true|false`) indicating if the code should stop at the first public method (`false`: default) or continue to find usages recursively even after reaching the "public surface".
 
 The output of the tool is another CSV file, with one line for each entry-point, columns separated by `TAB`
 

libs/entitlement/tools/public-callers-finder/build.gradle

@@ -51,7 +51,7 @@ repositories {
 }
 
 dependencies {
-  compileOnly(project(':libs:core'))
+  implementation(project(':libs:core'))
   implementation 'org.ow2.asm:asm:9.8'
   implementation 'org.ow2.asm:asm-util:9.8'
   implementation(project(':libs:entitlement:tools:common'))