diff --git a/docs/release-notes/breaking-changes.md b/docs/release-notes/breaking-changes.md index c15f23e8227bd..a9ecb54e7fdf7 100644 --- a/docs/release-notes/breaking-changes.md +++ b/docs/release-notes/breaking-changes.md @@ -12,10 +12,21 @@ If you are migrating from a version prior to version 9.0, you must first upgrade % ## Next version [elasticsearch-nextversion-breaking-changes] +```{applies_to} +stack: coming 9.0.6 +``` +## 9.0.6 [elasticsearch-9.0.6-breaking-changes] + +No breaking changes in this version. + ## 9.1.2 [elasticsearch-9.1.2-breaking-changes] No breaking changes in this version. +## 9.0.5 [elasticsearch-9.0.5-breaking-changes] + +No breaking changes in this version. + ## 9.1.1 [elasticsearch-9.1.1-breaking-changes] No breaking changes in this version. @@ -124,10 +135,6 @@ Test the upgrade in a non-production environment. Adapt your configuration to th For more information, view [#126843](https://github.com/elastic/elasticsearch/pull/126843) (issue: [#120993](https://github.com/elastic/elasticsearch/issues/120993)) ::: -## 9.0.5 [elasticsearch-9.0.5-breaking-changes] - -No breaking changes in this version. - ## 9.0.4 [elasticsearch-9.0.4-breaking-changes] No breaking changes in this version. diff --git a/docs/release-notes/changelog-bundles/9.0.6.yml b/docs/release-notes/changelog-bundles/9.0.6.yml new file mode 100644 index 0000000000000..f6cd697ad260f --- /dev/null +++ b/docs/release-notes/changelog-bundles/9.0.6.yml @@ -0,0 +1,72 @@ +version: 9.0.6 +released: false +generated: 2025-08-25T12:38:06.936945696Z +changelogs: + - pr: 132387 + summary: "[ExtraHop & QualysGAV] Add `manage`, `create_index`, `read`, `index`, `write`, `delete`, permission for third party agent indices `kibana_system`" + area: Authorization + type: enhancement + issues: + - 131825 + - pr: 132546 + summary: Improve EIS auth call logs and fix revocation bug + area: Machine Learning + type: bug + issues: [] + - pr: 132638 + summary: Better error message for sequences with only one clause plus UNTIL + area: EQL + type: bug + issues: [] + - pr: 132766 + summary: Change `reporting_user` role to leverage reserved kibana privileges + area: Authorization + type: deprecation + issues: [] + deprecation: + area: Authorization + title: Deprecate the built-in `reporting_user` role. + details: The `reporting_user` role is deprecated. Administrators should manage access to Kibana's reporting features via custom roles which grant the necessary privileges. + impact: This role will be removed in a future version. Administrators should migrate to custom roles to avoid interruption. + notable: false + essSettingChange: false + - pr: 132922 + summary: Change GeoIpCache and EnrichCache to LongAdder + area: Ingest Node + type: bug + issues: [] + - pr: 132945 + summary: Disable child span for streaming tasks + area: Machine Learning + type: bug + issues: [] + - pr: 132973 + summary: Preserve lost thread context in node inference action. A lost context causes a memory leak if APM tracing is enabled + area: Machine Learning + type: bug + issues: [] + - pr: 133004 + summary: Limit frequency of feature last-used time updates + area: License + type: bug + issues: [] + - pr: 133113 + summary: Limit the depth of a filter + area: Infra/REST API + type: enhancement + issues: [] + - pr: 133119 + summary: Update bundled JDK to Java 24.0.2+12 + area: Packaging + type: upgrade + issues: [] + - pr: 133134 + summary: Fix sequences with conditions involving keys and non-keys + area: EQL + type: bug + issues: [] + - pr: 133410 + summary: Upgrading to tika 3.2.2 + area: Ingest Node + type: upgrade + issues: [] diff --git a/docs/release-notes/deprecations.md b/docs/release-notes/deprecations.md index 9ca7b1164b7e3..b6fb19ffec91e 100644 --- a/docs/release-notes/deprecations.md +++ b/docs/release-notes/deprecations.md @@ -16,19 +16,29 @@ To give you insight into what deprecated features you’re using, {{es}}: % ## Next version [elasticsearch-nextversion-deprecations] +```{applies_to} +stack: coming 9.0.6 +``` +## 9.0.6 [elasticsearch-9.0.6-deprecations] + +Authorization: +* Change `reporting_user` role to leverage reserved kibana privileges [#132766](https://github.com/elastic/elasticsearch/pull/132766) + + + ## 9.1.2 [elasticsearch-9.1.2-deprecations] No deprecations in this version. -## 9.1.1 [elasticsearch-9.1.1-deprecations] +## 9.0.5 [elasticsearch-9.0.5-deprecations] No deprecations in this version. -## 9.1.0 [elasticsearch-9.1.0-deprecations] +## 9.1.1 [elasticsearch-9.1.1-deprecations] No deprecations in this version. -## 9.0.5 [elasticsearch-9.0.5-deprecations] +## 9.1.0 [elasticsearch-9.1.0-deprecations] No deprecations in this version. diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index cfef0acb38b3d..577da2974e1b2 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -20,6 +20,43 @@ To check for security updates, go to [Security announcements for the Elastic sta % ### Fixes [elasticsearch-next-fixes] % * +## 9.0.6 [elasticsearch-9.0.6-release-notes] +```{applies_to} +stack: coming 9.0.6 +``` + +### Features and enhancements [elasticsearch-9.0.6-features-enhancements] + +Authorization: +* [ExtraHop & QualysGAV] Add `manage`, `create_index`, `read`, `index`, `write`, `delete`, permission for third party agent indices `kibana_system` [#132387](https://github.com/elastic/elasticsearch/pull/132387) (issue: [#131825](https://github.com/elastic/elasticsearch/issues/131825)) + +Infra/REST API: +* Limit the depth of a filter [#133113](https://github.com/elastic/elasticsearch/pull/133113) + +Ingest Node: +* Upgrading to tika 3.2.2 [#133410](https://github.com/elastic/elasticsearch/pull/133410) + +Packaging: +* Update bundled JDK to Java 24.0.2+12 [#133119](https://github.com/elastic/elasticsearch/pull/133119) + +### Fixes [elasticsearch-9.0.6-fixes] + +EQL: +* Better error message for sequences with only one clause plus UNTIL [#132638](https://github.com/elastic/elasticsearch/pull/132638) +* Fix sequences with conditions involving keys and non-keys [#133134](https://github.com/elastic/elasticsearch/pull/133134) + +Ingest Node: +* Change GeoIpCache and EnrichCache to LongAdder [#132922](https://github.com/elastic/elasticsearch/pull/132922) + +License: +* Limit frequency of feature last-used time updates [#133004](https://github.com/elastic/elasticsearch/pull/133004) + +Machine Learning: +* Disable child span for streaming tasks [#132945](https://github.com/elastic/elasticsearch/pull/132945) +* Improve EIS auth call logs and fix revocation bug [#132546](https://github.com/elastic/elasticsearch/pull/132546) +* Preserve lost thread context in node inference action. A lost context causes a memory leak if APM tracing is enabled [#132973](https://github.com/elastic/elasticsearch/pull/132973) + + ## 9.1.2 [elasticsearch-9.1.2-release-notes] ### Features and enhancements [elasticsearch-9.1.2-features-enhancements] @@ -62,6 +99,116 @@ Exceptions in fetch sub-phases (for example, `setNextReader`) left the profiling [#132570](https://github.com/elastic/elasticsearch/pull/132570) ensures the `timer.stop()` call always stops. ::: + +## 9.0.5 [elasticsearch-9.0.5-release-notes] + +### Features and enhancements [elasticsearch-9.0.5-features-enhancements] + +Engine: +* The available disk space is now logged when merge tasks are scheduled. If the disk space is below the `indices.merge.disk.watermark.high` threshold, this information is recorded to help diagnose and troubleshoot situations where merges fail due to insufficient disk space. [#131711](https://github.com/elastic/elasticsearch/pull/131711) + +### Fixes [elasticsearch-9.0.5-fixes] + +Aggregations: +:::{dropdown} Validate parent aggregation type in `bucket_script` +The `bucket_script` pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation. +This caused a `ClassCastException` at runtime when the parent was not multi-bucket. +[#132320](https://github.com/elastic/elasticsearch/pull/132320) adds a validation step so the aggregation fails early, preventing the runtime error. (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) +::: + +Data streams: +:::{dropdown} Disables auto-sharding for LOOKUP index mode +Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to `LOOKUP`. +This happened because lookup mappers do not support scaling beyond one replica. +[#131429](https://github.com/elastic/elasticsearch/pull/131429) resolves this issue by disabling auto-sharding for data streams with `LOOKUP` index modes, avoiding unsupported replica settings. +::: + +EQL: +:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in `OR` expressions +Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an `OR` expression. +This occurred because lookup operators were internally rewritten into `IN` expressions, which are unsupported for IP-type fields. +[#132167](https://github.com/elastic/elasticsearch/pull/132167) resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) +::: + +:::{dropdown} Prevent double invocation of EQL listener +In some cases, the EQL listener could be resolved twice, potentially leading to unexpected behavior. +[#124918](https://github.com/elastic/elasticsearch/pull/124918) updates the control flow to exit early and ensure the listener is only invoked once. +::: + +ES|QL: +:::{dropdown} Disallow remote `ENRICH` after `LOOKUP JOIN` +Combining a `LOOKUP JOIN` with remote `ENRICH` could trigger a `ClassCastException` due to pipeline breaker interactions when limits or top-N queries were involved. [#131426](https://github.com/elastic/elasticsearch/pull/131426) adds a validation that forbids remote `ENRICH` after `LOOKUP JOIN`, preventing the runtime error. (issue: [#129372](https://github.com/elastic/elasticsearch/issues/129372)) +::: + +:::{dropdown} Fix `mv_expand` inconsistent column order +The `mv_expand` command could return columns in a different order depending on query execution paths. Now, the new attribute generated by `mv_expand` preserves the original field positions in the output. [#129745](https://github.com/elastic/elasticsearch/pull/129745) (issue: [#129000](https://github.com/elastic/elasticsearch/issues/129000)) +::: + +:::{dropdown} Fixes `ConcurrentModificationException` caused by live operator list +A `ConcurrentModificationException` caused test failures in `CrossClusterAsyncEnrichStopIT.testEnrichAfterStop` under certain conditions. +This happened because the ES|QL driver added a live operator list to the `DriverStatus` object, which could be modified while the status was being serialized. +[#132260](https://github.com/elastic/elasticsearch/pull/132260) fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads. +(issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) +::: + +Infra/Core: +:::{dropdown} Grants server module read/write permissions for deprecated `path.shared_data` setting +The server module is now granted read/write permissions for the deprecated `path.shared_data` setting. +[#131680](https://github.com/elastic/elasticsearch/pull/131680) resolves issues surfaced in internal testing and ensures compatibility with legacy configurations. +::: + +Ingest Node: +:::{dropdown} Correctly handle `download_database_on_pipeline_creation` in default or final pipelines +A bug in the `download_database_on_pipeline_creation` setting caused geoip databases not to download when the geoip processor was referenced from a pipeline processor in a default or final pipeline. +This resulted in documents being tagged with `_geoip_database_unavailable_GeoLite2-City.mmdb` instead of having geo data. +[#131236](https://github.com/elastic/elasticsearch/pull/131236) resolves the issue and ensures geoip databases download correctly in this scenario. +::: + +:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when `mapping_addition` is provided +When using the simulate ingest API with a `mapping_addition`, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed. +This caused mismatches between the index and simulation behavior. +[#132101](https://github.com/elastic/elasticsearch/pull/132101) resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution. +::: + +Machine Learning: +:::{dropdown} Fix memory usage estimation for ELSER models +Using the deployment ID instead of the model ID caused `isElserV1Or2Model` to fail for ELSER models, because deployment IDs don’t start with `.elser_model_2`. +[#131630](https://github.com/elastic/elasticsearch/pull/131630) updates the code to pass the model ID, ensuring memory usage is estimated correctly. +::: + +:::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation +A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections. +[#131990](https://github.com/elastic/elasticsearch/pull/131990) resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments. +::: + +Mapping: +:::{dropdown} Fixes decoding failure for non-ASCII field names in `_ignored_source` +A decoding error occurred when field names in `_ignored_source` contained non-ASCII characters. +This happened because `String.length()` was used to calculate the byte length of the field name, which only works correctly for ASCII characters. +[#132018](https://github.com/elastic/elasticsearch/pull/132018) resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding. +::: + +Search: +:::{dropdown} Correct shard status reporting in point-in-time responses +The Open PIT API incorrectly swapped the skipped and failed shard counts when partial search results were allowed. This caused the API to report failed shards as skipped and vice versa. [#131391](https://github.com/elastic/elasticsearch/pull/131391) fixes the field mapping so shard status is reported accurately. (issue: [#131026](https://github.com/elastic/elasticsearch/issues/131026)) +::: + +:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase +A missing removal of a query cancellation callback caused unintended timeouts or cancellations in later search phases when `allow_partial_search_results` was enabled, which could lead to `ArrayIndexOutOfBoundsException` errors. +[#130279](https://github.com/elastic/elasticsearch/pull/130279) resolves the issue and ensures predictable search execution. (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) +::: + +:::{dropdown} Preserve `boost` and `queryName` for semantic queries +Query rewrite logic dropped `boost` and `queryName` values for `match`, `knn`, and `sparse_vector` queries on `semantic_text` fields, causing query weighting and naming to be lost. [#129282](https://github.com/elastic/elasticsearch/pull/129282) resolves the issue so these values are now preserved correctly during query rewriting. +::: + +Snapshot/Restore: +:::{dropdown} Improve error handling when verifying an empty snapshot repository + +Verifying the integrity of a brand-new snapshot repository without any index blobs failed with a low-level error because the repository generation was `-1`, which cannot be sent over the wire. [#131677](https://github.com/elastic/elasticsearch/pull/131677) updates the logic to reject such requests early with a clearer, more helpful error message. +::: + + ## 9.1.1 [elasticsearch-9.1.1-release-notes] ### Fixes [elasticsearch-9.1.1-fixes] @@ -761,113 +908,6 @@ Vector Search: * Fix filtered knn vector search when query timeouts are enabled [#129440](https://github.com/elastic/elasticsearch/pull/129440) * Fix top level knn search with scroll [#126035](https://github.com/elastic/elasticsearch/pull/126035) -## 9.0.5 [elasticsearch-9.0.5-release-notes] - -### Features and enhancements [elasticsearch-9.0.5-features-enhancements] - -Engine: -* The available disk space is now logged when merge tasks are scheduled. If the disk space is below the `indices.merge.disk.watermark.high` threshold, this information is recorded to help diagnose and troubleshoot situations where merges fail due to insufficient disk space. [#131711](https://github.com/elastic/elasticsearch/pull/131711) - -### Fixes [elasticsearch-9.0.5-fixes] - -Aggregations: -:::{dropdown} Validate parent aggregation type in `bucket_script` -The `bucket_script` pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation. -This caused a `ClassCastException` at runtime when the parent was not multi-bucket. -[#132320](https://github.com/elastic/elasticsearch/pull/132320) adds a validation step so the aggregation fails early, preventing the runtime error. (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) -::: - -Data streams: -:::{dropdown} Disables auto-sharding for LOOKUP index mode -Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to `LOOKUP`. -This happened because lookup mappers do not support scaling beyond one replica. -[#131429](https://github.com/elastic/elasticsearch/pull/131429) resolves this issue by disabling auto-sharding for data streams with `LOOKUP` index modes, avoiding unsupported replica settings. -::: - -EQL: -:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in `OR` expressions -Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an `OR` expression. -This occurred because lookup operators were internally rewritten into `IN` expressions, which are unsupported for IP-type fields. -[#132167](https://github.com/elastic/elasticsearch/pull/132167) resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) -::: - -:::{dropdown} Prevent double invocation of EQL listener -In some cases, the EQL listener could be resolved twice, potentially leading to unexpected behavior. -[#124918](https://github.com/elastic/elasticsearch/pull/124918) updates the control flow to exit early and ensure the listener is only invoked once. -::: - -ES|QL: -:::{dropdown} Disallow remote `ENRICH` after `LOOKUP JOIN` -Combining a `LOOKUP JOIN` with remote `ENRICH` could trigger a `ClassCastException` due to pipeline breaker interactions when limits or top-N queries were involved. [#131426](https://github.com/elastic/elasticsearch/pull/131426) adds a validation that forbids remote `ENRICH` after `LOOKUP JOIN`, preventing the runtime error. (issue: [#129372](https://github.com/elastic/elasticsearch/issues/129372)) -::: - -:::{dropdown} Fix `mv_expand` inconsistent column order -The `mv_expand` command could return columns in a different order depending on query execution paths. Now, the new attribute generated by `mv_expand` preserves the original field positions in the output. [#129745](https://github.com/elastic/elasticsearch/pull/129745) (issue: [#129000](https://github.com/elastic/elasticsearch/issues/129000)) -::: - -:::{dropdown} Fixes `ConcurrentModificationException` caused by live operator list -A `ConcurrentModificationException` caused test failures in `CrossClusterAsyncEnrichStopIT.testEnrichAfterStop` under certain conditions. -This happened because the ES|QL driver added a live operator list to the `DriverStatus` object, which could be modified while the status was being serialized. -[#132260](https://github.com/elastic/elasticsearch/pull/132260) fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads. -(issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) -::: - -Infra/Core: -:::{dropdown} Grants server module read/write permissions for deprecated `path.shared_data` setting -The server module is now granted read/write permissions for the deprecated `path.shared_data` setting. -[#131680](https://github.com/elastic/elasticsearch/pull/131680) resolves issues surfaced in internal testing and ensures compatibility with legacy configurations. -::: - -Ingest Node: -:::{dropdown} Correctly handle `download_database_on_pipeline_creation` in default or final pipelines -A bug in the `download_database_on_pipeline_creation` setting caused geoip databases not to download when the geoip processor was referenced from a pipeline processor in a default or final pipeline. -This resulted in documents being tagged with `_geoip_database_unavailable_GeoLite2-City.mmdb` instead of having geo data. -[#131236](https://github.com/elastic/elasticsearch/pull/131236) resolves the issue and ensures geoip databases download correctly in this scenario. -::: - -:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when `mapping_addition` is provided -When using the simulate ingest API with a `mapping_addition`, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed. -This caused mismatches between the index and simulation behavior. -[#132101](https://github.com/elastic/elasticsearch/pull/132101) resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution. -::: - -Machine Learning: -:::{dropdown} Fix memory usage estimation for ELSER models -Using the deployment ID instead of the model ID caused `isElserV1Or2Model` to fail for ELSER models, because deployment IDs don’t start with `.elser_model_2`. -[#131630](https://github.com/elastic/elasticsearch/pull/131630) updates the code to pass the model ID, ensuring memory usage is estimated correctly. -::: - -:::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation -A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections. -[#131990](https://github.com/elastic/elasticsearch/pull/131990) resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments. -::: - -Mapping: -:::{dropdown} Fixes decoding failure for non-ASCII field names in `_ignored_source` -A decoding error occurred when field names in `_ignored_source` contained non-ASCII characters. -This happened because `String.length()` was used to calculate the byte length of the field name, which only works correctly for ASCII characters. -[#132018](https://github.com/elastic/elasticsearch/pull/132018) resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding. -::: - -Search: -:::{dropdown} Correct shard status reporting in point-in-time responses -The Open PIT API incorrectly swapped the skipped and failed shard counts when partial search results were allowed. This caused the API to report failed shards as skipped and vice versa. [#131391](https://github.com/elastic/elasticsearch/pull/131391) fixes the field mapping so shard status is reported accurately. (issue: [#131026](https://github.com/elastic/elasticsearch/issues/131026)) -::: - -:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase -A missing removal of a query cancellation callback caused unintended timeouts or cancellations in later search phases when `allow_partial_search_results` was enabled, which could lead to `ArrayIndexOutOfBoundsException` errors. -[#130279](https://github.com/elastic/elasticsearch/pull/130279) resolves the issue and ensures predictable search execution. (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) -::: - -:::{dropdown} Preserve `boost` and `queryName` for semantic queries -Query rewrite logic dropped `boost` and `queryName` values for `match`, `knn`, and `sparse_vector` queries on `semantic_text` fields, causing query weighting and naming to be lost. [#129282](https://github.com/elastic/elasticsearch/pull/129282) resolves the issue so these values are now preserved correctly during query rewriting. -::: - -Snapshot/Restore: -:::{dropdown} Improve error handling when verifying an empty snapshot repository - -Verifying the integrity of a brand-new snapshot repository without any index blobs failed with a low-level error because the repository generation was `-1`, which cannot be sent over the wire. [#131677](https://github.com/elastic/elasticsearch/pull/131677) updates the logic to reject such requests early with a clearer, more helpful error message. -::: ## 9.0.4 [elasticsearch-9.0.4-release-notes]