From cb3ad13dc979d07b414d1004776e892e82067bb9 Mon Sep 17 00:00:00 2001 From: elasticsearchmachine Date: Mon, 25 Aug 2025 12:10:05 +0000 Subject: [PATCH 1/3] Update docs for v9.1.3 release --- docs/release-notes/breaking-changes.md | 115 +----- .../release-notes/changelog-bundles/9.1.3.yml | 94 +++++ docs/release-notes/deprecations.md | 16 +- docs/release-notes/index.md | 328 ++++++------------ 4 files changed, 239 insertions(+), 314 deletions(-) create mode 100644 docs/release-notes/changelog-bundles/9.1.3.yml diff --git a/docs/release-notes/breaking-changes.md b/docs/release-notes/breaking-changes.md index c15f23e8227bd..5ca82aaff9382 100644 --- a/docs/release-notes/breaking-changes.md +++ b/docs/release-notes/breaking-changes.md @@ -12,10 +12,21 @@ If you are migrating from a version prior to version 9.0, you must first upgrade % ## Next version [elasticsearch-nextversion-breaking-changes] +```{applies_to} +stack: coming 9.1.3 +``` +## 9.1.3 [elasticsearch-9.1.3-breaking-changes] + +No breaking changes in this version. + ## 9.1.2 [elasticsearch-9.1.2-breaking-changes] No breaking changes in this version. +## 9.0.5 [elasticsearch-9.0.5-breaking-changes] + +No breaking changes in this version. + ## 9.1.1 [elasticsearch-9.1.1-breaking-changes] No breaking changes in this version. @@ -23,110 +34,18 @@ No breaking changes in this version. ## 9.1.0 [elasticsearch-9.1.0-breaking-changes] Discovery-Plugins: -:::{dropdown} Migrates `discovery-ec2` plugin to AWS SDK v2 -The `discovery-ec2` plugin now uses AWS SDK v2 instead of v1, as AWS plans to deprecate SDK v1 before the end of Elasticsearch 8.19’s support period. AWS SDK v2 introduces several behavior changes that affect configuration. - -**Impact:** -If you use the `discovery-ec2` plugin, your existing settings may no longer be compatible. Notable changes include, but may not be limited to: -- AWS SDK v2 does not support the EC2 IMDSv1 protocol. -- AWS SDK v2 does not support the `aws.secretKey` or `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system properties. -- AWS SDK v2 does not permit specifying a choice between HTTP and HTTPS so the `discovery.ec2.protocol` setting is no longer effective. -- AWS SDK v2 does not accept an access key without a secret key or vice versa. - -**Action:** -Test the upgrade in a non-production environment. Adapt your configuration to the new SDK functionality. This includes, but may not be limited to, the following items: -- If you use IMDS to determine the availability zone of a node or to obtain credentials for accessing the EC2 API, ensure that it supports the IMDSv2 protocol. -- If applicable, discontinue use of the `aws.secretKey` and `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system properties. -- If applicable, specify that you wish to use the insecure HTTP protocol to access the EC2 API by setting `discovery.ec2.endpoint` to a URL which starts with `http://`. -- Either supply both an access key and a secret key using the `discovery.ec2.access_key` and `discovery.ec2.secret_key` keystore settings, or configure neither of these settings. - -For more information, view [#122062](https://github.com/elastic/elasticsearch/pull/122062). -::: +* Upgrade `discovery-ec2` to AWS SDK v2 [#122062](https://github.com/elastic/elasticsearch/pull/122062) ES|QL: - -:::{dropdown} ES|QL now returns partial results by default -In previous versions, ES|QL queries failed entirely when any error occurred. As of 8.19.0, ES|QL returns partial results instead. - -**Impact:** -Callers must check the `is_partial` flag in the response to determine whether the result is complete. Relying on full results without checking this flag may lead to incorrect assumptions about the response. - -**Action:** -If partial results are not acceptable for your use case, you can disable this behavior by: -* Setting `allow_partial_results=false` in the query URL per request, or -* Setting the `esql.query.allow_partial_results` cluster setting to `false`. - -For more information, view [#127351](https://github.com/elastic/elasticsearch/pull/127351) (issue: [#122802](https://github.com/elastic/elasticsearch/issues/122802)) -::: - -:::{dropdown} Disallows parentheses in unquoted index patterns in ES|QL -To avoid ambiguity with subquery syntax, ES|QL no longer allows the use of `(` and `)` in unquoted index patterns. - -**Impact:** -Queries that include parentheses in unquoted index names will now result in a parsing exception. - -**Action:** -Update affected queries to quote index names that contain parentheses. For example, use `FROM "("foo")"` instead of `FROM (foo)`. -For more information, view [#130427](https://github.com/elastic/elasticsearch/pull/130427) (issue: [#130378](https://github.com/elastic/elasticsearch/issues/130378)) -::: - -:::{dropdown} Disallows mixing quoted and unquoted components in `FROM` index patterns -ES|QL no longer allows mixing quoted and unquoted parts in `FROM` index patterns (e.g. `FROM remote:"index"`). Previously, such patterns were parsed inconsistently and could result in misleading runtime errors. - -**Impact:** -Queries using partially quoted index patterns—such as quoting only the index or only the remote cluster—will now be rejected at parse time. This change simplifies grammar handling and avoids confusing validation failures. - -**Action:** -Ensure index patterns are either fully quoted or fully unquoted. For example: -* Valid: `FROM "remote:index"` or `FROM remote:index` -* Invalid: `FROM remote:"index"`, `FROM "remote":index` - -For more information, view [#127636](https://github.com/elastic/elasticsearch/pull/127636) (issue: [#122651](https://github.com/elastic/elasticsearch/issues/122651)) -::: - -:::{dropdown} `skip_unavailable` now catches all remote cluster runtime errors in ES|QL -When `skip_unavailable` is set to `true`, ES|QL now treats all runtime errors from that cluster as non-fatal. Previously, this setting only applied to connectivity issues (i.e. when a cluster was unavailable). - -**Impact:** -Errors such as missing indices on a remote cluster will no longer cause the query to fail. Instead, the cluster will appear in the response metadata as `skipped` or `partial`. - -**Action:** -If your workflows rely on detecting remote cluster errors, review your use of `skip_unavailable` and adjust error handling as needed. - -For more information, view [#128163](https://github.com/elastic/elasticsearch/pull/128163) -::: +* Allow partial results by default in ES|QL [#127351](https://github.com/elastic/elasticsearch/pull/127351) (issue: [#122802](https://github.com/elastic/elasticsearch/issues/122802)) +* Disallow brackets in unquoted index patterns [#130427](https://github.com/elastic/elasticsearch/pull/130427) (issue: [#130378](https://github.com/elastic/elasticsearch/issues/130378)) +* Disallow mixed quoted/unquoted patterns in FROM [#127636](https://github.com/elastic/elasticsearch/pull/127636) (issue: [#122651](https://github.com/elastic/elasticsearch/issues/122651)) +* Make `skip_unavailable` catch all errors [#128163](https://github.com/elastic/elasticsearch/pull/128163) Snapshot/Restore: +* Upgrade `repository-s3` to AWS SDK v2 [#126843](https://github.com/elastic/elasticsearch/pull/126843) (issue: [#120993](https://github.com/elastic/elasticsearch/issues/120993)) -:::{dropdown} Upgrades `repository-s3` plugin to AWS SDK v2 -The `repository-s3` plugin now uses AWS SDK v2 instead of v1, as AWS will deprecate SDK v1 before the end of Elasticsearch 8.19’s support period. The two SDKs differ in behavior, which may require updates to your configuration. - -**Impact:** -Existing `repository-s3` configurations may no longer be compatible. Notable differences in AWS SDK v2 include, but may not be limited to: -- AWS SDK v2 requires users to specify the region to use for signing requests, or else to run in an environment in which it can determine the correct region automatically. The older SDK used to determine the region based on the endpoint URL as specified with the `s3.client.${CLIENT_NAME}.endpoint` setting, together with other data drawn from the operating environment, and fell back to `us-east-1` if no better value was found. -- AWS SDK v2 does not support the EC2 IMDSv1 protocol. -- AWS SDK v2 does not support the `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system property. -- AWS SDK v2 does not permit specifying a choice between HTTP and HTTPS so the `s3.client.${CLIENT_NAME}.protocol` setting is deprecated and no longer has any effect. -- AWS SDK v2 does not permit control over throttling for retries, so the `s3.client.${CLIENT_NAME}.use_throttle_retries` setting is deprecated and no longer has any effect. -- AWS SDK v2 requires the use of the V4 signature algorithm, therefore, the `s3.client.${CLIENT_NAME}.signer_override` setting is deprecated and no longer has any effect. -- AWS SDK v2 does not support the `log-delivery-write` canned ACL. -- AWS SDK v2 counts 4xx responses differently in its metrics reporting. -- AWS SDK v2 always uses the regional STS endpoint, whereas AWS SDK v1 could use either a regional endpoint or the global `https://sts.amazonaws.com` one. - -**Action:** -Test the upgrade in a non-production environment. Adapt your configuration to the new SDK functionality. This includes, but may not be limited to, the following items: -- Specify the correct signing region using the `s3.client.${CLIENT_NAME}.region` setting on each node. {es} will try to determine the correct region based on the endpoint URL and other data drawn from the operating environment, but might not do so correctly in all cases. -- If you use IMDS to determine the availability zone of a node or to obtain credentials for accessing the EC2 API, ensure that it supports the IMDSv2 protocol. -- If applicable, discontinue use of the `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system property. -- If applicable, specify that you wish to use the insecure HTTP protocol to access the S3 API by setting `s3.client.${CLIENT_NAME}.endpoint` to a URL which starts with `http://`. -- If applicable, discontinue use of the `log-delivery-write` canned ACL. - -For more information, view [#126843](https://github.com/elastic/elasticsearch/pull/126843) (issue: [#120993](https://github.com/elastic/elasticsearch/issues/120993)) -::: -## 9.0.5 [elasticsearch-9.0.5-breaking-changes] - -No breaking changes in this version. ## 9.0.4 [elasticsearch-9.0.4-breaking-changes] diff --git a/docs/release-notes/changelog-bundles/9.1.3.yml b/docs/release-notes/changelog-bundles/9.1.3.yml new file mode 100644 index 0000000000000..12883ba8fcf51 --- /dev/null +++ b/docs/release-notes/changelog-bundles/9.1.3.yml @@ -0,0 +1,94 @@ +version: 9.1.3 +released: false +generated: 2025-08-25T12:09:52.381758857Z +changelogs: + - pr: 130336 + summary: "[EIS] Rename the elser 2 default model and the default inference endpoint" + area: Machine Learning + type: bug + issues: [] + - pr: 132546 + summary: Improve EIS auth call logs and fix revocation bug + area: Machine Learning + type: bug + issues: [] + - pr: 132638 + summary: Better error message for sequences with only one clause plus UNTIL + area: EQL + type: bug + issues: [] + - pr: 132646 + summary: Update EIS sparse and dense embedding max batch size to 16 + area: Machine Learning + type: bug + issues: [] + - pr: 132766 + summary: Change `reporting_user` role to leverage reserved kibana privileges + area: Authorization + type: deprecation + issues: [] + deprecation: + area: Authorization + title: Deprecate the built-in `reporting_user` role. + details: The `reporting_user` role is deprecated. Administrators should manage access to Kibana's reporting features via custom roles which grant the necessary privileges. + impact: This role will be removed in a future version. Administrators should migrate to custom roles to avoid interruption. + notable: false + essSettingChange: false + - pr: 132922 + summary: Change GeoIpCache and EnrichCache to LongAdder + area: Ingest Node + type: bug + issues: [] + - pr: 132945 + summary: Disable child span for streaming tasks + area: Machine Learning + type: bug + issues: [] + - pr: 132973 + summary: Preserve lost thread context in node inference action. A lost context causes a memory leak if APM tracing is enabled + area: Machine Learning + type: bug + issues: [] + - pr: 133004 + summary: Limit frequency of feature last-used time updates + area: License + type: bug + issues: [] + - pr: 133021 + summary: Fix update expiration for async query + area: ES|QL + type: bug + issues: + - 130619 + - pr: 133113 + summary: Limit the depth of a filter + area: Infra/REST API + type: enhancement + issues: [] + - pr: 133119 + summary: Update bundled JDK to Java 24.0.2+12 + area: Packaging + type: upgrade + issues: [] + - pr: 133134 + summary: Fix sequences with conditions involving keys and non-keys + area: EQL + type: bug + issues: [] + - pr: 133188 + summary: Don't fail search if bottom doc can't be formatted + area: Search + type: bug + issues: + - 125321 + - pr: 133347 + summary: Force rollover on write to true when data stream indices list is empty + area: Data streams + type: bug + issues: + - 133176 + - pr: 133410 + summary: Upgrading to tika 3.2.2 + area: Ingest Node + type: upgrade + issues: [] diff --git a/docs/release-notes/deprecations.md b/docs/release-notes/deprecations.md index 9ca7b1164b7e3..cd9d0e8bc31b1 100644 --- a/docs/release-notes/deprecations.md +++ b/docs/release-notes/deprecations.md @@ -16,19 +16,29 @@ To give you insight into what deprecated features you’re using, {{es}}: % ## Next version [elasticsearch-nextversion-deprecations] +```{applies_to} +stack: coming 9.1.3 +``` +## 9.1.3 [elasticsearch-9.1.3-deprecations] + +Authorization: +* Change `reporting_user` role to leverage reserved kibana privileges [#132766](https://github.com/elastic/elasticsearch/pull/132766) + + + ## 9.1.2 [elasticsearch-9.1.2-deprecations] No deprecations in this version. -## 9.1.1 [elasticsearch-9.1.1-deprecations] +## 9.0.5 [elasticsearch-9.0.5-deprecations] No deprecations in this version. -## 9.1.0 [elasticsearch-9.1.0-deprecations] +## 9.1.1 [elasticsearch-9.1.1-deprecations] No deprecations in this version. -## 9.0.5 [elasticsearch-9.0.5-deprecations] +## 9.1.0 [elasticsearch-9.1.0-deprecations] No deprecations in this version. diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index cfef0acb38b3d..3e4a92bdf10c8 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -20,148 +20,157 @@ To check for security updates, go to [Security announcements for the Elastic sta % ### Fixes [elasticsearch-next-fixes] % * +## 9.1.3 [elasticsearch-9.1.3-release-notes] +```{applies_to} +stack: coming 9.1.3 +``` + +### Features and enhancements [elasticsearch-9.1.3-features-enhancements] + +Infra/REST API: +* Limit the depth of a filter [#133113](https://github.com/elastic/elasticsearch/pull/133113) + +Ingest Node: +* Upgrading to tika 3.2.2 [#133410](https://github.com/elastic/elasticsearch/pull/133410) + +Packaging: +* Update bundled JDK to Java 24.0.2+12 [#133119](https://github.com/elastic/elasticsearch/pull/133119) + +### Fixes [elasticsearch-9.1.3-fixes] + +Data streams: +* Force rollover on write to true when data stream indices list is empty [#133347](https://github.com/elastic/elasticsearch/pull/133347) (issue: [#133176](https://github.com/elastic/elasticsearch/issues/133176)) + +EQL: +* Better error message for sequences with only one clause plus UNTIL [#132638](https://github.com/elastic/elasticsearch/pull/132638) +* Fix sequences with conditions involving keys and non-keys [#133134](https://github.com/elastic/elasticsearch/pull/133134) + +ES|QL: +* Fix update expiration for async query [#133021](https://github.com/elastic/elasticsearch/pull/133021) (issue: [#130619](https://github.com/elastic/elasticsearch/issues/130619)) + +Ingest Node: +* Change GeoIpCache and EnrichCache to LongAdder [#132922](https://github.com/elastic/elasticsearch/pull/132922) + +License: +* Limit frequency of feature last-used time updates [#133004](https://github.com/elastic/elasticsearch/pull/133004) + +Machine Learning: +* Disable child span for streaming tasks [#132945](https://github.com/elastic/elasticsearch/pull/132945) +* Improve EIS auth call logs and fix revocation bug [#132546](https://github.com/elastic/elasticsearch/pull/132546) +* Preserve lost thread context in node inference action. A lost context causes a memory leak if APM tracing is enabled [#132973](https://github.com/elastic/elasticsearch/pull/132973) +* Update EIS sparse and dense embedding max batch size to 16 [#132646](https://github.com/elastic/elasticsearch/pull/132646) +* [EIS] Rename the elser 2 default model and the default inference endpoint [#130336](https://github.com/elastic/elasticsearch/pull/130336) + +Search: +* Don't fail search if bottom doc can't be formatted [#133188](https://github.com/elastic/elasticsearch/pull/133188) (issue: [#125321](https://github.com/elastic/elasticsearch/issues/125321)) + + ## 9.1.2 [elasticsearch-9.1.2-release-notes] ### Features and enhancements [elasticsearch-9.1.2-features-enhancements] Authorization: -* Adds `manage`, `create_index`, `read`, `index`, `write`, and `delete` privileges for the `kibana_system` role on third-party agent indices used by ExtraHop (`logs-extrahop.investigation-*`) and Qualys GAV (`logs-qualys_gav.asset-*`). This ensures ILM policies can delete these indices without permission errors. [#132387](https://github.com/elastic/elasticsearch/pull/132387) (issue: [#131825](https://github.com/elastic/elasticsearch/issues/131825)) +* [ExtraHop & QualysGAV] Add `manage`, `create_index`, `read`, `index`, `write`, `delete`, permission for third party agent indices `kibana_system` [#132387](https://github.com/elastic/elasticsearch/pull/132387) (issue: [#131825](https://github.com/elastic/elasticsearch/issues/131825)) ### Fixes [elasticsearch-9.1.2-fixes] Aggregations: -:::{dropdown} Validates parent aggregation type in `bucket_script` -The `bucket_script` pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation. -This caused a `ClassCastException` at runtime when the parent was not multi-bucket. -[#132320](https://github.com/elastic/elasticsearch/pull/132320) adds a validation step so the aggregation fails early, preventing the runtime error. (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) -::: +* Aggs: Add validation to Bucket script pipeline agg [#132320](https://github.com/elastic/elasticsearch/pull/132320) (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) Codec: -:::{dropdown} Uses local segment `fieldInfos` for TSDB merge stats -Merging shrink TSDB or LogsDB indices in versions 8.19 or 9.1+ could fail when using `addIndexes` to combine Lucene segments directly. -In these cases, the `fieldInfos` value could differ between shards and the merged segment, causing incorrect merge statistics. -PR [#132597](https://github.com/elastic/elasticsearch/pull/132597) updates the process to use `fieldInfos` from each segment instead of the merged segment, ensuring accurate stats and preventing merge failures. -::: +* Use local segment `fieldInfos` to lookup tsdb merge stats [#132597](https://github.com/elastic/elasticsearch/pull/132597) ES|QL: -:::{dropdown} Fixes for `COPY_SIGN` function in ESQL -The `COPY_SIGN` function has been updated to better support the literal `NULL` in parameters. -[#132459](https://github.com/elastic/elasticsearch/pull/132459) -::: +* Small fixes for COPY_SIGN [#132459](https://github.com/elastic/elasticsearch/pull/132459) + +Mapping: +* Strings outside BMP have 2 chars per code points [#132593](https://github.com/elastic/elasticsearch/pull/132593) + +Search: +* Always stop the timer when profiling the fetch phase [#132570](https://github.com/elastic/elasticsearch/pull/132570) + + +## 9.0.5 [elasticsearch-9.0.5-release-notes] + +### Features and enhancements [elasticsearch-9.0.5-features-enhancements] + +Engine: +* Track & log when there is insufficient disk space available to execute merges [#131711](https://github.com/elastic/elasticsearch/pull/131711) + +### Fixes [elasticsearch-9.0.5-fixes] + +Aggregations: +* Aggs: Add validation to Bucket script pipeline agg [#132320](https://github.com/elastic/elasticsearch/pull/132320) (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) + +Data streams: +* Prevent auto-sharding for data streams in LOOKUP index mode [#131429](https://github.com/elastic/elasticsearch/pull/131429) + +EQL: +* Deal with internally created IN in a different way for EQL [#132167](https://github.com/elastic/elasticsearch/pull/132167) (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) +* Fix EQL double invoking listener [#124918](https://github.com/elastic/elasticsearch/pull/124918) + +ES|QL: +* Disallow remote enrich after lu join [#131426](https://github.com/elastic/elasticsearch/pull/131426) (issue: [#129372](https://github.com/elastic/elasticsearch/issues/129372)) +* ESQL: Fix `mv_expand` inconsistent column order [#129745](https://github.com/elastic/elasticsearch/pull/129745) (issue: [#129000](https://github.com/elastic/elasticsearch/issues/129000)) +* FIx Driver creating status with a live list of operators [#132260](https://github.com/elastic/elasticsearch/pull/132260) (issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) + +Infra/Core: +* Grant server module read/write entitlements for deprecated path setting "path.shared_data" [#131680](https://github.com/elastic/elasticsearch/pull/131680) + +Ingest Node: +* Correctly handling `download_database_on_pipeline_creation` within a pipeline processor within a default or final pipeline [#131236](https://github.com/elastic/elasticsearch/pull/131236) +* Simulate ingest API uses existing index mapping when `mapping_addition` is given [#132101](https://github.com/elastic/elasticsearch/pull/132101) + +Machine Learning: +* Fix memory usage estimation for ELSER models [#131630](https://github.com/elastic/elasticsearch/pull/131630) +* Prevent the trained model deployment memory estimation from double-counting allocations [#131990](https://github.com/elastic/elasticsearch/pull/131990) Mapping: -:::{dropdown} Calculates text string length correctly for code points outside BMP -Strings parsed with the optimized UTF-8 parsing path had incorrect length calculations for characters outside the basic multilingual plane (BMP). -These characters require two UTF-16 code units, but the optimized path did not account for this, causing mismatches with the non-optimized path. -[#132593](https://github.com/elastic/elasticsearch/pull/132593) fixes the calculation to ensure consistent and correct string lengths. -::: +* Fix decoding of non-ascii field names in ignored source [#132018](https://github.com/elastic/elasticsearch/pull/132018) Search: -:::{dropdown} Always stops the timer when profiling the fetch phase -Exceptions in fetch sub-phases (for example, `setNextReader`) left the profiling timer running, causing mismatched start/stop calls and errors. -[#132570](https://github.com/elastic/elasticsearch/pull/132570) ensures the `timer.stop()` call always stops. -::: +* Fix bug in point in time response [#131391](https://github.com/elastic/elasticsearch/pull/131391) (issue: [#131026](https://github.com/elastic/elasticsearch/issues/131026)) +* Fix missing removal of query cancellation callback in QueryPhase [#130279](https://github.com/elastic/elasticsearch/pull/130279) (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) +* Fix query rewrite logic to preserve `boosts` and `queryName` for `match`, `knn`, and `sparse_vector` queries on semantic_text fields [#129282](https://github.com/elastic/elasticsearch/pull/129282) + +Snapshot/Restore: +* Throw better exception if verifying empty repo [#131677](https://github.com/elastic/elasticsearch/pull/131677) + ## 9.1.1 [elasticsearch-9.1.1-release-notes] ### Fixes [elasticsearch-9.1.1-fixes] Data streams: -:::{dropdown} Disables auto-sharding for LOOKUP index mode -Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to `LOOKUP`. -This happened because lookup mappers do not support scaling beyond one replica. -[#131429](https://github.com/elastic/elasticsearch/pull/131429) resolves this issue by disabling auto-sharding for data streams with `LOOKUP` index mode, avoiding unsupported replica settings. -::: +* Prevent auto-sharding for data streams in LOOKUP index mode [#131429](https://github.com/elastic/elasticsearch/pull/131429) EQL: -:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in `OR` expressions -Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an `OR` expression. -This occurred because lookup operators were internally rewritten into `IN` expressions, which are unsupported for IP-type fields. -[#132167](https://github.com/elastic/elasticsearch/pull/132167) resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) -::: +* Deal with internally created IN in a different way for EQL [#132167](https://github.com/elastic/elasticsearch/pull/132167) (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) ES|QL: -:::{dropdown} Fixes inconsistent equality and hashcode behavior for `ConstantNullBlock` -Inconsistent equality checks caused `constantNullBlock.equals(anyDoubleBlock)` to return false, even when `doubleBlock.equals(constantNullBlock)` returned true. -This asymmetry led to unreliable comparisons and mismatched hashcodes when `ConstantNullBlock` was functionally equivalent to other standard blocks. -[#131817](https://github.com/elastic/elasticsearch/pull/131817) resolves the issue and ensures both equality and hashcode functions are symmetric for these block types. -::: - -:::{dropdown} Fixes `ConcurrentModificationException` caused by live operator list -A `ConcurrentModificationException` caused test failures in `CrossClusterAsyncEnrichStopIT.testEnrichAfterStop` under certain conditions. -This happened because the ES|QL driver added a live operator list to the `DriverStatus` object, which could be modified while the status was being serialized. -[#132260](https://github.com/elastic/elasticsearch/pull/132260) fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads. -(issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) -::: - -:::{dropdown} Prevents null pointer exception for `to_lower` and `to_upper` with no parameters -Calling the `to_lower` or `to_upper` functions with no parameters caused a null pointer exception (NPE), instead of returning a clear error. -This behavior was a result of an older implementation of these functions. -[#131917](https://github.com/elastic/elasticsearch/pull/131917) resolves the issue and ensures that empty parameter calls now return the correct error message. (issue: [#131913](https://github.com/elastic/elasticsearch/issues/131913)) -::: - -:::{dropdown} Fixes `aggregate_metric_double` decoding and `mv_expand` behavior on multi-index queries -Sorting across multiple indices failed when one index contained an `aggregate_metric_double` field and another did not. -In this case, the missing field was encoded as `NullBlock` but later incorrectly decoded as `AggregateMetricDoubleBlock`, which expects four values. This mismatch caused decoding errors. -[#131658](https://github.com/elastic/elasticsearch/pull/131658) resolves the issue and also improves `mv_expand` by returning the input block unchanged for unsupported `AggregateMetricDoubleBlock` values, avoiding unnecessary errors. -::: - -:::{dropdown} Fixes incorrect `ingest_took` value when combining bulk responses -Combining two `BulkResponse` objects with `ingest_took` set to `NO_INGEST_TOOK` resulted in a combined `ingest_took` value of `-2`, which was invalid. -This occurred because the combination logic failed to preserve the sentinel `NO_INGEST_TOOK` constant. -[#132088](https://github.com/elastic/elasticsearch/pull/132088) resolves the issue and ensures the result is correctly set to `NO_INGEST_TOOK` when applicable. -::: - -:::{dropdown} Disallows remote ENRICH after FORK in query plans -An invalid combination of `FORK` followed by a remote-only `ENRICH` caused incorrect query planning and failed executions. [#131945](https://github.com/elastic/elasticsearch/pull/131945) resolves the issue by explicitly disallowing this combination, preventing invalid plans from being executed. (issue: [#131445](https://github.com/elastic/elasticsearch/issues/131445)) -::: - -:::{dropdown} Adds support for splitting large pages on load to avoid memory pressure -Loading large rows from a single segment occasionally created oversized pages when decoding values row-by-row, particularly for text and geo fields. -This could cause memory pressure or degraded performance. -[#131053](https://github.com/elastic/elasticsearch/pull/131053) resolves the issue by estimating the size of each page as rows are loaded. -If the estimated size exceeds a configurable `jumbo` threshold (defaulting to one megabyte), row loading stops early, the page is returned, and remaining rows are processed in subsequent iterations. -This prevents loading incomplete or oversized pages during data aggregation. -::: +* Change equals and hashcode for `ConstantNullBlock` [#131817](https://github.com/elastic/elasticsearch/pull/131817) +* FIx Driver creating status with a live list of operators [#132260](https://github.com/elastic/elasticsearch/pull/132260) (issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) +* Fix NPE on empty to_lower/to_upper call [#131917](https://github.com/elastic/elasticsearch/pull/131917) (issue: [#131913](https://github.com/elastic/elasticsearch/issues/131913)) +* Fix `aggregate_metric_double` sorting and `mv_expand` issues [#131658](https://github.com/elastic/elasticsearch/pull/131658) +* Fix combine result for `ingest_took` [#132088](https://github.com/elastic/elasticsearch/pull/132088) +* Restrict remote ENRICH after FORK [#131945](https://github.com/elastic/elasticsearch/pull/131945) (issue: [#131445](https://github.com/elastic/elasticsearch/issues/131445)) +* Split large pages on load sometimes [#131053](https://github.com/elastic/elasticsearch/pull/131053) Infra/Core: -:::{dropdown} Grants server module read/write permissions for deprecated `path.shared_data` setting -Grants the server module read/write access to the deprecated `path.shared_data` setting. -[#131680](https://github.com/elastic/elasticsearch/pull/131680) resolves issues surfaced in internal testing and ensures compatibility with legacy configurations. -::: +* Grant server module read/write entitlements for deprecated path setting "path.shared_data" [#131680](https://github.com/elastic/elasticsearch/pull/131680) Ingest Node: -:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when `mapping_addition` is provided -When using the simulate ingest API with a `mapping_addition`, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed. -This caused mismatches between the index and simulation behavior. -[#132101](https://github.com/elastic/elasticsearch/pull/132101) resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution. -::: +* Simulate ingest API uses existing index mapping when `mapping_addition` is given [#132101](https://github.com/elastic/elasticsearch/pull/132101) Machine Learning: -:::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation -A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections. -[#131990](https://github.com/elastic/elasticsearch/pull/131990) resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments. -::: +* Prevent the trained model deployment memory estimation from double-counting allocations [#131990](https://github.com/elastic/elasticsearch/pull/131990) Mapping: -:::{dropdown} Fixes decoding failure for non-ASCII field names in `_ignored_source` -A decoding error occurred when field names in `_ignored_source` contained non-ASCII characters. -This happened because `String.length()` was used to calculate the byte length of the field name, which only works correctly for ASCII characters. -[#132018](https://github.com/elastic/elasticsearch/pull/132018) resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding. -::: +* Fix decoding of non-ascii field names in ignored source [#132018](https://github.com/elastic/elasticsearch/pull/132018) Search: -:::{dropdown} Fixes index sort compatibility for `date_nanos` fields in indices created before 7.14 -Indices created prior to version 7.14 that used an index sort on a `date_nanos` field could not be opened in more recent versions due to a mismatch in the default `index.sort.missing` value. -A change in version 7.14 modified the default from `Long.MIN_VALUE` to `0L`, which caused newer versions to reject those older indices. -[#132162](https://github.com/elastic/elasticsearch/pull/132162) resolves the issue by restoring the expected default value for indices created before 7.14, allowing them to open successfully in newer versions. (issue: [#132040](https://github.com/elastic/elasticsearch/issues/132040)) -::: - -:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase -The timeout cancellation callback registered in `QueryPhase` via `addQueryCancellation` was not removed after the query phase completed. -This caused unintended timeouts or cancellations during subsequent phases under specific conditions (such as large datasets, low timeouts, and partial search results enabled). -[#130279](https://github.com/elastic/elasticsearch/pull/130279) resolves the issue and ensures predictable behavior by reintroducing the cleanup logic. (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) -::: +* Fix default missing index sort value of `data_nanos` pre 7.14 [#132162](https://github.com/elastic/elasticsearch/pull/132162) (issue: [#132040](https://github.com/elastic/elasticsearch/issues/132040)) +* Fix missing removal of query cancellation callback in QueryPhase [#130279](https://github.com/elastic/elasticsearch/pull/130279) (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) ## 9.1.0 [elasticsearch-9.1.0-release-notes] @@ -504,7 +513,7 @@ Machine Learning: * Mark token pruning for sparse vector as GA [#128854](https://github.com/elastic/elasticsearch/pull/128854) * Move to the Cohere V2 API for new inference endpoints [#129884](https://github.com/elastic/elasticsearch/pull/129884) * Semantic Text Chunking Indexing Pressure [#125517](https://github.com/elastic/elasticsearch/pull/125517) -* Track memory used in the hierarchical results normalizer [#2831](https://github.com/elastic/ml-cpp/pull/2831) +* Track memory used in the hierarchical results normalizer [#2831](https://github.com/elastic/elasticsearch/pull/2831) * Upgrade AWS v2 SDK to 2.30.38 [#124738](https://github.com/elastic/elasticsearch/pull/124738) * [Inference API] Propagate product use case http header to EIS [#124025](https://github.com/elastic/elasticsearch/pull/124025) * [ML] Add HuggingFace Chat Completion support to the Inference Plugin [#127254](https://github.com/elastic/elasticsearch/pull/127254) @@ -761,113 +770,6 @@ Vector Search: * Fix filtered knn vector search when query timeouts are enabled [#129440](https://github.com/elastic/elasticsearch/pull/129440) * Fix top level knn search with scroll [#126035](https://github.com/elastic/elasticsearch/pull/126035) -## 9.0.5 [elasticsearch-9.0.5-release-notes] - -### Features and enhancements [elasticsearch-9.0.5-features-enhancements] - -Engine: -* The available disk space is now logged when merge tasks are scheduled. If the disk space is below the `indices.merge.disk.watermark.high` threshold, this information is recorded to help diagnose and troubleshoot situations where merges fail due to insufficient disk space. [#131711](https://github.com/elastic/elasticsearch/pull/131711) - -### Fixes [elasticsearch-9.0.5-fixes] - -Aggregations: -:::{dropdown} Validate parent aggregation type in `bucket_script` -The `bucket_script` pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation. -This caused a `ClassCastException` at runtime when the parent was not multi-bucket. -[#132320](https://github.com/elastic/elasticsearch/pull/132320) adds a validation step so the aggregation fails early, preventing the runtime error. (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) -::: - -Data streams: -:::{dropdown} Disables auto-sharding for LOOKUP index mode -Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to `LOOKUP`. -This happened because lookup mappers do not support scaling beyond one replica. -[#131429](https://github.com/elastic/elasticsearch/pull/131429) resolves this issue by disabling auto-sharding for data streams with `LOOKUP` index modes, avoiding unsupported replica settings. -::: - -EQL: -:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in `OR` expressions -Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an `OR` expression. -This occurred because lookup operators were internally rewritten into `IN` expressions, which are unsupported for IP-type fields. -[#132167](https://github.com/elastic/elasticsearch/pull/132167) resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) -::: - -:::{dropdown} Prevent double invocation of EQL listener -In some cases, the EQL listener could be resolved twice, potentially leading to unexpected behavior. -[#124918](https://github.com/elastic/elasticsearch/pull/124918) updates the control flow to exit early and ensure the listener is only invoked once. -::: - -ES|QL: -:::{dropdown} Disallow remote `ENRICH` after `LOOKUP JOIN` -Combining a `LOOKUP JOIN` with remote `ENRICH` could trigger a `ClassCastException` due to pipeline breaker interactions when limits or top-N queries were involved. [#131426](https://github.com/elastic/elasticsearch/pull/131426) adds a validation that forbids remote `ENRICH` after `LOOKUP JOIN`, preventing the runtime error. (issue: [#129372](https://github.com/elastic/elasticsearch/issues/129372)) -::: - -:::{dropdown} Fix `mv_expand` inconsistent column order -The `mv_expand` command could return columns in a different order depending on query execution paths. Now, the new attribute generated by `mv_expand` preserves the original field positions in the output. [#129745](https://github.com/elastic/elasticsearch/pull/129745) (issue: [#129000](https://github.com/elastic/elasticsearch/issues/129000)) -::: - -:::{dropdown} Fixes `ConcurrentModificationException` caused by live operator list -A `ConcurrentModificationException` caused test failures in `CrossClusterAsyncEnrichStopIT.testEnrichAfterStop` under certain conditions. -This happened because the ES|QL driver added a live operator list to the `DriverStatus` object, which could be modified while the status was being serialized. -[#132260](https://github.com/elastic/elasticsearch/pull/132260) fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads. -(issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) -::: - -Infra/Core: -:::{dropdown} Grants server module read/write permissions for deprecated `path.shared_data` setting -The server module is now granted read/write permissions for the deprecated `path.shared_data` setting. -[#131680](https://github.com/elastic/elasticsearch/pull/131680) resolves issues surfaced in internal testing and ensures compatibility with legacy configurations. -::: - -Ingest Node: -:::{dropdown} Correctly handle `download_database_on_pipeline_creation` in default or final pipelines -A bug in the `download_database_on_pipeline_creation` setting caused geoip databases not to download when the geoip processor was referenced from a pipeline processor in a default or final pipeline. -This resulted in documents being tagged with `_geoip_database_unavailable_GeoLite2-City.mmdb` instead of having geo data. -[#131236](https://github.com/elastic/elasticsearch/pull/131236) resolves the issue and ensures geoip databases download correctly in this scenario. -::: - -:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when `mapping_addition` is provided -When using the simulate ingest API with a `mapping_addition`, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed. -This caused mismatches between the index and simulation behavior. -[#132101](https://github.com/elastic/elasticsearch/pull/132101) resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution. -::: - -Machine Learning: -:::{dropdown} Fix memory usage estimation for ELSER models -Using the deployment ID instead of the model ID caused `isElserV1Or2Model` to fail for ELSER models, because deployment IDs don’t start with `.elser_model_2`. -[#131630](https://github.com/elastic/elasticsearch/pull/131630) updates the code to pass the model ID, ensuring memory usage is estimated correctly. -::: - -:::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation -A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections. -[#131990](https://github.com/elastic/elasticsearch/pull/131990) resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments. -::: - -Mapping: -:::{dropdown} Fixes decoding failure for non-ASCII field names in `_ignored_source` -A decoding error occurred when field names in `_ignored_source` contained non-ASCII characters. -This happened because `String.length()` was used to calculate the byte length of the field name, which only works correctly for ASCII characters. -[#132018](https://github.com/elastic/elasticsearch/pull/132018) resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding. -::: - -Search: -:::{dropdown} Correct shard status reporting in point-in-time responses -The Open PIT API incorrectly swapped the skipped and failed shard counts when partial search results were allowed. This caused the API to report failed shards as skipped and vice versa. [#131391](https://github.com/elastic/elasticsearch/pull/131391) fixes the field mapping so shard status is reported accurately. (issue: [#131026](https://github.com/elastic/elasticsearch/issues/131026)) -::: - -:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase -A missing removal of a query cancellation callback caused unintended timeouts or cancellations in later search phases when `allow_partial_search_results` was enabled, which could lead to `ArrayIndexOutOfBoundsException` errors. -[#130279](https://github.com/elastic/elasticsearch/pull/130279) resolves the issue and ensures predictable search execution. (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) -::: - -:::{dropdown} Preserve `boost` and `queryName` for semantic queries -Query rewrite logic dropped `boost` and `queryName` values for `match`, `knn`, and `sparse_vector` queries on `semantic_text` fields, causing query weighting and naming to be lost. [#129282](https://github.com/elastic/elasticsearch/pull/129282) resolves the issue so these values are now preserved correctly during query rewriting. -::: - -Snapshot/Restore: -:::{dropdown} Improve error handling when verifying an empty snapshot repository - -Verifying the integrity of a brand-new snapshot repository without any index blobs failed with a low-level error because the repository generation was `-1`, which cannot be sent over the wire. [#131677](https://github.com/elastic/elasticsearch/pull/131677) updates the logic to reject such requests early with a clearer, more helpful error message. -::: ## 9.0.4 [elasticsearch-9.0.4-release-notes] From 90b0a0fb34b42982a407d9b58888dd13d70e7e98 Mon Sep 17 00:00:00 2001 From: Charlotte Hoblik Date: Tue, 26 Aug 2025 09:42:20 +0200 Subject: [PATCH 2/3] edit breaking changes --- docs/release-notes/breaking-changes.md | 102 +++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 7 deletions(-) diff --git a/docs/release-notes/breaking-changes.md b/docs/release-notes/breaking-changes.md index 5ca82aaff9382..55abec4b5dfa8 100644 --- a/docs/release-notes/breaking-changes.md +++ b/docs/release-notes/breaking-changes.md @@ -34,18 +34,106 @@ No breaking changes in this version. ## 9.1.0 [elasticsearch-9.1.0-breaking-changes] Discovery-Plugins: -* Upgrade `discovery-ec2` to AWS SDK v2 [#122062](https://github.com/elastic/elasticsearch/pull/122062) +:::{dropdown} Migrates `discovery-ec2` plugin to AWS SDK v2 +The `discovery-ec2` plugin now uses AWS SDK v2 instead of v1, as AWS plans to deprecate SDK v1 before the end of Elasticsearch 8.19’s support period. AWS SDK v2 introduces several behavior changes that affect configuration. + +**Impact:** +If you use the `discovery-ec2` plugin, your existing settings may no longer be compatible. Notable changes include, but may not be limited to: +- AWS SDK v2 does not support the EC2 IMDSv1 protocol. +- AWS SDK v2 does not support the `aws.secretKey` or `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system properties. +- AWS SDK v2 does not permit specifying a choice between HTTP and HTTPS so the `discovery.ec2.protocol` setting is no longer effective. +- AWS SDK v2 does not accept an access key without a secret key or vice versa. + +**Action:** +Test the upgrade in a non-production environment. Adapt your configuration to the new SDK functionality. This includes, but may not be limited to, the following items: +- If you use IMDS to determine the availability zone of a node or to obtain credentials for accessing the EC2 API, ensure that it supports the IMDSv2 protocol. +- If applicable, discontinue use of the `aws.secretKey` and `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system properties. +- If applicable, specify that you wish to use the insecure HTTP protocol to access the EC2 API by setting `discovery.ec2.endpoint` to a URL which starts with `http://`. +- Either supply both an access key and a secret key using the `discovery.ec2.access_key` and `discovery.ec2.secret_key` keystore settings, or configure neither of these settings. + +For more information, view [#122062](https://github.com/elastic/elasticsearch/pull/122062). +::: ES|QL: -* Allow partial results by default in ES|QL [#127351](https://github.com/elastic/elasticsearch/pull/127351) (issue: [#122802](https://github.com/elastic/elasticsearch/issues/122802)) -* Disallow brackets in unquoted index patterns [#130427](https://github.com/elastic/elasticsearch/pull/130427) (issue: [#130378](https://github.com/elastic/elasticsearch/issues/130378)) -* Disallow mixed quoted/unquoted patterns in FROM [#127636](https://github.com/elastic/elasticsearch/pull/127636) (issue: [#122651](https://github.com/elastic/elasticsearch/issues/122651)) -* Make `skip_unavailable` catch all errors [#128163](https://github.com/elastic/elasticsearch/pull/128163) -Snapshot/Restore: -* Upgrade `repository-s3` to AWS SDK v2 [#126843](https://github.com/elastic/elasticsearch/pull/126843) (issue: [#120993](https://github.com/elastic/elasticsearch/issues/120993)) +:::{dropdown} ES|QL now returns partial results by default +In previous versions, ES|QL queries failed entirely when any error occurred. As of 8.19.0, ES|QL returns partial results instead. + +**Impact:** +Callers must check the `is_partial` flag in the response to determine whether the result is complete. Relying on full results without checking this flag may lead to incorrect assumptions about the response. + +**Action:** +If partial results are not acceptable for your use case, you can disable this behavior by: +* Setting `allow_partial_results=false` in the query URL per request, or +* Setting the `esql.query.allow_partial_results` cluster setting to `false`. + +For more information, view [#127351](https://github.com/elastic/elasticsearch/pull/127351) (issue: [#122802](https://github.com/elastic/elasticsearch/issues/122802)) +::: + +:::{dropdown} Disallows parentheses in unquoted index patterns in ES|QL +To avoid ambiguity with subquery syntax, ES|QL no longer allows the use of `(` and `)` in unquoted index patterns. + +**Impact:** +Queries that include parentheses in unquoted index names will now result in a parsing exception. + +**Action:** +Update affected queries to quote index names that contain parentheses. For example, use `FROM "("foo")"` instead of `FROM (foo)`. +For more information, view [#130427](https://github.com/elastic/elasticsearch/pull/130427) (issue: [#130378](https://github.com/elastic/elasticsearch/issues/130378)) +::: + +:::{dropdown} Disallows mixing quoted and unquoted components in `FROM` index patterns +ES|QL no longer allows mixing quoted and unquoted parts in `FROM` index patterns (e.g. `FROM remote:"index"`). Previously, such patterns were parsed inconsistently and could result in misleading runtime errors. +**Impact:** +Queries using partially quoted index patterns—such as quoting only the index or only the remote cluster—will now be rejected at parse time. This change simplifies grammar handling and avoids confusing validation failures. + +**Action:** +Ensure index patterns are either fully quoted or fully unquoted. For example: +* Valid: `FROM "remote:index"` or `FROM remote:index` +* Invalid: `FROM remote:"index"`, `FROM "remote":index` + +For more information, view [#127636](https://github.com/elastic/elasticsearch/pull/127636) (issue: [#122651](https://github.com/elastic/elasticsearch/issues/122651)) +::: + +:::{dropdown} `skip_unavailable` now catches all remote cluster runtime errors in ES|QL +When `skip_unavailable` is set to `true`, ES|QL now treats all runtime errors from that cluster as non-fatal. Previously, this setting only applied to connectivity issues (i.e. when a cluster was unavailable). + +**Impact:** +Errors such as missing indices on a remote cluster will no longer cause the query to fail. Instead, the cluster will appear in the response metadata as `skipped` or `partial`. + +**Action:** +If your workflows rely on detecting remote cluster errors, review your use of `skip_unavailable` and adjust error handling as needed. + +For more information, view [#128163](https://github.com/elastic/elasticsearch/pull/128163) +::: + +Snapshot/Restore: +:::{dropdown} Upgrades `repository-s3` plugin to AWS SDK v2 +The `repository-s3` plugin now uses AWS SDK v2 instead of v1, as AWS will deprecate SDK v1 before the end of Elasticsearch 8.19’s support period. The two SDKs differ in behavior, which may require updates to your configuration. + +**Impact:** +Existing `repository-s3` configurations may no longer be compatible. Notable differences in AWS SDK v2 include, but may not be limited to: +- AWS SDK v2 requires users to specify the region to use for signing requests, or else to run in an environment in which it can determine the correct region automatically. The older SDK used to determine the region based on the endpoint URL as specified with the `s3.client.${CLIENT_NAME}.endpoint` setting, together with other data drawn from the operating environment, and fell back to `us-east-1` if no better value was found. +- AWS SDK v2 does not support the EC2 IMDSv1 protocol. +- AWS SDK v2 does not support the `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system property. +- AWS SDK v2 does not permit specifying a choice between HTTP and HTTPS so the `s3.client.${CLIENT_NAME}.protocol` setting is deprecated and no longer has any effect. +- AWS SDK v2 does not permit control over throttling for retries, so the `s3.client.${CLIENT_NAME}.use_throttle_retries` setting is deprecated and no longer has any effect. +- AWS SDK v2 requires the use of the V4 signature algorithm, therefore, the `s3.client.${CLIENT_NAME}.signer_override` setting is deprecated and no longer has any effect. +- AWS SDK v2 does not support the `log-delivery-write` canned ACL. +- AWS SDK v2 counts 4xx responses differently in its metrics reporting. +- AWS SDK v2 always uses the regional STS endpoint, whereas AWS SDK v1 could use either a regional endpoint or the global `https://sts.amazonaws.com` one. + +**Action:** +Test the upgrade in a non-production environment. Adapt your configuration to the new SDK functionality. This includes, but may not be limited to, the following items: +- Specify the correct signing region using the `s3.client.${CLIENT_NAME}.region` setting on each node. {es} will try to determine the correct region based on the endpoint URL and other data drawn from the operating environment, but might not do so correctly in all cases. +- If you use IMDS to determine the availability zone of a node or to obtain credentials for accessing the EC2 API, ensure that it supports the IMDSv2 protocol. +- If applicable, discontinue use of the `com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system property. +- If applicable, specify that you wish to use the insecure HTTP protocol to access the S3 API by setting `s3.client.${CLIENT_NAME}.endpoint` to a URL which starts with `http://`. +- If applicable, discontinue use of the `log-delivery-write` canned ACL. + +For more information, view [#126843](https://github.com/elastic/elasticsearch/pull/126843) (issue: [#120993](https://github.com/elastic/elasticsearch/issues/120993)) +::: ## 9.0.4 [elasticsearch-9.0.4-breaking-changes] From ff25b4ebe0f102f5898e472f8bc822369928a9b7 Mon Sep 17 00:00:00 2001 From: Charlotte Hoblik Date: Tue, 26 Aug 2025 09:48:41 +0200 Subject: [PATCH 3/3] edit release notes --- docs/release-notes/index.md | 226 +++++++++++++++++++++++++++++------- 1 file changed, 186 insertions(+), 40 deletions(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index 3e4a92bdf10c8..4fb04a556ed43 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -70,24 +70,42 @@ Search: ### Features and enhancements [elasticsearch-9.1.2-features-enhancements] Authorization: -* [ExtraHop & QualysGAV] Add `manage`, `create_index`, `read`, `index`, `write`, `delete`, permission for third party agent indices `kibana_system` [#132387](https://github.com/elastic/elasticsearch/pull/132387) (issue: [#131825](https://github.com/elastic/elasticsearch/issues/131825)) +* Adds `manage`, `create_index`, `read`, `index`, `write`, and `delete` privileges for the `kibana_system` role on third-party agent indices used by ExtraHop (`logs-extrahop.investigation-*`) and Qualys GAV (`logs-qualys_gav.asset-*`). This ensures ILM policies can delete these indices without permission errors. [#132387](https://github.com/elastic/elasticsearch/pull/132387) (issue: [#131825](https://github.com/elastic/elasticsearch/issues/131825)) ### Fixes [elasticsearch-9.1.2-fixes] Aggregations: -* Aggs: Add validation to Bucket script pipeline agg [#132320](https://github.com/elastic/elasticsearch/pull/132320) (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) +:::{dropdown} Validates parent aggregation type in `bucket_script` +The `bucket_script` pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation. +This caused a `ClassCastException` at runtime when the parent was not multi-bucket. +[#132320](https://github.com/elastic/elasticsearch/pull/132320) adds a validation step so the aggregation fails early, preventing the runtime error. (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) +::: Codec: -* Use local segment `fieldInfos` to lookup tsdb merge stats [#132597](https://github.com/elastic/elasticsearch/pull/132597) +:::{dropdown} Uses local segment `fieldInfos` for TSDB merge stats +Merging shrink TSDB or LogsDB indices in versions 8.19 or 9.1+ could fail when using `addIndexes` to combine Lucene segments directly. +In these cases, the `fieldInfos` value could differ between shards and the merged segment, causing incorrect merge statistics. +PR [#132597](https://github.com/elastic/elasticsearch/pull/132597) updates the process to use `fieldInfos` from each segment instead of the merged segment, ensuring accurate stats and preventing merge failures. +::: ES|QL: -* Small fixes for COPY_SIGN [#132459](https://github.com/elastic/elasticsearch/pull/132459) +:::{dropdown} Fixes for `COPY_SIGN` function in ESQL +The `COPY_SIGN` function has been updated to better support the literal `NULL` in parameters. +[#132459](https://github.com/elastic/elasticsearch/pull/132459) +::: Mapping: -* Strings outside BMP have 2 chars per code points [#132593](https://github.com/elastic/elasticsearch/pull/132593) +:::{dropdown} Calculates text string length correctly for code points outside BMP +Strings parsed with the optimized UTF-8 parsing path had incorrect length calculations for characters outside the basic multilingual plane (BMP). +These characters require two UTF-16 code units, but the optimized path did not account for this, causing mismatches with the non-optimized path. +[#132593](https://github.com/elastic/elasticsearch/pull/132593) fixes the calculation to ensure consistent and correct string lengths. +::: Search: -* Always stop the timer when profiling the fetch phase [#132570](https://github.com/elastic/elasticsearch/pull/132570) +:::{dropdown} Always stops the timer when profiling the fetch phase +Exceptions in fetch sub-phases (for example, `setNextReader`) left the profiling timer running, causing mismatched start/stop calls and errors. +[#132570](https://github.com/elastic/elasticsearch/pull/132570) ensures the `timer.stop()` call always stops. +::: ## 9.0.5 [elasticsearch-9.0.5-release-notes] @@ -95,46 +113,108 @@ Search: ### Features and enhancements [elasticsearch-9.0.5-features-enhancements] Engine: -* Track & log when there is insufficient disk space available to execute merges [#131711](https://github.com/elastic/elasticsearch/pull/131711) +* The available disk space is now logged when merge tasks are scheduled. If the disk space is below the `indices.merge.disk.watermark.high` threshold, this information is recorded to help diagnose and troubleshoot situations where merges fail due to insufficient disk space. [#131711](https://github.com/elastic/elasticsearch/pull/131711) ### Fixes [elasticsearch-9.0.5-fixes] Aggregations: -* Aggs: Add validation to Bucket script pipeline agg [#132320](https://github.com/elastic/elasticsearch/pull/132320) (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) +:::{dropdown} Validate parent aggregation type in `bucket_script` +The `bucket_script` pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation. +This caused a `ClassCastException` at runtime when the parent was not multi-bucket. +[#132320](https://github.com/elastic/elasticsearch/pull/132320) adds a validation step so the aggregation fails early, preventing the runtime error. (issue: [#132272](https://github.com/elastic/elasticsearch/issues/132272)) +::: Data streams: -* Prevent auto-sharding for data streams in LOOKUP index mode [#131429](https://github.com/elastic/elasticsearch/pull/131429) +:::{dropdown} Disables auto-sharding for LOOKUP index mode +Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to `LOOKUP`. +This happened because lookup mappers do not support scaling beyond one replica. +[#131429](https://github.com/elastic/elasticsearch/pull/131429) resolves this issue by disabling auto-sharding for data streams with `LOOKUP` index modes, avoiding unsupported replica settings. +::: EQL: -* Deal with internally created IN in a different way for EQL [#132167](https://github.com/elastic/elasticsearch/pull/132167) (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) -* Fix EQL double invoking listener [#124918](https://github.com/elastic/elasticsearch/pull/124918) +:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in `OR` expressions +Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an `OR` expression. +This occurred because lookup operators were internally rewritten into `IN` expressions, which are unsupported for IP-type fields. +[#132167](https://github.com/elastic/elasticsearch/pull/132167) resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) +::: + +:::{dropdown} Prevent double invocation of EQL listener +In some cases, the EQL listener could be resolved twice, potentially leading to unexpected behavior. +[#124918](https://github.com/elastic/elasticsearch/pull/124918) updates the control flow to exit early and ensure the listener is only invoked once. +::: ES|QL: -* Disallow remote enrich after lu join [#131426](https://github.com/elastic/elasticsearch/pull/131426) (issue: [#129372](https://github.com/elastic/elasticsearch/issues/129372)) -* ESQL: Fix `mv_expand` inconsistent column order [#129745](https://github.com/elastic/elasticsearch/pull/129745) (issue: [#129000](https://github.com/elastic/elasticsearch/issues/129000)) -* FIx Driver creating status with a live list of operators [#132260](https://github.com/elastic/elasticsearch/pull/132260) (issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) +:::{dropdown} Disallow remote `ENRICH` after `LOOKUP JOIN` +Combining a `LOOKUP JOIN` with remote `ENRICH` could trigger a `ClassCastException` due to pipeline breaker interactions when limits or top-N queries were involved. [#131426](https://github.com/elastic/elasticsearch/pull/131426) adds a validation that forbids remote `ENRICH` after `LOOKUP JOIN`, preventing the runtime error. (issue: [#129372](https://github.com/elastic/elasticsearch/issues/129372)) +::: + +:::{dropdown} Fix `mv_expand` inconsistent column order +The `mv_expand` command could return columns in a different order depending on query execution paths. Now, the new attribute generated by `mv_expand` preserves the original field positions in the output. [#129745](https://github.com/elastic/elasticsearch/pull/129745) (issue: [#129000](https://github.com/elastic/elasticsearch/issues/129000)) +::: + +:::{dropdown} Fixes `ConcurrentModificationException` caused by live operator list +A `ConcurrentModificationException` caused test failures in `CrossClusterAsyncEnrichStopIT.testEnrichAfterStop` under certain conditions. +This happened because the ES|QL driver added a live operator list to the `DriverStatus` object, which could be modified while the status was being serialized. +[#132260](https://github.com/elastic/elasticsearch/pull/132260) fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads. +(issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) +::: Infra/Core: -* Grant server module read/write entitlements for deprecated path setting "path.shared_data" [#131680](https://github.com/elastic/elasticsearch/pull/131680) +:::{dropdown} Grants server module read/write permissions for deprecated `path.shared_data` setting +The server module is now granted read/write permissions for the deprecated `path.shared_data` setting. +[#131680](https://github.com/elastic/elasticsearch/pull/131680) resolves issues surfaced in internal testing and ensures compatibility with legacy configurations. +::: Ingest Node: -* Correctly handling `download_database_on_pipeline_creation` within a pipeline processor within a default or final pipeline [#131236](https://github.com/elastic/elasticsearch/pull/131236) -* Simulate ingest API uses existing index mapping when `mapping_addition` is given [#132101](https://github.com/elastic/elasticsearch/pull/132101) +:::{dropdown} Correctly handle `download_database_on_pipeline_creation` in default or final pipelines +A bug in the `download_database_on_pipeline_creation` setting caused geoip databases not to download when the geoip processor was referenced from a pipeline processor in a default or final pipeline. +This resulted in documents being tagged with `_geoip_database_unavailable_GeoLite2-City.mmdb` instead of having geo data. +[#131236](https://github.com/elastic/elasticsearch/pull/131236) resolves the issue and ensures geoip databases download correctly in this scenario. +::: + +:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when `mapping_addition` is provided +When using the simulate ingest API with a `mapping_addition`, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed. +This caused mismatches between the index and simulation behavior. +[#132101](https://github.com/elastic/elasticsearch/pull/132101) resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution. +::: Machine Learning: -* Fix memory usage estimation for ELSER models [#131630](https://github.com/elastic/elasticsearch/pull/131630) -* Prevent the trained model deployment memory estimation from double-counting allocations [#131990](https://github.com/elastic/elasticsearch/pull/131990) +:::{dropdown} Fix memory usage estimation for ELSER models +Using the deployment ID instead of the model ID caused `isElserV1Or2Model` to fail for ELSER models, because deployment IDs don’t start with `.elser_model_2`. +[#131630](https://github.com/elastic/elasticsearch/pull/131630) updates the code to pass the model ID, ensuring memory usage is estimated correctly. +::: + +:::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation +A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections. +[#131990](https://github.com/elastic/elasticsearch/pull/131990) resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments. +::: Mapping: -* Fix decoding of non-ascii field names in ignored source [#132018](https://github.com/elastic/elasticsearch/pull/132018) +:::{dropdown} Fixes decoding failure for non-ASCII field names in `_ignored_source` +A decoding error occurred when field names in `_ignored_source` contained non-ASCII characters. +This happened because `String.length()` was used to calculate the byte length of the field name, which only works correctly for ASCII characters. +[#132018](https://github.com/elastic/elasticsearch/pull/132018) resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding. +::: Search: -* Fix bug in point in time response [#131391](https://github.com/elastic/elasticsearch/pull/131391) (issue: [#131026](https://github.com/elastic/elasticsearch/issues/131026)) -* Fix missing removal of query cancellation callback in QueryPhase [#130279](https://github.com/elastic/elasticsearch/pull/130279) (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) -* Fix query rewrite logic to preserve `boosts` and `queryName` for `match`, `knn`, and `sparse_vector` queries on semantic_text fields [#129282](https://github.com/elastic/elasticsearch/pull/129282) +:::{dropdown} Correct shard status reporting in point-in-time responses +The Open PIT API incorrectly swapped the skipped and failed shard counts when partial search results were allowed. This caused the API to report failed shards as skipped and vice versa. [#131391](https://github.com/elastic/elasticsearch/pull/131391) fixes the field mapping so shard status is reported accurately. (issue: [#131026](https://github.com/elastic/elasticsearch/issues/131026)) +::: + +:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase +A missing removal of a query cancellation callback caused unintended timeouts or cancellations in later search phases when `allow_partial_search_results` was enabled, which could lead to `ArrayIndexOutOfBoundsException` errors. +[#130279](https://github.com/elastic/elasticsearch/pull/130279) resolves the issue and ensures predictable search execution. (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) +::: + +:::{dropdown} Preserve `boost` and `queryName` for semantic queries +Query rewrite logic dropped `boost` and `queryName` values for `match`, `knn`, and `sparse_vector` queries on `semantic_text` fields, causing query weighting and naming to be lost. [#129282](https://github.com/elastic/elasticsearch/pull/129282) resolves the issue so these values are now preserved correctly during query rewriting. +::: Snapshot/Restore: -* Throw better exception if verifying empty repo [#131677](https://github.com/elastic/elasticsearch/pull/131677) +:::{dropdown} Improve error handling when verifying an empty snapshot repository + +Verifying the integrity of a brand-new snapshot repository without any index blobs failed with a low-level error because the repository generation was `-1`, which cannot be sent over the wire. [#131677](https://github.com/elastic/elasticsearch/pull/131677) updates the logic to reject such requests early with a clearer, more helpful error message. +::: ## 9.1.1 [elasticsearch-9.1.1-release-notes] @@ -142,35 +222,101 @@ Snapshot/Restore: ### Fixes [elasticsearch-9.1.1-fixes] Data streams: -* Prevent auto-sharding for data streams in LOOKUP index mode [#131429](https://github.com/elastic/elasticsearch/pull/131429) +:::{dropdown} Disables auto-sharding for LOOKUP index mode +Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to `LOOKUP`. +This happened because lookup mappers do not support scaling beyond one replica. +[#131429](https://github.com/elastic/elasticsearch/pull/131429) resolves this issue by disabling auto-sharding for data streams with `LOOKUP` index mode, avoiding unsupported replica settings. +::: EQL: -* Deal with internally created IN in a different way for EQL [#132167](https://github.com/elastic/elasticsearch/pull/132167) (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) +:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in `OR` expressions +Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an `OR` expression. +This occurred because lookup operators were internally rewritten into `IN` expressions, which are unsupported for IP-type fields. +[#132167](https://github.com/elastic/elasticsearch/pull/132167) resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: [#118621](https://github.com/elastic/elasticsearch/issues/118621)) +::: ES|QL: -* Change equals and hashcode for `ConstantNullBlock` [#131817](https://github.com/elastic/elasticsearch/pull/131817) -* FIx Driver creating status with a live list of operators [#132260](https://github.com/elastic/elasticsearch/pull/132260) (issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) -* Fix NPE on empty to_lower/to_upper call [#131917](https://github.com/elastic/elasticsearch/pull/131917) (issue: [#131913](https://github.com/elastic/elasticsearch/issues/131913)) -* Fix `aggregate_metric_double` sorting and `mv_expand` issues [#131658](https://github.com/elastic/elasticsearch/pull/131658) -* Fix combine result for `ingest_took` [#132088](https://github.com/elastic/elasticsearch/pull/132088) -* Restrict remote ENRICH after FORK [#131945](https://github.com/elastic/elasticsearch/pull/131945) (issue: [#131445](https://github.com/elastic/elasticsearch/issues/131445)) -* Split large pages on load sometimes [#131053](https://github.com/elastic/elasticsearch/pull/131053) +:::{dropdown} Fixes inconsistent equality and hashcode behavior for `ConstantNullBlock` +Inconsistent equality checks caused `constantNullBlock.equals(anyDoubleBlock)` to return false, even when `doubleBlock.equals(constantNullBlock)` returned true. +This asymmetry led to unreliable comparisons and mismatched hashcodes when `ConstantNullBlock` was functionally equivalent to other standard blocks. +[#131817](https://github.com/elastic/elasticsearch/pull/131817) resolves the issue and ensures both equality and hashcode functions are symmetric for these block types. +::: + +:::{dropdown} Fixes `ConcurrentModificationException` caused by live operator list +A `ConcurrentModificationException` caused test failures in `CrossClusterAsyncEnrichStopIT.testEnrichAfterStop` under certain conditions. +This happened because the ES|QL driver added a live operator list to the `DriverStatus` object, which could be modified while the status was being serialized. +[#132260](https://github.com/elastic/elasticsearch/pull/132260) fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads. +(issue: [#131564](https://github.com/elastic/elasticsearch/issues/131564)) +::: + +:::{dropdown} Prevents null pointer exception for `to_lower` and `to_upper` with no parameters +Calling the `to_lower` or `to_upper` functions with no parameters caused a null pointer exception (NPE), instead of returning a clear error. +This behavior was a result of an older implementation of these functions. +[#131917](https://github.com/elastic/elasticsearch/pull/131917) resolves the issue and ensures that empty parameter calls now return the correct error message. (issue: [#131913](https://github.com/elastic/elasticsearch/issues/131913)) +::: + +:::{dropdown} Fixes `aggregate_metric_double` decoding and `mv_expand` behavior on multi-index queries +Sorting across multiple indices failed when one index contained an `aggregate_metric_double` field and another did not. +In this case, the missing field was encoded as `NullBlock` but later incorrectly decoded as `AggregateMetricDoubleBlock`, which expects four values. This mismatch caused decoding errors. +[#131658](https://github.com/elastic/elasticsearch/pull/131658) resolves the issue and also improves `mv_expand` by returning the input block unchanged for unsupported `AggregateMetricDoubleBlock` values, avoiding unnecessary errors. +::: + +:::{dropdown} Fixes incorrect `ingest_took` value when combining bulk responses +Combining two `BulkResponse` objects with `ingest_took` set to `NO_INGEST_TOOK` resulted in a combined `ingest_took` value of `-2`, which was invalid. +This occurred because the combination logic failed to preserve the sentinel `NO_INGEST_TOOK` constant. +[#132088](https://github.com/elastic/elasticsearch/pull/132088) resolves the issue and ensures the result is correctly set to `NO_INGEST_TOOK` when applicable. +::: + +:::{dropdown} Disallows remote ENRICH after FORK in query plans +An invalid combination of `FORK` followed by a remote-only `ENRICH` caused incorrect query planning and failed executions. [#131945](https://github.com/elastic/elasticsearch/pull/131945) resolves the issue by explicitly disallowing this combination, preventing invalid plans from being executed. (issue: [#131445](https://github.com/elastic/elasticsearch/issues/131445)) +::: + +:::{dropdown} Adds support for splitting large pages on load to avoid memory pressure +Loading large rows from a single segment occasionally created oversized pages when decoding values row-by-row, particularly for text and geo fields. +This could cause memory pressure or degraded performance. +[#131053](https://github.com/elastic/elasticsearch/pull/131053) resolves the issue by estimating the size of each page as rows are loaded. +If the estimated size exceeds a configurable `jumbo` threshold (defaulting to one megabyte), row loading stops early, the page is returned, and remaining rows are processed in subsequent iterations. +This prevents loading incomplete or oversized pages during data aggregation. +::: Infra/Core: -* Grant server module read/write entitlements for deprecated path setting "path.shared_data" [#131680](https://github.com/elastic/elasticsearch/pull/131680) +:::{dropdown} Grants server module read/write permissions for deprecated `path.shared_data` setting +Grants the server module read/write access to the deprecated `path.shared_data` setting. +[#131680](https://github.com/elastic/elasticsearch/pull/131680) resolves issues surfaced in internal testing and ensures compatibility with legacy configurations. +::: Ingest Node: -* Simulate ingest API uses existing index mapping when `mapping_addition` is given [#132101](https://github.com/elastic/elasticsearch/pull/132101) +:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when `mapping_addition` is provided +When using the simulate ingest API with a `mapping_addition`, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed. +This caused mismatches between the index and simulation behavior. +[#132101](https://github.com/elastic/elasticsearch/pull/132101) resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution. +::: Machine Learning: -* Prevent the trained model deployment memory estimation from double-counting allocations [#131990](https://github.com/elastic/elasticsearch/pull/131990) +:::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation +A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections. +[#131990](https://github.com/elastic/elasticsearch/pull/131990) resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments. +::: Mapping: -* Fix decoding of non-ascii field names in ignored source [#132018](https://github.com/elastic/elasticsearch/pull/132018) +:::{dropdown} Fixes decoding failure for non-ASCII field names in `_ignored_source` +A decoding error occurred when field names in `_ignored_source` contained non-ASCII characters. +This happened because `String.length()` was used to calculate the byte length of the field name, which only works correctly for ASCII characters. +[#132018](https://github.com/elastic/elasticsearch/pull/132018) resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding. +::: Search: -* Fix default missing index sort value of `data_nanos` pre 7.14 [#132162](https://github.com/elastic/elasticsearch/pull/132162) (issue: [#132040](https://github.com/elastic/elasticsearch/issues/132040)) -* Fix missing removal of query cancellation callback in QueryPhase [#130279](https://github.com/elastic/elasticsearch/pull/130279) (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) +:::{dropdown} Fixes index sort compatibility for `date_nanos` fields in indices created before 7.14 +Indices created prior to version 7.14 that used an index sort on a `date_nanos` field could not be opened in more recent versions due to a mismatch in the default `index.sort.missing` value. +A change in version 7.14 modified the default from `Long.MIN_VALUE` to `0L`, which caused newer versions to reject those older indices. +[#132162](https://github.com/elastic/elasticsearch/pull/132162) resolves the issue by restoring the expected default value for indices created before 7.14, allowing them to open successfully in newer versions. (issue: [#132040](https://github.com/elastic/elasticsearch/issues/132040)) +::: + +:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase +The timeout cancellation callback registered in `QueryPhase` via `addQueryCancellation` was not removed after the query phase completed. +This caused unintended timeouts or cancellations during subsequent phases under specific conditions (such as large datasets, low timeouts, and partial search results enabled). +[#130279](https://github.com/elastic/elasticsearch/pull/130279) resolves the issue and ensures predictable behavior by reintroducing the cleanup logic. (issue: [#130071](https://github.com/elastic/elasticsearch/issues/130071)) +::: ## 9.1.0 [elasticsearch-9.1.0-release-notes] @@ -513,7 +659,7 @@ Machine Learning: * Mark token pruning for sparse vector as GA [#128854](https://github.com/elastic/elasticsearch/pull/128854) * Move to the Cohere V2 API for new inference endpoints [#129884](https://github.com/elastic/elasticsearch/pull/129884) * Semantic Text Chunking Indexing Pressure [#125517](https://github.com/elastic/elasticsearch/pull/125517) -* Track memory used in the hierarchical results normalizer [#2831](https://github.com/elastic/elasticsearch/pull/2831) +* Track memory used in the hierarchical results normalizer [#2831](https://github.com/elastic/ml-cpp/pull/2831) * Upgrade AWS v2 SDK to 2.30.38 [#124738](https://github.com/elastic/elasticsearch/pull/124738) * [Inference API] Propagate product use case http header to EIS [#124025](https://github.com/elastic/elasticsearch/pull/124025) * [ML] Add HuggingFace Chat Completion support to the Inference Plugin [#127254](https://github.com/elastic/elasticsearch/pull/127254)