Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
207 changes: 206 additions & 1 deletion docs/reference/migration/migrate_8_19.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,210 @@ coming::[8.19.0]
[[breaking-changes-8.19]]
=== Breaking changes

There are no breaking changes in {es} 8.19.
The following changes in {es} 8.19 might affect your applications
and prevent them from operating normally.
Before upgrading to 8.19, review these changes and take the described steps
to mitigate the impact.

[discrete]
[[breaking_819_cluster_and_node_setting_changes]]
==== Cluster and node setting changes

[[upgrade_discovery_ec2_to_aws_sdk_v2]]
.Upgrade `discovery-ec2` to AWS SDK v2
[%collapsible]
====
*Details* +
In earlier versions of {es} the `discovery-ec2` plugin was based on the AWS SDK v1. AWS will withdraw support for this SDK before the end of the life of {es} {minor-version} so we must migrate to the newer AWS SDK v2.
Unfortunately there are several differences between the two AWS SDK versions which may require you to adjust your system configuration when upgrading to {es} {minor-version} or later. These differences include, but may not be limited to, the following items.
* AWS SDK v2 does not support the EC2 IMDSv1 protocol.
* AWS SDK v2 does not support the `aws.secretKey` or
`com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system properties.
* AWS SDK v2 does not permit specifying a choice between HTTP and HTTPS so
the `discovery.ec2.protocol` setting is no longer effective.
* AWS SDK v2 does not accept an access key without a secret key or vice
versa.
*Impact* +
If you use the `discovery-ec2` plugin, test your upgrade thoroughly before upgrading any production workloads.
Adapt your configuration to the new SDK functionality. This includes, but may not be limited to, the following items.
* If you use IMDS to determine the availability zone of a node or to obtain
credentials for accessing the EC2 API, ensure that it supports the IMDSv2
protocol.
* If applicable, discontinue use of the `aws.secretKey` and
`com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system properties.
* If applicable, specify that you wish to use the insecure HTTP protocol to
access the EC2 API by setting `discovery.ec2.endpoint` to a URL which
starts with `http://`.
* Either supply both an access key and a secret key using the keystore
settings `discovery.ec2.access_key` and `discovery.ec2.secret_key`, or
configure neither of these settings.
====

[[upgrade_repository_s3_to_aws_sdk]]
.Upgrade `repository-s3` to AWS SDK v2
[%collapsible]
====
*Details* +
In earlier versions of {es} the `repository-s3` plugin was based on the AWS SDK v1. AWS will withdraw support for this SDK before the end of the life of {es} {minor-version} so we must migrate to the newer AWS SDK v2.
Unfortunately there are several differences between the two AWS SDK versions which may require you to adjust your system configuration when upgrading to {es} {minor-version} or later. These differences include, but may not be limited to, the following items.
* AWS SDK v2 requires users to specify the region to use for signing
requests, or else to run in an environment in which it can determine the
correct region automatically. The older SDK would try to determine the
region based on the endpoint URL as specified with the
`s3.client.${CLIENT_NAME}.endpoint` setting, together with other data
drawn from the operating environment, and would ultimately fall back to
`us-east-1` if no better value could be found.
* AWS SDK v2 does not support the EC2 IMDSv1 protocol.
* AWS SDK v2 does not support the
`com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system property.
* AWS SDK v2 does not permit specifying a choice between HTTP and HTTPS so
the `s3.client.${CLIENT_NAME}.protocol` setting is deprecated.
* AWS SDK v2 does not permit control over throttling for retries, so the
the `s3.client.${CLIENT_NAME}.use_throttle_retries` setting is deprecated
and no longer has any effect.
* AWS SDK v2 requires the use of the V4 signature algorithm, so the
`s3.client.${CLIENT_NAME}.signer_override` setting is deprecated and no
longer has any effect.
* AWS SDK v2 does not support the `log-delivery-write` canned ACL.
* AWS SDK v2 counts 4xx responses differently in its metrics reporting.
* AWS SDK v2 always uses the regional STS endpoint, whereas AWS SDK v2
could use either a regional endpoint or the global
`https://sts.amazonaws.com` one.
*Impact* +
If you use the `repository-s3` module, test your upgrade thoroughly before upgrading any production workloads.
Adapt your configuration to the new SDK functionality. This includes, but may not be limited to, the following items.
* Specify the correct signing region using the
`s3.client.${CLIENT_NAME}.region` setting on each node. {es} will try and
determine the correct region based on the endpoint URL and other data
drawn from the operating environment but cannot guarantee to do so
correctly in all cases.
* If you use IMDS to determine the availability zone of a node or to obtain
credentials for accessing the EC2 API, ensure that it supports the IMDSv2
protocol.
* If applicable, discontinue use of the
`com.amazonaws.sdk.ec2MetadataServiceEndpointOverride` system property.
* If applicable, specify the protocol to use to access the S3 API by
setting `s3.client.${CLIENT_NAME}.endpoint` to a URL which starts with
`http://` or `https://`.
* If applicable, discontinue use of the `log-delivery-write` canned ACL.
====

[discrete]
[[breaking_819_es_ql_changes]]
==== ES|QL changes

[[allow_partial_results_by_default_in_es_ql]]
.Allow partial results by default in ES|QL
[%collapsible]
====
*Details* +
In earlier versions of {es}, ES|QL would fail the entire query if it encountered any error. ES|QL now returns partial results instead of failing when encountering errors.
*Impact* +
Callers should check the `is_partial` flag returned in the response to determine if the result is partial or complete. If returning partial results is not desired, this option can be overridden per request via an `allow_partial_results` parameter in the query URL or globally via the cluster setting `esql.query.allow_partial_results`.
====

[[cluster_setting_skip_unavailable_catches_all_runtime_errors]]
.Cluster setting "skip_unavailable" catches all runtime errors
[%collapsible]
====
*Details* +
If `skip_unavailable` is set to `true`, the runtime errors from this cluster do not lead to a failure of the query. Instead, the cluster is set to `skipped` or `partial` status, and the query execution continues. This is a breaking change from previous versions, where `skip_unavailable` only applied to errors related to a cluster being unavailable.
*Impact* +
The errors on remote clusters, e.g. missing indices, will not lead to a failure of the query. Instead, the cluster is set to `skipped` or `partial` status in the response metadata.
====

[[disallow_mixed_quoted_unquoted_patterns_in_from]]
.Disallow mixed quoted/unquoted patterns in FROM
[%collapsible]
====
*Details* +
Previously, the ES|QL grammar allowed users to individually quote constituent strings in index patterns such as "remote_cluster":"index_name". This would allow users to write complex malformed index patterns that often slip through grammar and the subsequent validation. This could result in runtime errors that can be misleading. This change simplifies the grammar to early reject such malformed index patterns at the parsing stage, allowing users to write simpler queries and see more relevant and meaningful errors.
*Impact* +
Users can write queries with simpler index patterns and see more meaningful and relevant errors.
====

[[unquoted_index_patterns_do_not_allow_characters]]
.Unquoted index patterns do not allow `(` and `)` characters
[%collapsible]
====
*Details* +
Previously, ES|QL accepted unquoted index patterns containing brackets, such as `FROM index(1) | ENRICH policy(2)`.
This query syntax is no longer valid because it could conflict with subquery syntax, where brackets are used as delimiters.
Brackets are now only allowed in quoted index patterns. For example: `FROM "index(1)" | ENRICH "policy(2)"`.
*Impact* +
This affects existing queries containing brackets in index or policy names, i.e. in FROM, ENRICH, and LOOKUP JOIN commands.
====


[discrete]
[[deprecated-8.19]]
=== Deprecations

The following functionality has been deprecated in {es} 8.19
and will be removed in a future version.
While this won't have an immediate impact on your applications,
we strongly encourage you to take the described steps to update your code
after upgrading to 8.19.

To find out if you are using any deprecated functionality,
enable <<deprecation-logging, deprecation logging>>.

[discrete]
[[deprecations_819_ingest]]
==== Ingest deprecations

[[deprecate_indices_merge_scheduler_use_thread_pool_setting]]
.Deprecate `indices.merge.scheduler.use_thread_pool` setting
[%collapsible]
====
*Details* +
This deprecates the `indices.merge.scheduler.use_thread_pool` node setting that was introduced in
*Impact* +
There should be no impact to users since the setting was not released before its deprecation here (and is not documented).
====

[discrete]
[[deprecations_819_rest_api]]
==== REST API deprecations

[[deprecate_ability_to_connect_to_nodes_of_versions_8_18_earlier]]
.Deprecate ability to connect to nodes of versions 8.18 and earlier
[%collapsible]
====
*Details* +
Versions 9.1.0 and later of {es} will not support communication with nodes of versions earlier than 8.19.0, so the ability to connect to nodes of earlier versions is deprecated in this version. This applies both to communication within a cluster and communication across clusters (e.g. for <<modules-cross-cluster-search,{ccs}>> or <<xpack-ccr,{ccr}>>).
{es} will report in its <<deprecation-logging, deprecation logging>> each time it opens a connection to a node that will not be supported from version 9.1.0 onwards. You must upgrade all your clusters to version 8.19.0 or later before upgrading any of your clusters to 9.1.0 or later.
*Impact* +
Upgrade all of your clusters to at least 8.19.0 before upgrading any of them to 9.1.0 or later.
====

Loading