Forward port release notes for v8.17.8

docs/reference/release-notes.asciidoc

@@ -10,6 +10,8 @@ This section summarizes the changes in each release.
 * <<release-notes-8.18.2>>
 * <<release-notes-8.18.1>>
 * <<release-notes-8.18.0>>
+* <<release-notes-8.17.8>>
+* <<release-notes-8.17.7>>
 * <<release-notes-8.17.6>>
 * <<release-notes-8.17.5>>
 * <<release-notes-8.17.4>>
@@ -97,6 +99,8 @@ include::release-notes/8.19.0.asciidoc[]
 include::release-notes/8.18.2.asciidoc[]
 include::release-notes/8.18.1.asciidoc[]
 include::release-notes/8.18.0.asciidoc[]
+include::release-notes/8.17.8.asciidoc[]
+include::release-notes/8.17.7.asciidoc[]
 include::release-notes/8.17.6.asciidoc[]
 include::release-notes/8.17.5.asciidoc[]
 include::release-notes/8.17.4.asciidoc[]

docs/reference/release-notes/8.17.7.asciidoc

@@ -0,0 +1,37 @@
+[[release-notes-8.17.7]]
+== {es} version 8.17.7
+
+coming[8.17.7]
+
+Also see <<breaking-changes-8.17,Breaking changes in 8.17>>.
+
+[[breaking-8.17.7]]
+[float]
+=== Breaking changes
+
+Snapshot/Restore::
+* Make S3 custom query parameter optional {es-pull}128043[#128043]
+
+[[bug-8.17.7]]
+[float]
+=== Bug fixes
+
+Audit::
+* Handle streaming request body in audit log {es-pull}127798[#127798]
+
+Downsampling::
+* Downsampling does not consider passthrough fields as dimensions {es-pull}127752[#127752] (issue: {es-issue}125156[#125156])
+
+Infra/Scripting::
+* Avoid nested docs in painless execute api {es-pull}127991[#127991] (issue: {es-issue}41004[#41004])
+
+Relevance::
+* Fix: Add `NamedWriteable` for `RuleQueryRankDoc` {es-pull}128153[#128153] (issue: {es-issue}126071[#126071])
+
+Security::
+* Remove dangling spaces wherever found {es-pull}127475[#127475]
+
+TSDB::
+* Skip the validation when retrieving the index mode during reindexing a time series data stream {es-pull}127824[#127824]
+
+

docs/reference/release-notes/8.17.8.asciidoc

@@ -0,0 +1,45 @@
+[[release-notes-8.17.8]]
+== {es} version 8.17.8
+
+Also see <<breaking-changes-8.17,Breaking changes in 8.17>>.
+
+[[bug-8.17.8]]
+[float]
+=== Bug fixes
+
+Aggregations::
+* Aggs: Fix significant terms not finding background docuemnts for nested fields {es-pull}128472[#128472] (issue: {es-issue}101163[#101163])
+
+Authorization::
+* Prevent invalid privileges in manage roles privilege {es-pull}128532[#128532] (issue: {es-issue}127496[#127496])
+
+Codec::
+* Use new source loader when lower `docId` is accessed {es-pull}128320[#128320]
+
+Data streams::
+* Bump APM data resource version {es-pull}129539[#129539]
+
+Infra/Core::
+* Update AbstractXContentParser to support parsers that don't provide text characters {es-pull}129005[#129005]
+
+Infra/Scripting::
+* Register mustache size limit setting {es-pull}119291[#119291]
+
+Machine Learning::
+* Account for Java direct memory on machine learning nodes to prevent out-of-memory crashes. {es-pull}128742[#128742]
+* Ensure that anomaly detection job state update retries if master node is temoporarily unavailable {es-pull}129391[#129391] (issue: {es-issue}126148[#126148])
+
+Searchable Snapshots::
+* Adjust unpromotable shard refresh request validation to allow `RefreshResult.NO_REFRESH` {es-pull}129176[#129176] (issue: {es-issue}129036[#129036])
+
+Security::
+* Fix error message when changing the password for a user in the file realm {es-pull}127621[#127621]
+
+[[enhancement-8.17.8]]
+[float]
+=== Enhancements
+
+Authorization::
+* Fix unsupported privileges error message during role and API key creation {es-pull}129158[#129158] (issue: {es-issue}128132[#128132])
+
+

docs/reference/release-notes/highlights.asciidoc

@@ -35,6 +35,110 @@ endif::[]
 
 // tag::notable-highlights[]
 
+[discrete]
+[[upgrade_repository_s3_to_aws_sdk_v2]]
+=== Upgrade `repository-s3` to AWS SDK v2
+In earlier versions of {es} the `repository-s3` plugin was based on the AWS SDK v1. AWS will withdraw support for this SDK before the end of the life of {es} {minor-version} so we have migrated this plugin to the newer AWS SDK v2.
+The two SDKs are not quite compatible, so please check the breaking changes documentation and test the new version thoroughly before upgrading any production workloads.
+
+{es-pull}126843[#126843]
+
+[discrete]
+[[add_ability_to_redirect_ingestion_failures_on_data_streams_to_failure_store]]
+=== Add ability to redirect ingestion failures on data streams to a failure store
+Documents that encountered ingest pipeline failures or mapping conflicts
+would previously be returned to the client as errors in the bulk and
+index operations. Many client applications are not equipped to respond
+to these failures. This leads to the failed documents often being
+dropped by the client which cannot hold the broken documents
+indefinitely. In many end user workloads, these failed documents
+represent events that could be critical signals for observability or
+security use cases.
+
+To help mitigate this problem, data streams can now maintain a "failure
+store" which is used to accept and hold documents that fail to be
+ingested due to preventable configuration errors. The data stream's
+failure store operates like a separate set of backing indices with their
+own mappings and access patterns that allow Elasticsearch to accept
+documents that would otherwise be rejected due to unhandled ingest
+pipeline exceptions or mapping conflicts.
+
+Users can enable redirection of ingest failures to the failure store on
+new data streams by specifying it in the new `data_stream_options` field
+inside of a component or index template:
+
+[source,yaml]
+----
+PUT _index_template/my-template
+{
+  "index_patterns": ["logs-test-*"],
+  "data_stream": {},
+  "template": {
+    "data_stream_options": {
+      "failure_store": {
+        "enabled": true
+      }
+    }
+  }
+}'
+----
+
+Existing data streams can be configured with the new data stream
+`_options` endpoint:
+
+[source,yaml]
+----
+PUT _data_stream/logs-test-apache/_options
+{
+  "failure_store": {
+    "enabled": "true"
+  }
+}
+----
+
+When redirection is enabled, any ingestion related failures will be
+captured in the failure store if the cluster is able to, along with the
+timestamp that the failure occurred, details about the error
+encountered, and the document that could not be ingested. Since failure
+stores are a kind of Elasticsearch index, we can search the data stream
+for the failures that it has collected. The failures are not shown by
+default as they are stored in different indices than the normal data
+stream data. In order to retrieve the failures, we use the `_search` API
+along with a new bit of index pattern syntax, the `::` selector.
+
+[source,yaml]
+----
+POST logs-test-apache::failures/_search
+----
+
+This index syntax informs the search operation to target the indices in
+its failure store instead of its backing indices. It can be mixed in a
+number of ways with other index patterns to include their failure store
+indices in the search operation:
+
+[source,yaml]
+----
+POST logs-*::failures/_search
+POST logs-*,logs-*::failures/_search
+POST *::failures/_search
+POST _query
+{
+  "query": "FROM my_data_stream*::failures"
+}
+----
+
+{es-pull}126973[#126973]
+
+[discrete]
+[[upgrade_to_lucene_9_12_2]]
+=== Upgrade to lucene 9.12.2
+* Reduce NeighborArray on-heap memory during HNSW graph building
+* Fix IndexSortSortedNumericDocValuesRangeQuery for integer sorting
+* ValueSource.fromDoubleValuesSource(dvs).getSortField() would throw errors when used if the DoubleValuesSource needed scores
+* Disable connectedComponents logic in HNSW graph building.
+
+{es-pull}129555[#129555]
+
 // end::notable-highlights[]