Skip to content

Commit b0050e5

Browse files
committed
Revert "Delete docs/reference/release-notes/highlights.asciidoc"
This reverts commit fc64594.
1 parent fc64594 commit b0050e5

File tree

1 file changed

+134
-0
lines changed

1 file changed

+134
-0
lines changed
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
[[release-highlights]]
2+
== What's new in {minor-version}
3+
4+
coming::[{minor-version}]
5+
6+
Here are the highlights of what's new and improved in {es} {minor-version}!
7+
ifeval::["{release-state}"!="unreleased"]
8+
For detailed information about this release, see the <<es-release-notes>> and
9+
<<breaking-changes>>.
10+
11+
// Add previous release to the list
12+
Other versions:
13+
14+
{ref-bare}/8.18/release-highlights.html[8.18]
15+
| {ref-bare}/8.17/release-highlights.html[8.17]
16+
| {ref-bare}/8.16/release-highlights.html[8.16]
17+
| {ref-bare}/8.15/release-highlights.html[8.15]
18+
| {ref-bare}/8.14/release-highlights.html[8.14]
19+
| {ref-bare}/8.13/release-highlights.html[8.13]
20+
| {ref-bare}/8.12/release-highlights.html[8.12]
21+
| {ref-bare}/8.11/release-highlights.html[8.11]
22+
| {ref-bare}/8.10/release-highlights.html[8.10]
23+
| {ref-bare}/8.9/release-highlights.html[8.9]
24+
| {ref-bare}/8.8/release-highlights.html[8.8]
25+
| {ref-bare}/8.7/release-highlights.html[8.7]
26+
| {ref-bare}/8.6/release-highlights.html[8.6]
27+
| {ref-bare}/8.5/release-highlights.html[8.5]
28+
| {ref-bare}/8.4/release-highlights.html[8.4]
29+
| {ref-bare}/8.3/release-highlights.html[8.3]
30+
| {ref-bare}/8.2/release-highlights.html[8.2]
31+
| {ref-bare}/8.1/release-highlights.html[8.1]
32+
| {ref-bare}/8.0/release-highlights.html[8.0]
33+
34+
endif::[]
35+
36+
// tag::notable-highlights[]
37+
38+
[discrete]
39+
[[upgrade_repository_s3_to_aws_sdk_v2]]
40+
=== Upgrade `repository-s3` to AWS SDK v2
41+
In earlier versions of {es} the `repository-s3` plugin was based on the AWS SDK v1. AWS will withdraw support for this SDK before the end of the life of {es} {minor-version} so we have migrated this plugin to the newer AWS SDK v2.
42+
The two SDKs are not quite compatible, so please check the breaking changes documentation and test the new version thoroughly before upgrading any production workloads.
43+
44+
{es-pull}126843[#126843]
45+
46+
[discrete]
47+
[[add_ability_to_redirect_ingestion_failures_on_data_streams_to_failure_store]]
48+
=== Add ability to redirect ingestion failures on data streams to a failure store
49+
Documents that encountered ingest pipeline failures or mapping conflicts
50+
would previously be returned to the client as errors in the bulk and
51+
index operations. Many client applications are not equipped to respond
52+
to these failures. This leads to the failed documents often being
53+
dropped by the client which cannot hold the broken documents
54+
indefinitely. In many end user workloads, these failed documents
55+
represent events that could be critical signals for observability or
56+
security use cases.
57+
58+
To help mitigate this problem, data streams can now maintain a "failure
59+
store" which is used to accept and hold documents that fail to be
60+
ingested due to preventable configuration errors. The data stream's
61+
failure store operates like a separate set of backing indices with their
62+
own mappings and access patterns that allow Elasticsearch to accept
63+
documents that would otherwise be rejected due to unhandled ingest
64+
pipeline exceptions or mapping conflicts.
65+
66+
Users can enable redirection of ingest failures to the failure store on
67+
new data streams by specifying it in the new `data_stream_options` field
68+
inside of a component or index template:
69+
70+
[source,yaml]
71+
----
72+
PUT _index_template/my-template
73+
{
74+
"index_patterns": ["logs-test-*"],
75+
"data_stream": {},
76+
"template": {
77+
"data_stream_options": {
78+
"failure_store": {
79+
"enabled": true
80+
}
81+
}
82+
}
83+
}'
84+
----
85+
86+
Existing data streams can be configured with the new data stream
87+
`_options` endpoint:
88+
89+
[source,yaml]
90+
----
91+
PUT _data_stream/logs-test-apache/_options
92+
{
93+
"failure_store": {
94+
"enabled": "true"
95+
}
96+
}
97+
----
98+
99+
When redirection is enabled, any ingestion related failures will be
100+
captured in the failure store if the cluster is able to, along with the
101+
timestamp that the failure occurred, details about the error
102+
encountered, and the document that could not be ingested. Since failure
103+
stores are a kind of Elasticsearch index, we can search the data stream
104+
for the failures that it has collected. The failures are not shown by
105+
default as they are stored in different indices than the normal data
106+
stream data. In order to retrieve the failures, we use the `_search` API
107+
along with a new bit of index pattern syntax, the `::` selector.
108+
109+
[source,yaml]
110+
----
111+
POST logs-test-apache::failures/_search
112+
----
113+
114+
This index syntax informs the search operation to target the indices in
115+
its failure store instead of its backing indices. It can be mixed in a
116+
number of ways with other index patterns to include their failure store
117+
indices in the search operation:
118+
119+
[source,yaml]
120+
----
121+
POST logs-*::failures/_search
122+
POST logs-*,logs-*::failures/_search
123+
POST *::failures/_search
124+
POST _query
125+
{
126+
"query": "FROM my_data_stream*::failures"
127+
}
128+
----
129+
130+
{es-pull}126973[#126973]
131+
132+
// end::notable-highlights[]
133+
134+

0 commit comments

Comments
 (0)