Skip to content

Commit c53f4a3

Browse files
author
elasticsearchmachine
committed
Forward port release notes for v8.16.1
1 parent 2ac267d commit c53f4a3

File tree

2 files changed

+267
-0
lines changed

2 files changed

+267
-0
lines changed

‎docs/reference/migration/migrate_8_16.asciidoc‎

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
[[migrating-8.16]]
2+
== Migrating to 8.16
3+
++++
4+
<titleabbrev>8.16</titleabbrev>
5+
++++
6+
7+
This section discusses the changes that you need to be aware of when migrating
8+
your application to {es} 8.16.
9+
10+
See also <<release-highlights>> and <<es-release-notes>>.
11+
12+
coming::[8.16.0]
13+
14+
15+
[discrete]
16+
[[breaking-changes-8.16]]
17+
=== Breaking changes
18+
19+
The following changes in {es} 8.16 might affect your applications
20+
and prevent them from operating normally.
21+
Before upgrading to 8.16, review these changes and take the described steps
22+
to mitigate the impact.
23+
24+
[discrete]
25+
[[breaking_816_analysis_changes]]
26+
==== Analysis changes
27+
28+
[[set_lenient_to_true_by_default_when_using_updateable_synonyms]]
29+
.Set lenient to true by default when using updateable synonyms
30+
[%collapsible]
31+
====
32+
*Details* +
33+
When a `synonym` or `synonym_graph` token filter is configured with `updateable: true`, the default `lenient`
34+
value will now be `true`.
35+
36+
*Impact* +
37+
`synonym` or `synonym_graph` token filters configured with `updateable: true` will ignore invalid synonyms by
38+
default. This prevents shard initialization errors on invalid synonyms.
39+
====
40+
41+
[discrete]
42+
[[breaking_816_mapping_changes]]
43+
==== Mapping changes
44+
45+
[[jdk_locale_database_change]]
46+
.JDK locale database change
47+
[%collapsible]
48+
====
49+
*Details* +
50+
{es} 8.16 changes the version of the JDK that is included from version 22 to version 23. This changes the locale database that is used by Elasticsearch from the COMPAT database to the CLDR database. This change can cause significant differences to the textual date formats accepted by Elasticsearch, and to calculated week-dates.
51+
52+
If you run {es} 8.16 on JDK version 22 or below, it will use the COMPAT locale database to match the behavior of 8.15. However, starting with {es} 9.0, {es} will use the CLDR database regardless of JDK version it is run on.
53+
54+
*Impact* +
55+
This affects you if you use custom date formats using textual or week-date field specifiers. If you use date fields or calculated week-dates that change between the COMPAT and CLDR databases, then this change will cause Elasticsearch to reject previously valid date fields as invalid data. You might need to modify your ingest or output integration code to account for the differences between these two JDK versions.
56+
57+
Starting in version 8.15.2, Elasticsearch will log deprecation warnings if you are using date format specifiers that might change on upgrading to JDK 23. These warnings are visible in Kibana.
58+
59+
For detailed guidance, refer to <<custom-date-format-locales,Differences in locale information between JDK versions>> and the https://ela.st/jdk-23-locales[Elastic blog].
60+
====
61+
62+
[discrete]
63+
[[breaking_816_es_ql_changes]]
64+
==== ES|QL changes
65+
66+
[[esql_entirely_remove_meta_functions]]
67+
.ESQL: Entirely remove META FUNCTIONS
68+
[%collapsible]
69+
====
70+
*Details* +
71+
Removes an undocumented syntax from ESQL: META FUNCTION. This was never
72+
reliable or really useful. Consult the documentation instead.
73+
74+
*Impact* +
75+
Removes an undocumented syntax from ESQL: META FUNCTION
76+
====
77+
78+
[discrete]
79+
[[breaking_816_rest_api_changes]]
80+
==== REST API changes
81+
82+
[[reworking_rrf_retriever_to_be_evaluated_during_rewrite_phase]]
83+
.Reworking RRF retriever to be evaluated during rewrite phase
84+
[%collapsible]
85+
====
86+
*Details* +
87+
In this release (8.16), we have introduced major changes to the retrievers framework
88+
and how they can be evaluated, focusing mainly on compound retrievers
89+
like `rrf` and `text_similarity_reranker`, which allowed us to support full
90+
composability (i.e. any retriever can be nested under any compound retriever),
91+
as well as supporting additional search features like collapsing, explaining,
92+
aggregations, and highlighting.
93+
94+
To ensure consistency, and given that this rework is not available until 8.16,
95+
`rrf` and `text_similarity_reranker` retriever queries would now
96+
throw an exception in a mixed cluster scenario, where there are nodes
97+
both in current or later (i.e. >= 8.16) and previous ( <= 8.15) versions.
98+
99+
As part of the rework, we have also removed the `_rank` property from
100+
the responses of an `rrf` retriever.
101+
102+
*Impact* +
103+
- Users will not be able to use the `rrf` and `text_similarity_reranker` retrievers in a mixed cluster scenario
104+
with previous releases (i.e. prior to 8.16), and the request will throw an `IllegalArgumentException`.
105+
- `_rank` has now been removed from the output of the `rrf` retrievers so trying to directly parse the field
106+
will throw an exception
107+
====
108+
109+
[[update_data_stream_lifecycle_telemetry_to_track_global_retention]]
110+
.Update data stream lifecycle telemetry to track global retention
111+
[%collapsible]
112+
====
113+
*Details* +
114+
In this release we introduced global retention settings that fulfil the following criteria:
115+
116+
- a data stream managed by the data stream lifecycle,
117+
- a data stream that is not an internal data stream.
118+
119+
As a result, we defined different types of retention:
120+
121+
- **data retention**: the retention configured on data stream level by the data stream user or owner
122+
- **default global retention:** the retention configured by an admin on a cluster level and applied to any
123+
data stream that doesn't have data retention and fulfils the criteria.
124+
- **max global retention:** the retention configured by an admin to guard against having long retention periods.
125+
Any data stream that fulfills the criteria will adhere to the data retention unless it exceeds the max retention,
126+
in which case the max global retention applies.
127+
- **effective retention:** the retention that applies on the data stream that fulfill the criteria at a given moment
128+
in time. It takes into consideration all the retention above and resolves it to the retention that will take effect.
129+
130+
Considering the above changes, having a field named `retention` in the usage API was confusing. For this reason, we
131+
renamed it to `data_retention` and added telemetry about the other configurations too.
132+
133+
*Impact* +
134+
Users that use the field `data_lifecycle.retention` should use the `data_lifecycle.data_retention`
135+
====
136+
137+
138+
[discrete]
139+
[[deprecated-8.16]]
140+
=== Deprecations
141+
142+
The following functionality has been deprecated in {es} 8.16
143+
and will be removed in a future version.
144+
While this won't have an immediate impact on your applications,
145+
we strongly encourage you to take the described steps to update your code
146+
after upgrading to 8.16.
147+
148+
To find out if you are using any deprecated functionality,
149+
enable <<deprecation-logging, deprecation logging>>.
150+
151+
[discrete]
152+
[[deprecations_816_analysis]]
153+
==== Analysis deprecations
154+
155+
[[deprecate_dutch_kp_lovins_stemmer_as_they_are_removed_in_lucene_10]]
156+
.Deprecate dutch_kp and lovins stemmer as they are removed in Lucene 10
157+
[%collapsible]
158+
====
159+
*Details* +
160+
kp, dutch_kp, dutchKp and lovins stemmers are deprecated and will be removed.
161+
162+
*Impact* +
163+
These stemmers will be removed and will be no longer supported.
164+
====
165+
166+
[[deprecate_edge_ngram_side_parameter]]
167+
.deprecate `edge_ngram` side parameter
168+
[%collapsible]
169+
====
170+
*Details* +
171+
edge_ngram will no longer accept the side parameter.
172+
173+
*Impact* +
174+
Users will need to update any usage of edge_ngram token filter that utilizes `side`. If the `back` value was used, they can achieve the same behavior by using the `reverse` token filter.
175+
====
176+
177+
[discrete]
178+
[[deprecations_816_crud]]
179+
==== CRUD deprecations
180+
181+
[[deprecate_dot_prefixed_indices_composable_template_index_patterns]]
182+
.Deprecate dot-prefixed indices and composable template index patterns
183+
[%collapsible]
184+
====
185+
*Details* +
186+
Indices beginning with a dot '.' are reserved for system and internal indices, and should not be used by and end-user. Additionally, composable index templates that contain patterns for dot-prefixed indices should also be avoided, as these patterns are meant for internal use only. In a future Elasticsearch version, creation of these dot-prefixed indices will no longer be allowed.
187+
188+
*Impact* +
189+
Requests performing an action that would create an index beginning with a dot (indexing a document, manual creation, reindex), or creating an index template with index patterns beginning with a dot, will contain a deprecation header warning about dot-prefixed indices in the response.
190+
====
191+
192+
[discrete]
193+
[[deprecations_816_rest_api]]
194+
==== REST API deprecations
195+
196+
[[adding_deprecation_warnings_for_rrf_using_rank_sub_searches]]
197+
.Adding deprecation warnings for rrf using rank and `sub_searches`
198+
[%collapsible]
199+
====
200+
*Details* +
201+
Search API parameter `sub_searches` will no longer be a supported and will be removed in future releases. Similarly, `rrf` can only be used through the specified `retriever` and no longer though the `rank` parameter
202+
203+
*Impact* +
204+
Requests specifying rrf through `rank` and/or `sub_searches` elements will be disallowed in a future version. Users should instead utilize the new `retriever` parameter.
205+
====
206+
207+
[[deprecate_legacy_params_from_range_query]]
208+
.Deprecate legacy params from range query
209+
[%collapsible]
210+
====
211+
*Details* +
212+
Range query will not longer accept `to`, `from`, `include_lower`, and `include_upper` parameters.
213+
214+
*Impact* +
215+
Instead use `gt`, `gte`, `lt` and `lte` parameters.
216+
====
217+

‎docs/reference/release-notes/8.16.1.asciidoc‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
[[release-notes-8.16.1]]
2+
== {es} version 8.16.1
3+
4+
Also see <<breaking-changes-8.16,Breaking changes in 8.16>>.
5+
6+
[[bug-8.16.1]]
7+
[float]
8+
=== Bug fixes
9+
10+
Data streams::
11+
* Add missing header in `put_data_lifecycle` rest-api-spec {es-pull}116292[#116292]
12+
13+
Mapping::
14+
* Ignore conflicting fields during dynamic mapping update {es-pull}114227[#114227] (issue: {es-issue}114228[#114228])
15+
16+
Ranking::
17+
* Propagating nested `inner_hits` to the parent compound retriever {es-pull}116408[#116408] (issue: {es-issue}116397[#116397])
18+
19+
Relevance::
20+
* Fix handling of bulk requests with semantic text fields and delete ops {es-pull}116942[#116942]
21+
22+
Search::
23+
* Fix handling of time exceeded exception in fetch phase {es-pull}116676[#116676]
24+
* Semantic text simple partial update {es-pull}116478[#116478]
25+
* Validate missing shards after the coordinator rewrite {es-pull}116382[#116382]
26+
27+
Vector Search::
28+
* Update Semantic Query To Handle Zero Size Responses {es-pull}116277[#116277] (issue: {es-issue}116083[#116083])
29+
30+
[[enhancement-8.16.1]]
31+
[float]
32+
=== Enhancements
33+
34+
Data streams::
35+
* Apm-data: disable date_detection for all apm data streams {es-pull}116995[#116995]
36+
37+
Ingest Node::
38+
* Add postal_code support to the City and Enterprise databases {es-pull}114193[#114193]
39+
* Add support for registered country fields for maxmind geoip databases {es-pull}114521[#114521]
40+
* Support IPinfo database configurations {es-pull}114548[#114548]
41+
* Support more maxmind fields in the geoip processor {es-pull}114268[#114268]
42+
43+
Relevance::
44+
* Add tracking for query rule types {es-pull}116357[#116357]
45+
46+
Snapshot/Restore::
47+
* Improve message about insecure S3 settings {es-pull}116915[#116915]
48+
* Split searchable snapshot into multiple repo operations {es-pull}116918[#116918]
49+
50+

0 commit comments

Comments
 (0)