@@ -31,13 +31,170 @@ Other versions:
3131
3232endif::[]
3333
34- // The notable-highlights tag marks entries that
35- // should be featured in the Stack Installation and Upgrade Guide:
3634// tag::notable-highlights[]
37- // [discrete]
38- // === Heading
39- //
40- // Description.
35+
36+ [discrete]
37+ [[esql_inlinestats]]
38+ === ESQL: INLINESTATS
39+ This adds the `INLINESTATS` command to ESQL which performs a STATS and
40+ then enriches the results into the output stream. So, this query:
41+
42+ [source,esql]
43+ ----
44+ FROM test
45+ | INLINESTATS m=MAX(a * b) BY b
46+ | WHERE m == a * b
47+ | SORT a DESC, b DESC
48+ | LIMIT 3
49+ ----
50+
51+ Produces output like:
52+
53+ | a | b | m |
54+ | --- | --- | ----- |
55+ | 99 | 999 | 98901 |
56+ | 99 | 998 | 98802 |
57+ | 99 | 997 | 98703 |
58+
59+ {es-pull}109583[#109583]
60+
61+ [discrete]
62+ [[always_allow_rebalancing_by_default]]
63+ === Always allow rebalancing by default
64+ In earlier versions of {es} the `cluster.routing.allocation.allow_rebalance` setting defaults to
65+ `indices_all_active` which blocks all rebalancing moves while the cluster is in `yellow` or `red` health. This was
66+ appropriate for the legacy allocator which might do too many rebalancing moves otherwise. Today's allocator has
67+ better support for rebalancing a cluster that is not in `green` health, and expects to be able to rebalance some
68+ shards away from over-full nodes to avoid allocating shards to undesirable locations in the first place. From
69+ version 8.16 `allow_rebalance` setting defaults to `always` unless the legacy allocator is explicitly enabled.
70+
71+ {es-pull}111015[#111015]
72+
73+ [discrete]
74+ [[add_global_retention_in_data_stream_lifecycle]]
75+ === Add global retention in data stream lifecycle
76+ Data stream lifecycle now supports configuring retention on a cluster level,
77+ namely global retention. Global retention \nallows us to configure two different
78+ retentions:
79+
80+ - `data_streams.lifecycle.retention.default` is applied to all data streams managed
81+ by the data stream lifecycle that do not have retention defined on the data stream level.
82+ - `data_streams.lifecycle.retention.max` is applied to all data streams managed by the
83+ data stream lifecycle and it allows any data stream \ndata to be deleted after the `max_retention` has passed.
84+
85+ {es-pull}111972[#111972]
86+
87+ [discrete]
88+ [[enable_zstandard_compression_for_indices_with_index_codec_set_to_best_compression]]
89+ === Enable ZStandard compression for indices with index.codec set to best_compression
90+ Before DEFLATE compression was used to compress stored fields in indices with index.codec index setting set to
91+ best_compression, with this change ZStandard is used as compression algorithm to stored fields for indices with
92+ index.codec index setting set to best_compression. The usage ZStandard results in less storage usage with a
93+ similar indexing throughput depending on what options are used. Experiments with indexing logs have shown that
94+ ZStandard offers ~12% lower storage usage and a ~14% higher indexing throughput compared to DEFLATE.
95+
96+ {es-pull}112665[#112665]
97+
98+ [discrete]
99+ [[8_x_remove_zstd_feature_flag_for_index_codec_best_compression]]
100+ === [8.x] Remove zstd feature flag for index codec best compression
101+ Backports the following commits to 8.x: - Remove zstd feature flag for
102+ index codec best compression. (#112665)
103+
104+ {es-pull}112857[#112857]
105+
106+ [discrete]
107+ [[esql_introduce_per_agg_filter]]
108+ === ESQL: Introduce per agg filter
109+ Add support for aggregation scoped filters that work dynamically on the
110+ data in each group.
111+
112+ [source,esql]
113+ ----
114+ | STATS success = COUNT(*) WHERE 200 <= code AND code < 300,
115+ redirect = COUNT(*) WHERE 300 <= code AND code < 400,
116+ client_err = COUNT(*) WHERE 400 <= code AND code < 500,
117+ server_err = COUNT(*) WHERE 500 <= code AND code < 600,
118+ total_count = COUNT(*)
119+ ----
120+
121+ Implementation wise, the base AggregateFunction has been extended to
122+ allow a filter to be passed on. This is required to incorporate the
123+ filter as part of the aggregate equality/identity which would fail with
124+ the filter as an external component.
125+ As part of the process, the serialization for the existing aggregations
126+ had to be fixed so AggregateFunction implementations so that it
127+ delegates to their parent first.
128+
129+ {es-pull}113735[#113735]
130+
41131// end::notable-highlights[]
42132
43133
134+ [discrete]
135+ [[esql_multi_value_fields_supported_in_geospatial_predicates]]
136+ === ESQL: Multi-value fields supported in Geospatial predicates
137+ Supporting multi-value fields in `WHERE` predicates is a challenge due to not knowing whether `ALL` or `ANY`
138+ of the values in the field should pass the predicate.
139+ For example, should the field `age:[10,30]` pass the predicate `WHERE age>20` or not?
140+ This ambiguity does not exist with the spatial predicates
141+ `ST_INTERSECTS` and `ST_DISJOINT`, because the choice between `ANY` or `ALL`
142+ is implied by the predicate itself.
143+ Consider a predicate checking a field named `location` against a test geometry named `shape`:
144+
145+ * `ST_INTERSECTS(field, shape)` - true if `ANY` value can intersect the shape
146+ * `ST_DISJOINT(field, shape)` - true only if `ALL` values are disjoint from the shape
147+
148+ This works even if the shape argument is itself a complex or compound geometry.
149+
150+ Similar logic exists for `ST_CONTAINS` and `ST_WITHIN` predicates, but these are not as easily solved
151+ with `ANY` or `ALL`, because a collection of geometries contains another collection if each of the contained
152+ geometries is within at least one of the containing geometries. Evaluating this requires that the multi-value
153+ field is first combined into a single geometry before performing the predicate check.
154+
155+ * `ST_CONTAINS(field, shape)` - true if the combined geometry contains the shape
156+ * `ST_WITHIN(field, shape)` - true if the combined geometry is within the shape
157+
158+ {es-pull}112063[#112063]
159+
160+ [discrete]
161+ [[enhance_sort_push_down_to_lucene_to_cover_references_to_fields_st_distance_function]]
162+ === Enhance SORT push-down to Lucene to cover references to fields and ST_DISTANCE function
163+ The most used and likely most valuable geospatial search query in Elasticsearch is the sorted proximity search,
164+ finding items within a certain distance of a point of interest and sorting the results by distance.
165+ This has been possible in ES|QL since 8.15.0, but the sorting was done in-memory, not pushed down to Lucene.
166+ Now the sorting is pushed down to Lucene, which results in a significant performance improvement.
167+
168+ Queries that perform both filtering and sorting on distance are supported. For example:
169+
170+ [source,esql]
171+ ----
172+ FROM test
173+ | EVAL distance = ST_DISTANCE(location, TO_GEOPOINT("POINT(37.7749, -122.4194)"))
174+ | WHERE distance < 1000000
175+ | SORT distance ASC, name DESC
176+ | LIMIT 10
177+ ----
178+
179+ In addition, the support for sorting on EVAL expressions has been extended to cover references to fields:
180+
181+ [source,esql]
182+ ----
183+ FROM test
184+ | EVAL ref = field
185+ | SORT ref ASC
186+ | LIMIT 10
187+ ----
188+
189+ {es-pull}112938[#112938]
190+
191+ [discrete]
192+ [[cross_cluster_search_telemetry]]
193+ === Cross-cluster search telemetry
194+ The cross-cluster search telemetry is collected when cross-cluster searches
195+ are performed, and is returned as "ccs" field in `_cluster/stats` output.
196+ It also add a new parameter `include_remotes=true` to the `_cluster/stats` API
197+ which will collect data from connected remote clusters.
198+
199+ {es-pull}113825[#113825]
200+
0 commit comments