Forward port release notes for v8.17.0

Author: elasticsearchmachine

docs/reference/migration/migrate_8_17.asciidoc

@@ -18,3 +18,54 @@ coming::[8.17.0]
 
 There are no breaking changes in {es} 8.17.
 
+
+[discrete]
+[[deprecated-8.17]]
+=== Deprecations
+
+The following functionality has been deprecated in {es} 8.17
+and will be removed in a future version.
+While this won't have an immediate impact on your applications,
+we strongly encourage you to take the described steps to update your code
+after upgrading to 8.17.
+
+To find out if you are using any deprecated functionality,
+enable <<deprecation-logging, deprecation logging>>.
+
+[discrete]
+[[deprecations_817_mapping]]
+==== Mapping deprecations
+
+[[deprecate_source_mode_in_mappings]]
+.Deprecate `_source.mode` in mappings
+[%collapsible]
+====
+*Details* +
+Configuring `_source.mode` in mappings is deprecated and will be removed in future versions. Use `index.mapping.source.mode` index setting instead.
+
+*Impact* +
+Use `index.mapping.source.mode` index setting instead
+====
+
+[discrete]
+[[deprecations_817_rest_api]]
+==== REST API deprecations
+
+[[format_of_non_detailed_error_responses_changing_in_v9]]
+.The format of non-detailed error responses is changing in v9
+[%collapsible]
+====
+*Details* +
+When an error occurs when processing a request, Elasticsearch returns information on that error in the REST response.
+If `http:detailed_errors.enabled: false` is specified in node settings with the v8 REST API and below,
+the format of this response changes significantly.
+Starting with the v9 REST API, the JSON structure of responses with errors when the `http.detailed_errors.enabled: false` option is set
+will be the same as when detailed errors are enabled (which is the default).
+To keep using the existing format for non-detailed error responses, use the v8 REST API.
+
+*Impact* +
+If you have set `http.detailed_errors.enabled: false` (the default is `true`)
+the structure of JSON when any exceptions occur will change with the v9 REST API.
+To keep using the existing format, use the v8 REST API.
+====
+

docs/reference/release-notes/8.17.0.asciidoc

@@ -1,8 +1,204 @@
 [[release-notes-8.17.0]]
 == {es} version 8.17.0
 
-coming[8.17.0]
-
 Also see <<breaking-changes-8.17,Breaking changes in 8.17>>.
 
+[[license-8.17.0]]
+[float]
+=== License changes
+
+[float] 
+==== Change to synthetic `_source` licensing
+
+Starting with this release, the <<synthetic-source,synthetic `_source`>> feature is available exclusively with the Enterprise subscription. Synthetic `_source` is used in logs data streams (`logsdb` index mode), time series data streams (TSDS, using `time_series` index mode), application performance monitoring (APM), and Universal Profiling. 
+
+If you are using these capabilities and are not on an Enterprise license, the change will result in increased storage requirements for new data, as the synthetic `_source` setting will be ignored. Existing indices that used synthetic `_source` will remain seamlessly accessible. 
+
+Refer to the subscription page for https://www.elastic.co/subscriptions/cloud[Elastic Cloud] and {subscriptions}[Elastic Stack/self-managed] for the breakdown of available features and their associated subscription tiers. For further details and subscription options, contact your Elastic sales representative or https://www.elastic.co/contact[contact us]. 
+
+[[bug-8.17.0]]
+[float]
+=== Bug fixes
+
+Analysis::
+* Adjust analyze limit exception to be a `bad_request` {es-pull}116325[#116325]
+
+CCS::
+* Fix long metric deserialize & add - auto-resize needs to be set manually {es-pull}117105[#117105] (issue: {es-issue}116914[#116914])
+
+CRUD::
+* Standardize error code when bulk body is invalid {es-pull}114869[#114869]
+
+Data streams::
+* Acquire stats searcher for data stream stats {es-pull}117953[#117953]
+
+EQL::
+* Don't use a `BytesStreamOutput` to copy keys in `BytesRefBlockHash` {es-pull}114819[#114819] (issue: {es-issue}114599[#114599])
+
+ES|QL::
+* Added stricter range type checks and runtime warnings for ENRICH {es-pull}115091[#115091] (issues: {es-issue}107357[#107357], {es-issue}116799[#116799])
+* Don't return TEXT type for functions that take TEXT {es-pull}114334[#114334] (issues: {es-issue}111537[#111537], {es-issue}114333[#114333])
+* ESQL: Fix sorts containing `_source` {es-pull}116980[#116980] (issue: {es-issue}116659[#116659])
+* ES|QL: Fix stats by constant expression {es-pull}114899[#114899]
+* Fix BWC for ES|QL cluster request {es-pull}117865[#117865]
+* Fix CCS exchange when multi cluster aliases point to same cluster {es-pull}117297[#117297]
+* Fix COUNT filter pushdown {es-pull}117503[#117503] (issue: {es-issue}115522[#115522])
+* Fix NPE in `EnrichLookupService` on mixed clusters with <8.14 versions {es-pull}116583[#116583] (issues: {es-issue}116529[#116529], {es-issue}116544[#116544])
+* Fix stats by constant expresson with alias {es-pull}117551[#117551]
+* Fix validation of SORT by aggregate functions {es-pull}117316[#117316]
+* Fixing remote ENRICH by pushing the Enrich inside `FragmentExec` {es-pull}114665[#114665] (issue: {es-issue}105095[#105095])
+* Ignore cancellation exceptions {es-pull}117657[#117657]
+* Limit size of `Literal#toString` {es-pull}117842[#117842]
+* Opt into extra data stream resolution {es-pull}118378[#118378]
+* Use `SearchStats` instead of field.isAggregatable in data node planning {es-pull}115744[#115744] (issue: {es-issue}115737[#115737])
+* [ESQL] Fix Binary Comparisons on Date Nanos {es-pull}116346[#116346]
+* [ES|QL] To_DatePeriod and To_TimeDuration return better error messages on `union_type` fields {es-pull}114934[#114934]
+
+Infra/CLI::
+* Fix NPE on plugin sync {es-pull}115640[#115640] (issue: {es-issue}114818[#114818])
+
+Ingest Node::
+* Fix enrich cache size setting name {es-pull}117575[#117575]
+* Fix log message format bugs {es-pull}118354[#118354]
+* Fix reconstituting version string from components {es-pull}117213[#117213] (issue: {es-issue}116950[#116950])
+* Reducing error-level stack trace logging for normal events in `GeoIpDownloader` {es-pull}114924[#114924]
+
+License::
+* Distinguish `LicensedFeature` by family field {es-pull}116809[#116809]
+
+Logs::
+* Prohibit changes to index mode, source, and sort settings during resize {es-pull}115812[#115812]
+
+Machine Learning::
+* Fix deberta tokenizer bug caused by bug in normalizer {es-pull}117189[#117189]
+* Fix for Deberta tokenizer when input sequence exceeds 512 tokens {es-pull}117595[#117595]
+* Hides `hugging_face_elser` service from the `GET _inference/_services API` {es-pull}116664[#116664] (issue: {es-issue}116644[#116644])
+* Mitigate IOSession timeouts {es-pull}115414[#115414] (issues: {es-issue}114385[#114385], {es-issue}114327[#114327], {es-issue}114105[#114105], {es-issue}114232[#114232])
+* Propagate scoring function through random sampler {es-pull}116957[#116957] (issue: {es-issue}110134[#110134])
+* Wait for the worker service to shutdown before closing task processor {es-pull}117920[#117920] (issue: {es-issue}117563[#117563])
+
+Mapping::
+* Address mapping and compute engine runtime field issues {es-pull}117792[#117792] (issue: {es-issue}117644[#117644])
+* Always Emit Inference ID in Semantic Text Mapping {es-pull}117294[#117294]
+* Fix concurrency issue with `ReinitializingSourceProvider` {es-pull}118370[#118370] (issue: {es-issue}118238[#118238])
+* Fix false positive date detection with trailing dot {es-pull}116953[#116953] (issue: {es-issue}116946[#116946])
+* Parse the contents of dynamic objects for [subobjects:false] {es-pull}117762[#117762] (issue: {es-issue}117544[#117544])
+
+Network::
+* Use underlying `ByteBuf` `refCount` for `ReleasableBytesReference` {es-pull}116211[#116211]
+
+Ranking::
+* Fix for propagating filters from compound to inner retrievers {es-pull}117914[#117914]
+
+Search::
+* Add missing `async_search` query parameters to rest-api-spec {es-pull}117312[#117312]
+* Don't skip shards in coord rewrite if timestamp is an alias {es-pull}117271[#117271]
+* Fields caps does not honour ignore_unavailable {es-pull}116021[#116021] (issue: {es-issue}107767[#107767])
+* _validate does not honour ignore_unavailable {es-pull}116656[#116656] (issue: {es-issue}116594[#116594])
+
+Vector Search::
+* Correct bit * byte and bit * float script comparisons {es-pull}117404[#117404]
+
+Watcher::
+* Watch Next Run Interval Resets On Shard Move or Node Restart {es-pull}115102[#115102] (issue: {es-issue}111433[#111433])
+
+[[deprecation-8.17.0]]
+[float]
+=== Deprecations
+
+Infra/REST API::
+* Add a basic deprecation warning that the JSON format for non-detailed error responses is changing in v9 {es-pull}114739[#114739] (issue: {es-issue}89387[#89387])
+
+Mapping::
+* Deprecate `_source.mode` in mappings {es-pull}116689[#116689]
+
+[[enhancement-8.17.0]]
+[float]
+=== Enhancements
+
+Authorization::
+* Add a `monitor_stats` privilege and allow that privilege for remote cluster privileges {es-pull}114964[#114964]
+
+Data streams::
+* Adding a deprecation info API warning for data streams with old indices {es-pull}116447[#116447]
+
+ES|QL::
+* Add ES|QL `bit_length` function {es-pull}115792[#115792]
+* ESQL: Honor skip_unavailable setting for nonmatching indices errors at planning time {es-pull}116348[#116348] (issue: {es-issue}114531[#114531])
+* ESQL: Remove parent from `FieldAttribute` {es-pull}112881[#112881]
+* ESQL: extract common filter from aggs {es-pull}115678[#115678]
+* ESQL: optimise aggregations filtered by false/null into evals {es-pull}115858[#115858]
+* ES|QL CCS uses `skip_unavailable` setting for handling disconnected remote clusters {es-pull}115266[#115266] (issue: {es-issue}114531[#114531])
+* ES|QL: add metrics for functions {es-pull}114620[#114620]
+* Esql Enable Date Nanos (tech preview) {es-pull}117080[#117080]
+* [ES|QL] Implicit casting string literal to intervals {es-pull}115814[#115814] (issue: {es-issue}115352[#115352])
+
+Indices APIs::
+* Ensure class resource stream is closed in `ResourceUtils` {es-pull}116437[#116437]
+
+Inference::
+* Add version prefix to Inference Service API path {es-pull}117366[#117366]
+* Update sparse text embeddings API route for Inference Service {es-pull}118368[#118368]
+
+Infra/Core::
+* Support for unsigned 64 bit numbers in Cpu stats {es-pull}114681[#114681] (issue: {es-issue}112274[#112274])
+
+Ingest Node::
+* Adding support for additional mapping to simulate ingest API {es-pull}114742[#114742]
+* Adding support for simulate ingest mapping adddition for indices with mappings that do not come from templates {es-pull}115359[#115359]
+
+Logs::
+* Add logsdb telemetry {es-pull}115994[#115994]
+* Add num docs and size to logsdb telemetry {es-pull}116128[#116128]
+* Feature: re-structure document ID generation favoring _id inverted index compression {es-pull}104683[#104683]
+
+Machine Learning::
+* Add special case for elastic reranker in inference API {es-pull}116962[#116962]
+* Adding inference endpoint validation for `AzureAiStudioService` {es-pull}113713[#113713]
+* Adds support for `input_type` field to Vertex inference service {es-pull}116431[#116431]
+* Enable built-in Inference Endpoints and default for Semantic Text {es-pull}116931[#116931]
+* Increase default `queue_capacity` to 10_000 and decrease max `queue_capacity` to 100_000 {es-pull}115041[#115041]
+* [Inference API] Add API to get configuration of inference services {es-pull}114862[#114862]
+* [Inference API] Improve chunked results error message {es-pull}115807[#115807]
+
+Recovery::
+* Attempt to clean up index before remote transfer {es-pull}115142[#115142] (issue: {es-issue}104473[#104473])
+
+Relevance::
+* Add query rules retriever {es-pull}114855[#114855]
+
+Search::
+* Add Search Phase APM metrics {es-pull}113194[#113194]
+* Add `docvalue_fields` Support for `dense_vector` Fields {es-pull}114484[#114484] (issue: {es-issue}108470[#108470])
+* Add initial support for `semantic_text` field type {es-pull}113920[#113920]
+* Adds access to flags no_sub_matches and no_overlapping_matches to hyphenation-decompounder-tokenfilter {es-pull}115459[#115459] (issue: {es-issue}97849[#97849])
+* Better sizing `BytesRef` for Strings in Queries {es-pull}115655[#115655]
+* Enable `_tier` based coordinator rewrites for all indices (not just mounted indices) {es-pull}115797[#115797]
+
+Vector Search::
+* Add support for bitwise inner-product in painless {es-pull}116082[#116082]
+* Improve halfbyte transposition performance, marginally improving bbq performance {es-pull}117350[#117350]
+
+[[feature-8.17.0]]
+[float]
+=== New features
+
+Data streams::
+* Add default ILM policies and switch to ILM for apm-data plugin {es-pull}115687[#115687]
+
+ES|QL::
+* Add support for `BYTE_LENGTH` scalar function {es-pull}116591[#116591]
+* Esql/lookup join grammar {es-pull}116515[#116515]
+* Remove snapshot build restriction for match and qstr functions {es-pull}114482[#114482]
+
+Search::
+* ESQL - Add match operator (:) {es-pull}116819[#116819]
+
+[[upgrade-8.17.0]]
+[float]
+=== Upgrades
+
+Security::
+* Upgrade Bouncy Castle FIPS dependencies {es-pull}112989[#112989]
+