[9.1](backport #47073) [docs] Restructure release notes (#47098)

* [docs] Restructure release notes (#47073)

* restructure release notes in prep for automation

* add release notes for 9.0.8 and 9.1.5

* add config.changelog.yaml

* add missing breaking changes

(cherry picked from commit b4b52fb)

# Conflicts:
#	docs/release-notes/index.md

* resolve conflicts

---------

Co-authored-by: Colleen McGinnis <[email protected]>

Author: mergify[bot]

config.changelog.yaml

@@ -0,0 +1,4 @@
+owner: elastic
+repo: beats
+rendered_changelog_destination: docs/release-notes/_snippets
+file_type: markdown

docs/docset.yml

@@ -5,6 +5,7 @@ cross_links:
   - docs-content
   - ecs
   - ecs-logging
+  - elastic-agent
   - elasticsearch
   - integration-docs
   - logstash

docs/release-notes/_snippets/9.0.0/breaking-changes.md

@@ -0,0 +1,66 @@
+## 9.0.0 [beats-9.0.0-breaking-changes]
+
+% Description and impact of the breaking change.
+% For more information, check [PR #](PR link).
+
+::::{dropdown} Set default Kafka version to 2.1.0 in Kafka output and Filebeat.
+For more information, check [#41662]({{beats-pull}}41662).
+::::
+
+::::{dropdown} Replaced default Ubuntu-based images with UBI-minimal-based ones.
+For more information, check  [#42150]({{beats-pull}}42150).
+::::
+
+::::{dropdown} Removed support for a single "-" to precede multi-letter command line arguments.  Use "--" instead.
+For more information, check [#42117]({{beats-issue}}42117) [#42209]({{beats-pull}}42209).
+::::
+
+::::{dropdown} Filebeat fails to start if there is any input with a duplicated ID. It logs the duplicated IDs and the offending inputs configurations.
+For more information, check [#41731]({{beats-pull}}41731).
+::::
+
+::::{dropdown} Filestream inputs with duplicated IDs will fail to start. An error is logged showing the ID and the full input configuration.
+For more information, check [#41938]({{beats-issue}}41938) [#41954]({{beats-pull}}41954).
+::::
+
+::::{dropdown} Filestream inputs can define "allow_deprecated_id_duplication: true" to run keep the previous behaviour of running inputs with duplicated IDs.
+For more information, check [#41938]({{beats-issue}}41938) [#41954]({{beats-pull}}41954).
+::::
+
+::::{dropdown} Filestream inputs now starts ingesting files only if they are 1024 bytes or larger because the default file identity has changed from native to fingerprint.
+
+At startup Filebeat automatically updates the state from known, active files (i.e: files that are still present on the disk and have not changed path since Filebeat was stopped) to use the new file identity. If Filebeat cannot migrate the state to the new file identity, the file will be re-ingested. To preserve the behaviour from 8.x, set `file_identity.native: ~` and `prospector.scanner.fingerprint.enabled: false`.
+
+Refer to the file identity documentation for more details. You can also check [#40197]({{beats-issue}}40197) [#41762]({{beats-pull}}41762).
+::::
+
+::::{dropdown} Filebeat fails to start when its configuration contains usage of the deprecated log or container inputs. However, they can still be used when "allow_deprecated_use: true" is set in their configuration.
+For more information, check [#42295]({{beats-pull}}42295).
+::::
+
+::::{dropdown} Upgrade osquery version to 5.13.1.
+For more information, check [#40849]({{beats-pull}}40849).
+::::
+
+::::{dropdown} Use base-16 for reporting serial_number value in TLS fields in line with the ECS recommendation.
+For more information, check [#41542]({{beats-pull}}41542).
+::::
+
+::::{dropdown} Default to use raw API and delete older XML implementation.
+For more information, check [#42275]({{beats-pull}}42275).
+::::
+
+::::{dropdown} The Beats logger and file output rotate files when necessary. The beat now forces a file rotation when unexpectedly writing to a file through a symbolic link.
+::::
+
+::::{dropdown} Remove kibana.settings metricset since the API was removed in 8.0 in Metricbeat.
+For more information, check [#30592]({{beats-issue}}30592). [#42937]({{beats-pull}}42937).
+::::
+
+::::{dropdown} Removed support for the Enterprise Search module in Metricbeat.
+For more information, check [#42915]({{beats-pull}}42915).
+::::
+
+::::{dropdown} Allow faccessat(2) in seccomp.
+For more information, check [#43322]({{beats-pull}}43322).
+::::

docs/release-notes/_snippets/9.0.0/deprecations.md

@@ -0,0 +1,3 @@
+## 9.0.0 [elastic-agent-9.0.0-deprecations]
+
+_No deprecations._

docs/release-notes/_snippets/9.0.0/index.md

@@ -0,0 +1,62 @@
+## 9.0.0 [beats-9.0.0-release-notes]
+
+### Features and enhancements [beats-9.0.0-features-enhancements]
+
+* Improves logging in system/socket in Auditbeat. [#41571]({{beats-pull}}41571)
+* Adds out of the box support for Amazon EventBridge notifications over SQS to S3 input in Filebeat. [#40006]({{beats-pull}}40006)
+* Update CEL mito extensions to v1.16.0 in Filebeat. [#41727]({{beats-pull}}41727)
+* Filebeat's registry is now added to the Elastic-Agent diagnostics bundle. [#33238]({{beats-issue}}33238) and [#41795]({{beats-pull}}41795)
+* Adds `unifiedlogs` input for MacOS in Filebeat. [#41791]({{beats-pull}}41791)
+* Adds evaluation state dump debugging option to CEL input in Filebeat. [#41335]({{beats-pull}}41335)
+* Rate limiting operability improvements in the Okta provider of the Entity Analytics input in Filebeat. [#40106]({{beats-issue}}40106) and [#41977]({{beats-pull}}41977)
+* Rate limiting fault tolerance improvements in the Okta provider of the Entity Analytics input in Filebeat. [#40106]({{beats-issue}}40106) [#42094]({{beats-pull}}42094)
+* Introduces ignore older and start timestamp filters for AWS S3 input in Filebeat. [#41804]({{beats-pull}}41804)
+* Journald input now can report its status to Elastic-Agent in Filebeat. [#39791]({{beats-issue}}39791) and [#42462]({{beats-pull}}42462)
+* Publish events progressively in the Okta provider of the Entity Analytics input in Filebeat. [#40106]({{beats-issue}}40106) and [#42567]({{beats-pull}}42567)
+* Journald `include_matches.match` now accepts `+` to represent a logical disjunction (OR) in Filebeat. [#40185]({{beats-issue}}40185) and #[42517]({{beats-pull}}42517)
+* The journald input is now generally available in Filebeat. [#42107]({{beats-pull}}42107)
+* Adds support for RFC7231 methods to HTTP monitors in Heartbeat. [#41975]({{beats-pull}}41975)
+* Adds `use_kubeadm` config option in kubernetes module in order to toggle kubeadm-config API requests in Metricbeat. [#40086]({{beats-pull}}40086)
+* Preserve queries for debugging when `merge_results: true` in SQL module in Metricbeat. [#42271]({{beats-pull}}42271)
+* Collect more fields from ES node/stats metrics and only those that are necessary in Metricbeat. [#42421]({{beats-pull}}42421)
+* Adds benchmark module in Metricbeat. [#41801]({{beats-pull}}41801)
+* Increase maximum query timeout to 24 hours in Osquerybeat. [42356]({{beats-pull}}42356)
+* Properly set events `UserData` when experimental API is used in Winlogbeat. [#41525]({{beats-pull}}41525)
+* Include XML is respected for experimental API in Winlogbeat. [#41525]({{beats-pull}}41525)
+* Forwarded events use renderedtext info for experimental API in Winlogbeat. [#41525]({{beats-pull}}41525)
+* Language setting is respected for experimental API in Winlogbeat. [#41525]({{beats-pull}}41525)
+* Language setting also added to decode XML wineventlog processor in Winlogbeat. [#41525]({{beats-pull}}41525)
+* Format embedded messages in the experimental API in Winlogbeat. [#41525]({{beats-pull}}41525)
+* Make the experimental API GA and rename it to winlogbeat-raw in Winlogbeat. [#39580]({{beats-issue}}39580) and [#41770]({{beats-pull}}41770)
+* Removes 22 clause limitation in Winlogbeat. [#35047]({{beats-issue}}35047) and [#42187]({{beats-pull}}42187)
+* Adds handling for recoverable publisher disabled errorsin Winlogbeat. [#35316]({{beats-issue}}35316) and [#42187]({{beats-pull}}42187)
+* Removes Functionbeat binaries from CI pipelines. [#40745]({{beats-issue}}40745) and [#41506]({{beats-pull}}41506)
+* Update Go version to 1.24.0. [#42705]({{beats-pull}}42705)
+* Add `etw` input fallback to attach an already existing session in Filebeat. [#42847]({{beats-pull}}42847)
+* Update CEL mito extensions to v1.17.0 in Filebeat. [#42851]({{beats-pull}}42851)
+* Winlog input  in Filebeat cam now report its status to Elastic Agent. [#43089]({{beats-pull}}43089)
+* Add configuration option to limit HTTP Endpoint body size in Filebeat. [#43171]({{beats-pull}}43171)
+* Add a new `match_by_parent_instance` option to `perfmon` module in Metricbeat. [#43002]({{beats-pull}}43002)
+* Add a warning log to `metricbeat.vsphere` in Metricbeat in case vSphere connection has been configured as insecure. [#43104]({{beats-pull}}43104)
+
+### Fixes [beats-9.0.0-fixes]
+
+* hasher: Add a cached hasher for upcoming backend in Auditbeat. [#41952]({{beats-pull}}41952)
+* Split common tty definitions in Auditbeat. [#42004]({{beats-pull}}42004)
+* Redact authorization headers in HTTPJSON debug logs in Filebeat. [#41920]({{beats-pull}}41920)
+* Further rate limiting fix in the Okta provider of the Entity Analytics input in Filebeat. [#40106]({{beats-issue}}40106) and [#41977]({{beats-pull}}41977)
+* The `_id` generation process for S3 events has been updated to incorporate the LastModified field. This enhancement ensures that the `_id` is unique in Filebeat. [#42078]({{beats-pull}}42078)
+* Fixes truncation of bodies in request tracing by limiting bodies to 10% of the maximum file size in Filebeat. [#42327]({{beats-pull}}42327)
+* [Journald] Fixes handling of `journalctl` restart. A known symptom was broken multiline messages when there was a restart of journalctl while aggregating the lines in Filebeat. [#41331]({{beats-issue}}41331) and [#42595]({{beats-pull}}42595)
+* Fixwa bug where Metricbeat unintentionally triggers Windows ASR in Metricbeat. [#42177]({{beats-pull}}42177)
+* Removes `hostname` field from ZooKeeper's `mntr` data stream in Metricbeat. [41887]({{beats-pull}}41887)
+* Properly marshal nested structs in ECS fields, fixing issues with mixed cases in field names in Packetbeat. [42116]({{beats-pull}}42116)
+* Fixed race conditions in the global ratelimit processor that could drop events or apply rate limiting incorrectly in Filebeat. [42966]({{beats-pull}}42966)
+* Prevent computer details being returned for user queries by Activedirectory Entity Analytics provider in Filebeat. [#11818]({{beats-issue}}11818) and [#42796]({{beats-pull}}42796)
+* Handle unexpected EOF error in aws-s3 input and enforce retrying using download failed error in Filebeat. [#42420]({{beats-pull}}42420)
+* Prevent azureblobstorage input from logging key details during blob fetch operations in Filebeat. [#43169]({{beats-pull}}43169)
+* Add AWS OwningAccount support for cross account monitoring in Metricbeat. [#40570]({{beats-issue}}40570) and [#40691]({{beats-pull}}40691)
+* Fix logging argument number mismatch in Metricbeat(Redis). [#43072]({{beats-pull}}43072)
+* Reset EventLog if error EOF is encountered in Winlogbeat. [#42826]({{beats-pull}}42826)
+* Implement backoff on error retrial in Winlogbeat. [#42826]({{beats-pull}}42826)
+* Fix boolean key in security pipelines and sync pipelines with integration in Winlogbeat. [#43027]({{beats-pull}}43027)

docs/release-notes/_snippets/9.0.1/breaking-changes.md

@@ -0,0 +1,3 @@
+## 9.0.1 [beats-9.0.1-breaking-changes]
+
+_No breaking changes._

docs/release-notes/_snippets/9.0.1/deprecations.md

@@ -0,0 +1,3 @@
+## 9.0.1 [elastic-agent-9.0.1-deprecations]
+
+_No deprecations._

docs/release-notes/_snippets/9.0.1/index.md

@@ -0,0 +1,17 @@
+## 9.0.1 [beats-9.0.1-release-notes]
+
+### Features and enhancements [beats-9.0.1-features-enhancements]
+
+* For all Beats: Publish `cloud.availability_zone` by `add_cloud_metadata` processor in Azure environments. [#42601]({{beats-issue}}42601) [#43618]({{beats-pull}}43618)
+* Add pagination batch size support to Entity Analytics input's Okta provider in Filebeat. [#43655]({{beats-pull}}43655)
+* Update CEL mito extensions version to v1.19.0 in Filebeat. [#44098]({{beats-pull}}44098)
+* Upgrade node version to latest LTS v18.20.7 in Heartbeat. [#43511]({{beats-pull}}43511)
+* Add `enable_batch_api` option in Azure monitor to allow metrics collection of multiple resources using Azure batch API in Metricbeat. [#41790]({{beats-pull}}41790)
+
+### Fixes [beats-9.0.1-fixes]
+
+* For all Beats: Handle permission errors while collecting data from Windows services and don't interrupt the overall collection by skipping affected services. [#40765]({{beats-issue}}40765) [#43665]({{beats-pull}}43665).
+* Fixed WebSocket input panic on sudden network error or server crash in Filebeat. [#44063]({{beats-issue}}44063) [44068]({{beats-pull}}44068).
+* [Filestream] Log the "reader closed" message on the debug level to avoid log spam in Filebeat. [#44051]({{beats-pull}}44051)
+* Fix links to CEL mito extension functions in input documentation in Filebeat. [#44098]({{beats-pull}}44098)
+

docs/release-notes/_snippets/9.0.2/breaking-changes.md

@@ -0,0 +1,3 @@
+## 9.0.2 [beats-9.0.2-breaking-changes]
+
+_No breaking changes._

docs/release-notes/_snippets/9.0.2/deprecations.md

@@ -0,0 +1,3 @@
+## 9.0.2 [elastic-agent-9.0.2-deprecations]
+
+_No deprecations._