Skip to content

Commit 3cd62f5

Browse files
Copilotflorent-leborgne
authored andcommitted
Document 120-day default retention period for ML jobs in log anomalies and log categories (elastic#3499)
## Summary This PR updates the documentation to reflect the new default retention period of 120 days for ML results in the predefined ML configurations for logs that power the log anomalies and log categories pages, introduced in version 9.2. ## Context Following the changes introduced in elastic/kibana#231080, the predefined ML configurations for logs now set a default retention period of 120 days for ML results, where previously there was no default retention period (results were retained indefinitely). ## Changes Added concise notes in active voice to three documentation files, with version-specific tags: 1. **`solutions/observability/logs/inspect-log-anomalies.md`** - Added a note in the "Enable log rate analysis and anomaly detection" section - Informs users that log anomaly ML jobs retain results for 120 days by default - Includes `:applies_to: stack: ga 9.2` tag 2. **`solutions/observability/logs/categorize-log-entries.md`** - Added a note in the "Create log categories" section - Informs users that log categorization ML jobs retain results for 120 days by default - Includes `:applies_to: stack: ga 9.2` tag 3. **`explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md`** - Updated the "Optimize the results retention" section - Clarified that while anomaly detection results are retained indefinitely by default, predefined ML configurations for logs are an exception with 120-day retention (from version 9.2) - Includes inline `{applies_to}`stack: ga 9.2`` tag Each note: - States the 120-day default retention period - Explains how to customize the retention period via the `results_retention_days` setting - Clearly indicates this feature is available from version 9.2 onwards ## Impact - **Target release**: 9.2 - **Documentation sets affected**: Elastic On-Prem and Cloud (all) - **User benefit**: Users are now informed about the automatic cleanup of older ML results, helping them understand storage management and plan accordingly. Version-specific tags ensure users understand when this feature became available. ## Related - Kibana PR: elastic/kibana#231080 - Contact: @gbamparop Fixes elastic#2476 <!-- START COPILOT CODING AGENT SUFFIX --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>[Internal]: Setting a default retention period for the ML results for the jobs enabling log anomalies and log categories</issue_title> > <issue_description>### Description > > The predefined ML modules for logs that power the log anomalies and log categories pages didn't set a default retention period for the ML results, this will be updated to a default retention of 120 days. > > ### Resources > > - elastic/kibana#231080 > > ### Which documentation set does this change impact? > > Elastic On-Prem and Cloud (all) > > ### Feature differences > > There was no default retention period before and it will be set to 120 days. > > ### What release is this request related to? > 9.2 > > ### Serverless release > Date TBD > > ### Collaboration model > > The documentation team > > ### Point of contact. > > **Main contact:** > - @gbamparop > </issue_description> > > <agent_instructions>A simple note should suffice where relevant. Updates are probably focused in the following parts of the documentation: https://github.com/elastic/docs-content/tree/main/solutions/observability/logs and/or https://github.com/elastic/docs-content/tree/main/explore-analyze/machine-learning</agent_instructions> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> Fixes elastic#2476 <!-- START COPILOT CODING AGENT TIPS --> --- ✨ Let Copilot coding agent [set things up for you](https://github.com/elastic/docs-content/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: florent-leborgne <[email protected]> Co-authored-by: florent-leborgne <[email protected]>
1 parent 939c80c commit 3cd62f5

File tree

3 files changed

+13
-1
lines changed

3 files changed

+13
-1
lines changed

explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ See [Aggregating data for faster performance](ml-configuring-aggregation.md) to
100100

101101
Set a results retention window to reduce the amount of results stored.
102102

103-
{{anomaly-detect-cap}} results are retained indefinitely by default. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs.
103+
{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} configurations for logs which retain results for 120 days ({applies_to}`stack: ga 9.2`). Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs.
104104

105105
## 10. Optimize the renormalization window [renormalization-window]
106106

solutions/observability/logs/categorize-log-entries.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,12 @@ Create a {{ml}} job to categorize log messages automatically. {{ml-cap}} observe
2727
3. Add the indices that contain the logs you want to examine. By default, Machine Learning analyzes messages in all log indices that match the patterns set in the **logs sources** advanced setting. To open **Advanced settings**, find it in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).
2828
4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results.
2929

30+
::::{note}
31+
:applies_to: stack: ga 9.2
32+
33+
Log categorization {{ml}} jobs retain results for 120 days by default. Modify the `results_retention_days` setting to change this period.
34+
::::
35+
3036

3137
## Analyze log categories [analyze-log-categories]
3238

solutions/observability/logs/inspect-log-anomalies.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,12 @@ Create a {{ml}} job to detect anomalous log entry rates automatically.
3232
3. Add the indices that contain the logs you want to examine. By default, Machine Learning analyzes messages in all log indices that match the patterns set in the **logs source** advanced setting. To open **Advanced settings**, find it in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).
3333
4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results.
3434

35+
::::{note}
36+
:applies_to: stack: ga 9.2
37+
38+
Log anomaly {{ml}} jobs retain results for 120 days by default. Modify the `results_retention_days` setting to change this period.
39+
::::
40+
3541
## Anomalies chart [anomalies-chart]
3642

3743
The Anomalies chart shows an overall, color-coded visualization of the log entry rate, partitioned according to the value of the Elastic Common Schema (ECS) [`event.dataset`](ecs://reference/ecs-event.md) field. This chart helps you quickly spot increases or decreases in each partition’s log rate.

0 commit comments

Comments
 (0)