-
Notifications
You must be signed in to change notification settings - Fork 25.7k
[ML] Add daily task to manage .ml-state indices #137603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Add a rollover check for the AD results indices to the nightly ML maintenance task. The concrete AD reults indices now have a six digit suffix. This is necessary to keep track of rollover behaviour and to determine which index is the "latest" in the series. WIP
…nage_ad_results_indices
…nage_ad_results_indices
…manage_ad_results_indices
…manage_ad_results_indices
…nage_ad_results_indices
…manage_ad_results_indices
…manage_ad_results_indices
…nage_ad_results_indices
…manage_ad_results_indices
…nage_ad_results_indices
* Added the ability to set results_index_rollover_max_size to -1B to configure the nightly maintenance task not to trigger indices rollover
…nage_ad_results_indices
Add a daily maintenance task to roll over .ml-state indices if the index size exceeds a configurable default size (default 50GB). This replaces the previous method of using ILM to manage the state indices, as that was not a workable solution for serverless. This builds on the work done in PR elastic#136065 which provides similar functionality for results indices.
…nage_ad_state_indices
🔍 Preview links for changed docs |
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
|
Hi @edsavage, I've created a changelog YAML for you. |
Add a daily maintenance task to roll over .ml-state indices if the index size exceeds a configurable default size (default 50GB). This replaces the previous method of using ILM to manage the state indices, as that was not a workable solution for serverless. This builds on the work done in PR elastic#136065 which provides similar functionality for results indices. WIP
…nage_ad_state_indices
Add a daily maintenance task to roll over .ml-state indices if the index size exceeds a configurable default size (default 50GB).
This replaces the previous method of using ILM to manage the state indices, as that was not a workable solution for serverless.
This builds on the work done in PR #136065 which provides similar functionality for results indices.