[FR] [DAC] Consideration: Safeguards for Kibana Rule Loading Race Conditions

[eric-forte-elastic]

Repository Feature

Detections-as-Code (DaC) - (primarily custom rule management)

Problem Description

Thanks @17cell for providing this info!

We are getting reports from customers where they are running into race conditions when loading rules into Kibana via DaC commands (primarily kibana import-rules) where there are numerous rule exceptions Similarly to #4577, we may want to explore having a migation to attempt to prevent this from occurring from within this repo.

For a specific case, we have a customer with Approx 200 rules where ~60 of them have exceptions. When the rules are loaded into Kibana, some of the rules are updated in place before the exceptions are applied leading to an influx of false positive alerts that would have been caught/ignored by the exceptions. The exceptions are then applied but only after these alerts have been generated leading to alert pollution in the Security view.

As a workaround, this customer is loading the rules as disabled and then shortly after re-enabling them as then they will have the exceptions correctly attached.

Desired Solution

We may want to provide a CLI flag to also load the rules as disabled, wait for a short time (e.g. 20-30 seconds) and then enable the rules in an effort to reduce the impact of this issue.

Considered Alternatives

Generally speaking, if this can be easily reproduced then this should be addressed in Kibana.

Additional Context

No response