[Internal]: [Dataset quality] Introducing Failed docs rule alternative

[yngrdyn]

Description

What: We want to enable users to monitor failed docs through alerts.
When: The feature will be available for 8.19 and 9.1, and serverless in the next release.
Why: This feature will help users to keep/improve the quality in their dataStreams. The rule will be based on custom threshold rule, the idea will be to guide users to monitor failed docs using that rule as a base. This is an alternative since we are not planning to introduce a dedicate rule or a custom UI for configuring failed docs alerts soon.

Resources

I've been playing with Custom threshold rule to create a failed docs alert.
The following video works only in an Observability profile, in Classic profile the custom threshold rule is not presented from discover

Screen.Recording.2025-08-26.at.11.mp4

For Classic profiles some extra steps need to be performed

Screen.Recording.2025-08-26.at.11.28.20.mov

In both cases users should go to stack management > [rules | alerts] to be able to see the new rule or the alerts raised.

Which documentation set does this change impact?

Elastic On-Prem and Cloud (all)

Feature differences

The feature is identical in stateful and serverless

What release is this request related to?

8.19

Serverless release

The week of Sep 1st

Collaboration model

The documentation team

Point of contact.

Main contact: @yngrdyn

Stakeholders: @gbamparop, @LucaWintergerst