Skip to content

Make PersistentVolume75PercentInThreeDays less reactive or more restriced #2823

@viktor-f

Description

@viktor-f

Description

The PersistentVolume75PercentInThreeDays alert can fire quite often, even if there are not really any issue with a volume.
There are both scenarios where smaller "bumps" in the volume usage seem to trigger the alert, as well as scenarios where there is a rather quick increase in usage for volumes that idle on low volume usage (e.g. thanos compactor).

We want to look at this alert and see how we can get fewer false positives from it.
One alerternative would be to make it less reactive, perhaps so that there needs to be an increase for longer before it triggers.
Another alternative could be to look at excluding some volumes that often triggers this alert even if it does not end up at a too high usage.

Additional context

No response

Definition of done

  • The PersistentVolume75PercentInThreeDays alert produces fewer false positives

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomerskind/improvementImprovement of existing features, e.g. code cleanup or optimizations.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions