Skip to content

Add CEL-based drop rules to fault-remediation for per-action event filtering #1072

Description

@jtschelling

Problem

Fault-remediation currently routes events to remediation actions purely by recommendedAction string. There's no mechanism to filter events based on other HealthEvent fields like errorCode, agent, or checkName.

This means if multiple sources generate events with the same recommendedAction (e.g., CONTACT_SUPPORT), there's no way to selectively remediate only a subset of them. All events with that action will trigger remediation, or none will.

Without filtering, operators are forced to either:

  1. Remediate all events for a given action regardless of source (noisy, potentially incorrect)
  2. Not configure that action at all (blocks legitimate use cases)

Proposed Solution: Drop Rules

Add a global, CEL-based event filter at the fault-remediation entry point. Drop rules are evaluated in handleRemediationEvent() before action selection — if any rule matches, the event is silently dropped (marked as processed, no side effects).

Config

# Only remediate CONTACT_SUPPORT events with a specific errorCode
[[dropRules]]
expression = "event.recommendedAction == 5.0 && !('MANUAL_SUPPORT_REQUEST' in event.errorCode)"

Rules use the same CEL environment as fault-quarantine (event.* variables, ext.Strings()), so operators familiar with FQ rule expressions can write drop rules without learning a new syntax.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions