Skip to content

Latest commit

 

History

History
107 lines (80 loc) · 3.33 KB

File metadata and controls

107 lines (80 loc) · 3.33 KB

Apache NiFi Alerting System

The flow continuously monitors system and NiFi metrics (CPU, RAM, Threads, Disk usage, etc.), evaluates them against predefined thresholds, and triggers alerts via HTTP and Email when anomalies are detected.

The alerting pipeline is built around the following logical stages:

  • Flow Triggering
  • Metrics Collection
  • Metrics Extraction
  • Threshold Evaluation
  • Routing
  • Notification

Main Alerting Process Group

Main Alerting Process Group

Metrics System Alerting – Sub Process Group

Metrics Alerting – Sub Process Group

Metrics Alerting System Flow

Metrics System Alerting Flow p1 Metrics System Alerting Flow p2

Processors Description

  • GenerateFlowFile

Acts as a scheduler to trigger the monitoring flow periodically. GenerateFlowFile

  • InvokeHTTP

Metrics Collection InvokeHTTP

Calls a monitoring endpoint (NiFi API / system metrics API).

GET /nifi-api/system-diagnostics

system metrics API

  • EvaluateJsonPath

Extracts metrics from the JSON response. EvaluateJsonPath

  • UpdateAttribute

Normalizes metrics and defines the necessary attributes, such as metric thresholds for alerts. UpdateAttribute

  • RouteOnAttribute

Alert Detection : Compares metrics against thresholds and routes FlowFiles. RouteOnAttribute

  • ReplaceText

Alert Message Formatting : Builds a clean and readable alert message. ReplaceText

Notification Processors

  • PutEmail

Sends alert notifications via email. PutEmail

  • InvokeHTTP (Webhook / API)

Sends alerts to external systems (Slack, Teams, Monitoring API, etc.). InvokeHTTP (Webhook / API)

Queue Alerting – Sub Process Group

Queue Alerting – Sub Process Group

Queue Alerting Flow

Queue Alerting Flow

Processors Description

In this sub-process group, we use the same processors; the only difference is the API endpoint used to extract different NiFi metrics.

  • InvokeHTTP

Metrics Collection InvokeHTTP2

Calls a monitoring endpoint (NiFi API / system metrics API).

GET /nifi-api/flow/status

system metrics API2

  • EvaluateJsonPath

Extracts metrics from the JSON response.

EvaluateJsonPath2

Results

Email Notification

  • CPU metric alert CPU metric alert
  • Threads metric alert threads metric alert
  • Que FlowFile metric alert Que FlowFile metric alert

Webhook / API Notification

  • CPU metric alert CPU metric alert1
  • Threads metric alert Threads metric alert1
  • Que FlowFile metric alert Que FlowFile metric alert1

Requirements

  • Apache NiFi 1.22
  • Access to metrics endpoint
  • SMTP server (for email alerts)
  • Optional webhook endpoint