Skip to content

Commit fd09f17

Browse files
committed
docs(): alerting docs, grafana#6490
1 parent 8713dd8 commit fd09f17

File tree

5 files changed

+100
-44
lines changed

5 files changed

+100
-44
lines changed

docs/sources/alerting/notifications.md

Lines changed: 72 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -12,70 +12,109 @@ weight = 2
1212

1313
# Alert Notifications
1414

15+
{{< imgbox max-width="40%" img="/img/docs/v4/alert_notifications_menu.png" caption="Alerting notifications" >}}
1516

1617
> Alerting is only available in Grafana v4.0 and above.
1718
19+
When an alert changes state it sends out notifications. Each alert rule can have
20+
multiple notifications. But in order to add a notification to an alert rule you first need
21+
to add and configure a `notification` object. This is done from the Alerting/Notifications page.
1822

19-
When an alert changes state it sends a notification. One alert can be associated with multiple notifications. You can also configure notifications to be sent for all alerts within Grafana to make sure you won’t miss to configure notifications for an alert.
23+
## Notification Setup
2024

25+
On the notifications list page hit the `New Notification` button to go the the page where you
26+
can configure and setup a new notification.
2127

22-
You can find the alert notification page in the main menu under alerting.
28+
You you specify name and type, and type specific options. You can also test the notification to make
29+
sure it's working and setup correctly.
2330

31+
<!-- You can reach this page from the Alerting submenu or Alert List page header. -->
32+
<!-- When you configure a notification you can have it be a global notifiations, meaning -->
33+
<!-- it will be sent for all alerts within Grafana. This is useful to make sure you won’t miss to configure -->
34+
<!-- notifications for an alert. You can find the alert notification page in the main menu under alerting. -->
35+
<!-- -->
2436

25-
## Add a notifications to an Alert
26-
You can add and remove notifications from an alert by going to the `Notifications` sub menu in the alerting tab.
37+
<!-- ## Add a notifications to an Alert -->
38+
<!-- You can add and remove notifications from an alert by going to the `Notifications` sub menu in the alerting tab. -->
39+
<!-- -->
40+
<!-- -->
41+
<!-- <img class="no-shadow" src="/img/docs/v4/alerttab_notifications_submenu.png"> -->
42+
<!-- -->
43+
<!-- -->
44+
<!-- Click the `+` button to add a new notification and the `x` to remove. Notifications with a blue backgrounds are enabled by default for all alerts and cannot be modified from this view. -->
45+
<!-- -->
46+
<!-- -->
47+
<!-- <img class="no-shadow" src="/img/docs/v4/add_remove_notifications.png"> -->
48+
<!-- -->
2749

50+
### Send on all alerts
2851

29-
<img class="no-shadow" src="/img/docs/v4/alerttab_notifications_submenu.png">
52+
This option will make this notification used for all alert rules, existing and new.
3053

54+
## Supported notification types
3155

32-
Click the `+` button to add a new notification and the `x` to remove. Notifications with a blue backgrounds are enabled by default for all alerts and cannot be modified from this view.
56+
Grafana ships with a set of notification types. More will be added in future releases.
3357

34-
35-
<img class="no-shadow" src="/img/docs/v4/add_remove_notifications.png">
36-
37-
38-
## Default notifications
39-
40-
41-
On the notifications list page (`/alerting/notifications`) you can see all notifiers that have been enabled by default. To make an notification enabled by default
42-
you can check the `Send on all alerts` checkbox on the notification edit page.
43-
44-
45-
## Testing
46-
You can test an notification on the edit page by clicking the `Send Test` button. Grafana will then send test data based on the info on the edit page. This is a simple way of making sure that notifications are working as intended.
47-
48-
## Supported notifiers
4958
### Email
5059

51-
52-
To enable email notification you have to setup [SMTP settings](/installation/configuration/#smtp) in the Grafana config.
53-
Email notification will upload an image of the alert graph to an external image destination if available or fallback on attaching the image in the email.
54-
60+
To enable email notification you have to setup [SMTP settings](/installation/configuration/#smtp)
61+
in the Grafana config. Email notification will upload an image of the alert graph to an
62+
external image destination if available or fallback on attaching the image in the email.
5563

5664
### Webhook
57-
The webhook notification is a simple way to send information about an state change over HTTP to a custom endpoint.
58-
Using this notification you could integrated Grafana into any system you choose, by yourself.
5965

66+
The webhook notification is a simple way to send information about an state change over HTTP to a custom endpoint.
67+
Using this notification you could integrated Grafana into any system you choose, by yourself.
68+
69+
Example json schema:
70+
```json
71+
{
72+
"title": "My alert",
73+
"ruleId": 1,
74+
"ruleName": "Load peaking!",
75+
"ruleUrl": "http://url.to.grafana/db/dashboard/my_dashboard?panelId=2",
76+
"state": "Alerting",
77+
"imageUrl": "http://s3.image.url",
78+
"evalMatches": [
79+
{
80+
"metric": "requests",
81+
"tags": {},
82+
"value": 122
83+
}
84+
]
85+
}
86+
```
6087

6188
### Slack
62-
To set up slack you need to configure an incoming webhook url at slack. You can follow their guide for how to do that https://api.slack.com/incoming-webhooks
63-
If you want to include screenshots of the firing alerts in the slack messages you have to configure the [external image destination](/alerting/notifications/#graph-screenshots-in-notifiations) in Grafana.
89+
90+
{{< imgbox max-width="40%" img="/img/docs/v4/slack_notification.png" caption="Alerting Slack Notification" >}}
91+
92+
To set up slack you need to configure an incoming webhook url at slack. You can follow their guide for how
93+
to do that https://api.slack.com/incoming-webhooks If you want to include screenshots of the firing alerts
94+
in the slack messages you have to configure the [external image destination](#external-image-store) in Grafana.
6495

6596
Setting | Description
6697
---------- | -----------
6798
Recipient | allows you to override the slack recipient.
6899
Mention | make it possible to include a mention in the slack notification sent by Grafana. Ex @here or @channel
69100

70-
### Pagerduty
71-
To set up pagerduty, all you have to do is to provide an api key.
101+
### PagerDuty
72102

103+
To set up PagerDuty, all you have to do is to provide an api key.
73104

74105
> Our pagerduty integration only support trigger events at the moment. You have to resolve them by yourself.
75106
76107

77-
## Enable screenshot of alerting graphs
78-
Some notifiers requires you to configure the [external image uploader](/installation/configuration/#external-image-storage) to be able to upload screenshots of the alerts. This is not a requirement for any notifier within Grafana but we strongly encourage you to use it since the alert notifications give the recipient very valuable information.
108+
# Enable images in notifications {#external-image-store}
109+
110+
Grafan can render the panel associated with the alert rule and include that in the notification. Some types
111+
of notifications require that this image be publicly accessable (Slack for example). In order to support
112+
images in notifications like Slack Grafana can upload the image to an image store. It currently supports
113+
Amazon S3 for this and Webdav. So to set that up you need to configure the
114+
[external image uploader](/installation/configuration/#external-image-storage) in your grafana-server ini
115+
config file.
116+
117+
This is not an optional requirement, you can get slack and email notifications without setting this up.
79118

80119

81120

docs/sources/alerting/rules.md

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,18 +17,23 @@ weight = 1
1717

1818
{{< imgbox max-width="40%" img="/img/docs/v4/drag_handles_gif.gif" caption="Alerting overview" >}}
1919

20-
Alerting in Grafana allows you to attach rules to your graph panels. When you save the dashboard
20+
Alerting in Grafana allows you to attach rules to your dashboard panels. When you save the dashboard
2121
Grafana will extract the alert rules into a separate alert rule storage and schedule them for evaluation.
2222

2323
In the alert tab of the graph panel you can configure how often the alert rule should be evaluated
2424
and the conditions that need to be met for the alert to change state and trigger its
25-
[notifications]({{< relref "/notifications.md" >}}).
25+
[notifications]({{< relref "notifications.md" >}}).
2626

2727
## Execution
2828

2929
The alert rules are evaluated in the Grafana backend in a scheduler and query execution engine that is part
30-
of core Grafana. We have not implemented clustering for this scheduler yet, so if you run multiple
31-
instances of grafana-server you have to make sure [execute_alerts]({{< relref "/installation/configuration.md#alerting" >}})
30+
of core Grafana. Only some data soures are supported right now. They include `Graphite`, `Prometheus`,
31+
`InfluxDB` and `OpenTSDB`.
32+
33+
### Clustering
34+
35+
We have not implemented clustering yet. So if you run multiple instances of grafana-server
36+
you have to make sure [execute_alerts]({{< relref "/installation/configuration.md#alerting" >}})
3237
is true on only one instance or otherwise you will get duplicated notifications.
3338

3439
<div class="clearfix"></div>
@@ -38,8 +43,7 @@ is true on only one instance or otherwise you will get duplicated notifications.
3843
{{< imgbox max-width="40%" img="/img/docs/v4/alerting_conditions.png" caption="Alerting Conditions" >}}
3944

4045
Currently only the graph panel supports alert rules but this will be added to the **Singlestat** and **Table**
41-
panels as well in a future release. The rule config allows you to specify a name, how often the rule
42-
should be evaluated and a series of conditions that all need to be true for the alert to fire.
46+
panels as well in a future release.
4347

4448
### Name & Evaluation interval
4549

@@ -95,6 +99,14 @@ Keep Last State | Keep the current alert rule state, what ever it is.
9599
If you an unreliable time series store that where queries sometime timeout or fail randomly you can set this option
96100
t `Keep Last State` to basically ignore them.
97101

102+
## Notifications
103+
104+
In alert tab you can also specify alert rule notifications along with a detailed messsage about the alert rule.
105+
The message can contain anything, information about how you might solve the issue, link to runbook etc.
106+
107+
The actual notifications are configured and shared between multiple alerts. Read the
108+
[Notifications]({{< relref "notifications.md" >}}) guide for how to configure and setup notifications.
109+
98110
## Troubleshooting
99111

100112
{{< imgbox max-width="40%" img="/img/docs/v4/alert_test_rule.png" caption="Test Rule" >}}

pkg/services/alerting/models.go

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,9 @@ type ResultLogEntry struct {
1414
}
1515

1616
type EvalMatch struct {
17-
Value float64
18-
Metric string
19-
Tags map[string]string
17+
Value float64 `json:"value"`
18+
Metric string `json:"metric"`
19+
Tags map[string]string `json:"tags"`
2020
}
2121

2222
type Level struct {

pkg/services/alerting/notifiers/webhook.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,11 +51,11 @@ func (this *WebhookNotifier) Notify(evalContext *alerting.EvalContext) error {
5151

5252
ruleUrl, err := evalContext.GetRuleUrl()
5353
if err == nil {
54-
bodyJSON.Set("rule_url", ruleUrl)
54+
bodyJSON.Set("ruleUrl", ruleUrl)
5555
}
5656

5757
if evalContext.ImagePublicUrl != "" {
58-
bodyJSON.Set("image_url", evalContext.ImagePublicUrl)
58+
bodyJSON.Set("imageUrl", evalContext.ImagePublicUrl)
5959
}
6060

6161
body, _ := bodyJSON.MarshalJSON()

public/app/features/alerting/partials/alert_list.html

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,11 @@ <h1>Alert List</h1>
88
<i class="fa fa-info-circle"></i>
99
How to add an alert
1010
</a>
11+
12+
<a class="btn btn-inverse" href="alerting/notifications" >
13+
<i class="fa fa-cog"></i>
14+
Configure notifications
15+
</a>
1116
</div>
1217

1318
<div class="gf-form-group">

0 commit comments

Comments
 (0)