You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In Kubernetes 1.14 there was a major [metrics overhaul](https://github.com/kubernetes/enhancements/issues/1206) implemented. Therefore v0.1.x of this repository is the last release to support Kubernetes 1.13 and previous version on a best effort basis.
28
28
29
29
Some alerts now use Prometheus filters made available in Prometheus 2.11.0, which makes this version of Prometheus a dependency.
30
30
@@ -34,18 +34,16 @@ Warning: By default the expressions will generate *grafana 7.2+* compatible rule
34
34
35
35
## How to use
36
36
37
-
This mixin is designed to be vendored into the repo with your infrastructure config.
38
-
To do this, use [jsonnet-bundler](https://github.com/jsonnet-bundler/jsonnet-bundler):
37
+
This mixin is designed to be vendored into the repo with your infrastructure config. To do this, use [jsonnet-bundler](https://github.com/jsonnet-bundler/jsonnet-bundler):
39
38
40
39
You then have three options for deploying your dashboards
41
40
1. Generate the config files and deploy them yourself
42
-
1. Use ksonnet to deploy this mixin along with Prometheus and Grafana
43
-
1. Use prometheus-operator to deploy this mixin (TODO)
41
+
2. Use ksonnet to deploy this mixin along with Prometheus and Grafana
42
+
3. Use prometheus-operator to deploy this mixin (TODO)
44
43
45
44
## Generate config files
46
45
47
-
You can manually generate the alerts, dashboards and rules files, but first you
48
-
must install some tools:
46
+
You can manually generate the alerts, dashboards and rules files, but first you must install some tools:
49
47
50
48
```
51
49
$ go install github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@latest
@@ -68,13 +66,11 @@ $ make prometheus_rules.yaml
68
66
$ make dashboards_out
69
67
```
70
68
71
-
The `prometheus_alerts.yaml` and `prometheus_rules.yaml` file then need to passed
72
-
to your Prometheus server, and the files in `dashboards_out` need to be imported
73
-
into you Grafana server. The exact details will depending on how you deploy your
74
-
monitoring stack to Kubernetes.
69
+
The `prometheus_alerts.yaml` and `prometheus_rules.yaml` file then need to passed to your Prometheus server, and the files in `dashboards_out` need to be imported into you Grafana server. The exact details will depending on how you deploy your monitoring stack to Kubernetes.
75
70
76
71
### Dashboards for Windows Nodes
77
-
There are separate dashboards for windows resources.
72
+
73
+
There exist separate dashboards for windows resources.
a [ksonnet](https://github.com/ksonnet/ksonnet) module to deploy a fully-fledged
97
-
Prometheus-based monitoring system for Kubernetes:
90
+
Alternatively you can also use the mixin with [prometheus-ksonnet](https://github.com/kausalco/public/tree/master/prometheus-ksonnet), a [ksonnet](https://github.com/ksonnet/ksonnet) module to deploy a fully-fledged Prometheus-based monitoring system for Kubernetes:
98
91
99
92
Make sure you have the ksonnet v0.8.0:
100
93
@@ -115,8 +108,7 @@ $ cd <application name>
115
108
$ ks env add default
116
109
```
117
110
118
-
Grab the kubernetes-jsonnet module using and its dependencies, which include
119
-
the kubernetes-mixin:
111
+
Grab the kubernetes-jsonnet module using and its dependencies, which include the kubernetes-mixin:
120
112
121
113
```
122
114
$ go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb
@@ -158,9 +150,7 @@ Kubernetes-mixin can support dashboards across multiple clusters. You need eithe
158
150
159
151
## Customising the mixin
160
152
161
-
Kubernetes-mixin allows you to override the selectors used for various jobs,
162
-
to match those used in your Prometheus set. You can also customize the dashboard
163
-
names and add grafana tags.
153
+
Kubernetes-mixin allows you to override the selectors used for various jobs, to match those used in your Prometheus set. You can also customize the dashboard names and add grafana tags.
164
154
165
155
In a new directory, add a file `mixin.libsonnet`:
166
156
@@ -225,32 +215,32 @@ local utils = import 'lib/utils.libsonnet';
225
215
}
226
216
)
227
217
```
218
+
228
219
Create new file: `lib/kubernetes_customised_alerts.jsonnet` with the following:
Running `jsonnet -S lib/kubernetes_customised_alerts.jsonnet` will build the alerts with your customisations.
234
226
235
227
Same result can be achieved by modyfying the existing `config.libsonnet` with the content of `kubernetes_mixin_override.libsonnet`.
236
228
237
229
## Background
238
230
239
231
### Alert Severities
232
+
240
233
While the community has not yet fully agreed on alert severities and their to be used, this repository assumes the following paradigms when setting the severities:
241
234
242
235
* Critical: An issue, that needs to page a person to take instant action
243
236
* Warning: An issue, that needs to be worked on but in the regular work queue or for during office hours rather than paging the oncall
244
237
* Info: Is meant to support a trouble shooting process by informing about a non-normal situation for one or more systems but not worth a page or ticket on its own.
245
238
246
-
247
239
### Architecture and Technical Decisions
248
240
249
-
* For more motivation, see
250
-
"[The RED Method: How to instrument your services](https://kccncna17.sched.com/event/CU8K/the-red-method-how-to-instrument-your-services-b-tom-wilkie-kausal?iframe=no&w=100%&sidebar=yes&bg=no)" talk from CloudNativeCon Austin.
241
+
* For more motivation, see "[The RED Method: How to instrument your services](https://kccncna17.sched.com/event/CU8K/the-red-method-how-to-instrument-your-services-b-tom-wilkie-kausal?iframe=no&w=100%&sidebar=yes&bg=no)" talk from CloudNativeCon Austin.
251
242
* For more information about monitoring mixins, see this [design doc](DESIGN.md).
252
243
253
244
## Note
254
245
255
-
You can use the external tool call [prom-metrics-check](https://github.com/ContainerSolutions/prom-metrics-check) to validate the created dashboards. This tool allows you to check if the metrics installed and used in Grafana dashboards exist in the Prometheus instance.
256
-
Please have a look at https://github.com/ContainerSolutions/prom-metrics-check.
246
+
You can use the external tool call [prom-metrics-check](https://github.com/ContainerSolutions/prom-metrics-check) to validate the created dashboards. This tool allows you to check if the metrics installed and used in Grafana dashboards exist in the Prometheus instance. Please have a look at https://github.com/ContainerSolutions/prom-metrics-check.
0 commit comments