Skip to content

Commit e54b482

Browse files
BON4BON4
authored
[DPE-7693] | [DPE-7694] Add Canonical Observability Stack (COS) integration (#10)
* [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Basic COS * [DPE-7693] Fix config unit test * [DPE-7693] Fix config unit test * [DPE-7693] Fix config unit test * [DPE-7693] Fix config unit test
1 parent 74217ed commit e54b482

File tree

15 files changed

+6700
-8
lines changed

15 files changed

+6700
-8
lines changed

‎charmcraft.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ charm-libs:
2323
version: "0"
2424
- lib: tls_certificates_interface.tls_certificates
2525
version: "4"
26+
- lib: grafana_agent.cos_agent
27+
version: "0"
2628

2729
config:
2830
options:
@@ -35,6 +37,11 @@ config:
3537
type: string
3638
default: Test Cluster
3739

40+
provides:
41+
cos-agent:
42+
interface: cos_agent
43+
optional: true
44+
3845
peers:
3946
cassandra-peers:
4047
interface: cassandra_peers

‎docs/how-to/monitoring.md

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
# How to set up monitoring
2+
3+
Charmed Apache Cassnadra Snap come with the [JMX exporter](https://github.com/prometheus/jmx_exporter/).
4+
The metrics can be queried by accessing the `http://<cassandra-unit-ip>:7071/metrics` endpoint.
5+
6+
Additionally, the charm provides integration with the [Canonical Observability Stack](https://charmhub.io/topics/canonical-observability-stack).
7+
8+
(how-to-monitoring-enable-monitoring)=
9+
## Enable monitoring
10+
11+
Deploy the `cos-lite` bundle in a Kubernetes environment. This can be done by following the
12+
[deployment tutorial](https://charmhub.io/topics/canonical-observability-stack/tutorials/install-microk8s).
13+
Since the Charmed Apache Cassandra is deployed directly on a cloud infrastructure environment, it is needed to offer the endpoints of the COS relations.
14+
The [offers-overlay](https://github.com/canonical/cos-lite-bundle/blob/main/overlays/offers-overlay.yaml)
15+
can be used, and this step is shown in the COS tutorial.
16+
17+
### Offer interfaces via the COS controller
18+
19+
Switch to COS K8s environment and offer COS interfaces to be cross-model related with Charmed Apache Kafka VM model:
20+
21+
```shell
22+
juju switch <k8s_controller>:<cos_model_name>
23+
24+
juju offer grafana:grafana-dashboard grafana-dashboards
25+
juju offer loki:logging loki-logging
26+
juju offer prometheus:receive-remote-write prometheus-receive-remote-write
27+
```
28+
29+
### Consume offers via the Apache Cassnadra model
30+
31+
Switch back to the Charmed Apache Cassandra model, find offers and relate with them:
32+
33+
```shell
34+
juju switch <machine_controller_name>:<cassandra_model_name>
35+
36+
juju find-offers <k8s_controller>:
37+
```
38+
39+
A similar output should appear, if `k8s` is the K8s controller name and `cos` the model where `cos-lite` has been deployed:
40+
41+
```shell
42+
Store URL Access Interfaces
43+
k8s admin/cos.grafana-dashboards admin grafana_dashboard:grafana-dashboard
44+
k8s admin/cos.loki-logging admin loki_push_api:logging
45+
k8s admin/cos.prometheus-receive-remote-write admin prometheus-receive-remote-write:receive-remote-write
46+
...
47+
```
48+
49+
Consume offers to be reachable in the current model:
50+
51+
```shell
52+
juju consume <k8s_controller>:admin/<cos_model_name>.prometheus-receive-remote-write
53+
juju consume <k8s_controller>:admin/<cos_model_name>.loki-logging
54+
juju consume <k8s_controller>:admin/<cos_model_name>.grafana-dashboards
55+
```
56+
57+
Now, deploy `grafana-agent` (subordinate charm) and relate it with Charmed Apache Cassandra:
58+
59+
```shell
60+
juju deploy grafana-agent
61+
juju relate cassandra:cos-agent grafana-agent
62+
```
63+
64+
Finally, relate `grafana-agent` with consumed COS offers:
65+
66+
```shell
67+
juju relate grafana-agent grafana-dashboards
68+
juju relate grafana-agent loki-logging
69+
juju relate grafana-agent prometheus-receive-remote-write
70+
```
71+
72+
Wait for all components to settle down on a `active/idle` state on both models, e.g. `<cassandra_model_name>` and `<cos_model_name>`.
73+
74+
After this is complete, the monitoring COS stack should be up and running and ready to be used.
75+
76+
### Connect Grafana web interface
77+
78+
To connect to the Grafana web interface, follow the [Browse dashboards](https://charmhub.io/topics/canonical-observability-stack/tutorials/install-microk8s?_ga=2.201254254.1948444620.1704703837-757109492.1701777558#heading--browse-dashboards) section of the MicroK8s "Getting started" guide.
79+
80+
```shell
81+
juju run grafana/leader get-admin-password --model <k8s_cos_controller>:<cos_model_name>
82+
```
83+
84+
## Alerts and dashboards
85+
86+
This guide shows you how to integrate an existing set of rules and/or dashboards to your Charmed Apache Cassndra deployment to be consumed with the [Canonical Observability Stack (COS)](https://charmhub.io/topics/canonical-observability-stack).
87+
To do so, we will sync resources stored in a git repository to COS Lite.
88+
89+
### Prerequisites
90+
91+
Deploy the `cos-lite` bundle in a Kubernetes environment and integrate Charmed Apache Cassandra to the COS offers, as shown in the [How to Enable Monitoring](how-to-monitoring-enable-monitoring) guide.
92+
This guide will refer to the models that charms are deployed into as:
93+
94+
* `<cos-model>` for the model containing observability charms (and deployed on K8s)
95+
* `<apps-model>` for the model containing Charmed Apache Cassandra
96+
* `<apps-model>` for other optional charms (e.g. TLS-certificates operators, `grafana-agent`, `data-integrator`, etc.).
97+
98+
### Create a repository with a custom monitoring setup
99+
100+
Create an empty git repository, or in an existing one, save your alert rules and dashboard models under the `<path_to_prom_rules>`, `<path_to_loki_rules>` and `<path_to_models>` folders.
101+
102+
If you want a primer to rule writing, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/).
103+
104+
Then, push your changes to the remote repository.
105+
106+
### Deploy the COS configuration charm
107+
108+
Deploy the [COS configuration](https://charmhub.io/cos-configuration-k8s) charm in the `<cos-model>` model:
109+
110+
```shell
111+
juju deploy cos-configuration-k8s cos-config \
112+
--config git_repo=<repository_url> \
113+
--config git_branch=<branch> \
114+
```
115+
116+
The COS configuration charm keeps the monitoring stack in sync with our repository, by forwarding resources to Prometheus, Loki and Grafana.
117+
Refer to the [documentation](https://charmhub.io/cos-configuration-k8s/configure) for all configuration options, including how to access a private repository.
118+
Adding, updating or deleting an alert rule or a dashboard in the repository will be reflected in the monitoring stack.
119+
120+
```{note}
121+
You need to manually refresh `cos-config`'s local repository with the *sync-now* action if you do not want to wait for the next [update-status event](https://documentation.ubuntu.com/juju/3.6/reference/hook/#update-status) to pull the latest changes.
122+
```
123+
124+
### Forward the rules and dashboards
125+
126+
The path to the resource folders can be set after deployment:
127+
128+
```shell
129+
juju config cos-config \
130+
--config prometheus_alert_rules_path=<path_to_prom_rules> \
131+
--config loki_alert_rules_path=<path_to_loki_rules> \
132+
--config grafana_dashboards_path=<path_to_models>
133+
```
134+
135+
Then, integrate the charm with the COS operators to forward the rules and dashboards:
136+
137+
```shell
138+
juju integrate cos-config prometheus
139+
juju integrate cos-config grafana
140+
juju integrate cos-config loki
141+
```
142+
143+
After this is complete, the monitoring COS stack should be up and ready to fire alerts based on our rules. As for the dashboards, they should be available in the Grafana interface.
144+
145+
### Pre‑built dashboards and alerting rules
146+
147+
This repository already contains a set of ready‑to‑use monitoring assets:
148+
149+
* **Grafana dashboard:** `src/grafana_dashboards/grafana-dashboard.json`
150+
* **Loki alerting rules:** `src/alert_rules/loki/loki_alerts.yaml`
151+
* **Prometheus alerting rules:** `src/alert_rules/prometheus/prometheus_alerts.yaml`
152+
153+
Point the corresponding `*_path` configuration options to these files (or their parent directories) if you would like COS to pick them up automatically.
154+
155+
### Conclusion
156+
157+
In this guide, we enabled monitoring on a Charmed Apache Kafka deployment and integrated alert rules and dashboards by syncing a git repository to the COS stack.

0 commit comments

Comments
 (0)