Skip to content

Confusing Names for Alert Namespaces in Grafana #144

@marcusboden

Description

@marcusboden

Bug Description

When using this charm with multiple files, the alerts are placed into the a single namespace in grafana. Here's an example:
I have two files containing some alerts for chrony and HostDown:

root@cos-configuration-0:/var/lib/juju/storage/content-from-git/0/repo/prometheus_alert_rules# ls -l HostDown.rules chrony.rules 
-rw-r--r-- 1 root root  841 Jan 15 10:18 HostDown.rules
-rw-r--r-- 1 root root 1499 Jan 15 10:18 chrony.rules

Inside:

root@cos-configuration-0:/var/lib/juju/storage/content-from-git/0/repo/prometheus_alert_rules# head HostDown.rules

groups:
  - name: HostDown
    rules:
      - alert: HostDown
        annotations:
          summary: The Host {{ $labels.agent_hostname }} ({{ $labels.juju_unit }}) in model {{ $labels.juju_model }} is down.
          description: |
            The Host has not reported any "node_load1" metrics in the last 5 mintes. This indicates that the host is offline. Please check the host immediately!\
            The alert will fire for three days, if the host does not come back up again. If this is expected (decommissioning, redeployment, etc.), silence this alert for three days.
root@cos-configuration-0:/var/lib/juju/storage/content-from-git/0/repo/prometheus_alert_rules# head chrony.rules

groups:
- name: ChronyAlerts
  rules:
  - alert: ChronyOffsetWarning
    annotations:
      description: |-
        Chrony Offset is getting high.
          VALUE = {{ $value }}
          LABELS: {{ $labels }}

This is how it looks in Grafana:

Image

Or looking at the whole namespace:

Image

The namespace is named after the HostDown alerts. It should be named after the charm (i.e. cos-configuration).

To Reproduce

  1. deploy cos lite or ha
  2. set up two different files with cos-configuration-k8s
  3. Look at grafana

Environment

cos-configuration        3.5.0    active      1  cos-configuration-k8s        2/edge          79  10.152.183.192  no       
grafana                  9.5.3    active      1  grafana-k8s                  2/edge         162  10.152.183.224  no       

Relevant log output

none

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions