Alertmanager: root route must specify a default receiver

[larsskj]

I have installed kube-prometheus-stack using the Helm chart in what's currently latest version. My Prometheus operator is version 0.63.0.

The Helm chart installs a default configuration for Alertmanager like this:

global:
  resolve_timeout: 5m
route:
  receiver: "null"
  group_by:
  - namespace
  routes:
  - receiver: "null"
    matchers:
    - alertname =~ "InfoInhibitor|Watchdog"
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 12h
inhibit_rules:
- target_matchers:
  - severity =~ warning|info
  source_matchers:
  - severity = critical
  equal:
  - namespace
  - alertname
- target_matchers:
  - severity = info
  source_matchers:
  - severity = warning
  equal:
  - namespace
  - alertname
- target_matchers:
  - severity = info
  source_matchers:
  - alertname = InfoInhibitor
  equal:
  - namespace
receivers:
- name: "null"
templates:
- /etc/alertmanager/config/*.tmpl

This is all very fine - but looking at the logs from the operator, I see a continuous stream of error messages like:

level=error ts=2023-05-07T18:16:14.552927978Z caller=klog.go:116 component=k8s_client_runtime func=ErrorDepth msg="sync \"monitoring/prometheus-stack-alertmanager\" failed: provision alertmanager configuration: failed to initialize from secret: root route must specify a default receiver"

This error message is repeated about every 15 mins.

Would anybody happen to know what goes on here? I've tried modifying the file, but to no avail: The operator detects and reloads the changed configuration, but the stream of error messages remain the same.

[larsskj]

Ahh - sorry to bother you: Found the solution myself.

I have experimented with various settings, and apparently the Helm chart had left me with two different config files - and I edited the wrong one. Removing the bad one and replacing it with the proper one solved the problem.