Prometheus metrics for gha-rs-controller/gha-scale-set not showing.

[namhyuck-kim-otsk]

Hello everyone.

I've installed ARC (Github Official) and enabled metrics.

The PodMonitor manifest is as follows:

---
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: gha-rs-controller
  namespace: monitoring
  labels:
    release: kube-prometheus-stack
spec:
  podMetricsEndpoints:
  - interval: 30s
    targetPort: 8080
    path: /metrics
  namespaceSelector:
    matchNames:
    - arc-systems
  selector:
    matchLabels:
      app.kubernetes.io/part-of: gha-rs-controller
---
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: gha-runner-scale-set
  namespace: monitoring
  labels:
    release: kube-prometheus-stack
spec:
  podMetricsEndpoints:
  - interval: 30s
    targetPort: 8080
    path: /metrics
  namespaceSelector:
    matchNames:
    - arc-runners
  selector:
    matchLabels:
      app.kubernetes.io/part-of: gha-runner-scale-set

I've also installed kube-prometheus-stack on the same cluster and imported the dashboard json from here

The metrics seem to be working but many of the widgets are showing no data like the screenshot below.

Is this because the json file is out of date?

Anyone experiencing the same thing?

Thanks in advance.

[namhyuck-kim-otsk]

It seems the listeners aren't emitting the metrics.

[namhyuck-kim-otsk]

The PodMonitors should be in the same namespace as the controller and scaleset.