prometheus-community compatibility

[kirankumarhm]

If I want to have to install prometheus provided by the community, is there any way I can configure kafka related dashboard?

https://prometheus-community.github.io/helm-charts

[scholzj]

Strimzi provides several things:

• The metrics endpoints which you can scrape from Prometheus
• Sample Grafana dashboards
• Sample Prometheus Alert definitions
• Sample PodMonitor files for use with the Prometheus Operator

In general, you can use these with any Grafana and Prometheus installation regardless of the installation method. But I'm afraid we cannot provide a detailed support how to do it for all the different ways how to deploy and configure them. So you might need to figure that out or ask in the appropriate community.

[andreyolv]

When using the prometheus community helm chart, use these configurations in the extraScrapeConfigs parameter:

    extraScrapeConfigs: |
      - job_name: 'kafka-exporter'
        static_configs:
        - targets:
          - kafka-exporter.kafka-strimzi:9404
       
      - job_name: 'kafka-strimzi-pods'
        scrape_interval: 10s
        scrape_timeout: 10s
        metrics_path: /metrics
        scheme: http
        kubernetes_sd_configs:
        - role: pod
          namespaces:
            names:
            - kafka-strimzi
        relabel_configs:
        - regex: __meta_kubernetes_pod_label_(strimzi_io_.+)
          action: labelmap
        - source_labels: [__meta_kubernetes_namespace]
          target_label: namespace
        - source_labels: [__meta_kubernetes_pod_name]
          target_label: kubernetes_pod_name
        - source_labels: [__meta_kubernetes_pod_node_name]
          target_label: node_name
        - source_labels: [__meta_kubernetes_pod_host_ip]
          target_label: node_ip
        - source_labels: [__meta_kubernetes_pod_container_port_number]
          regex: 9404
          action: keep

There is no service for kafka exporter, so a possibility is to create a service manually like:

apiVersion: v1
kind: Service
metadata:
  name: kafka-exporter
  namespace: kafka-strimzi
spec:
  selector:
    app.kubernetes.io/name: kafka-exporter
  ports:
  - name: "9404"
    port: 9404

The first job_name 'kafka-exporter' extract the kafka-exporter service.

The second job_name 'kafka-strimzi-pods' extract all pods in the kafka-strimz namespace, only in the 9404 ports, and get the metadata for kubernetes labeling in prometheus, labels what will be variables in grafana. The target_label is exactly the same name as the variables of grafana templates.

It is not possible to extract the metrics of brokers, zookeepers, connectors and cruise control through the service as done in kafka-exporter, because the label of prometheus is needed at the pod level, and if you try in the service, the labels are missing.

This worked perfectly for me.

I think it would be interesting, if possible, to add this example for the prometheus community helm chart in the documentation, to make it more complete.