linkerd-prometheus configuration

[ahmetgeymen]

Is it possible to configure linkerd-prometheus to pull metrics from apps beside linkerd-proxy. I want to set config settings like ports and endpoints with basic auth headers so I can use one prometheus service in my cluster that comes with linkerd.

Thus I will be able to see both app and network metrics together. Is it possible?

Thanks.

[ihcsim]

I'd recommend setting up a separate Prometheus instance for your use case. The data stored in the Linkerd Prometheus is retained for up to only 6 hours, and on restart/upgrade, they get wiped.

As of the more recent edge releases, the Linkerd Prometheus is now an optional add-on. You can disable it following the instructions here. I think the exact flag to disable to Prometheus add-on is prometheus.enable=false. It's in the values.yaml file. Then set up your own Prometheus to scrape Linkerd metrics following this doc.

[ahmetgeymen]

I understand, with supporting Linkerd Prometheus as an optional add-on, it seems there will not be any configuration as I asked.

Thanks for the reply and recommendation.

[Abrishges]

Although this response may be late, it could still be valuable to someone. I recently encountered a situation where I required additional metrics for Canary deployment with Flagger. Interestingly, you can generate additional targets for linkerd-prometheus, allowing you to scrape them effortlessly. These supplementary metrics will complement the default ones.

To accomplish this, create an extra viz-values.yaml file, and then utilize the -f flag to supply this file as an override for the default values

This configuration will add additional targets in prometheus.yaml file at this location.

   scrapeConfigs:
    - job_name: "kubernetes-pods"
      kubernetes_sd_configs:
        - role: pod
      relabel_configs:
        - source_labels:
            [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
          action: keep
          regex: true
        - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
          action: replace
          target_label: __metrics_path__
          regex: (.+)
        - source_labels:
            [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
          action: replace
          regex: ([^:]+)(?::\d+)?;(\d+)
          replacement: $1:$2
          target_label: __address__ ```