kafka exporter

[sreejesh-radhakrishnan-db]

I have installed kafka-metric.yaml as given in the example , its deployed on to my GKE.

installed POD Monitor to report my kafka exporter details to Google Cloud Observability platform.

I cannot see any pre-req required for KAFKA exporter to work, Am I missing anything so that I can see KAFKA pods having port: tcp-prometheus - 9409 also as part of the pod?

how who or when will my KAFKA pods start to expose data via port: tcp-prometheus (assuming its 9409?) ? Do I need to do some pre-req?

Also as Google suggested - I have tried to see if my kafka pod is really exposing any data using this

      POD_NAME=gpds-uat-kafka-0
      NAMESPACE=gpds-kafka
      PORT_NUMBER=9404
      METRICS_PATH=/metrics
      kubectl get --raw /api/v1/namespaces/$NAMESPACE/pods/$POD_NAME:$PORT_NUMBER/proxy/$METRICS_PATH

and it said TIMEOUT

[scholzj]

Please check the older discussions about the difference between Kafka Exporter and Prometheus metrics exposed from Kafka itself. These are completely different things and apart from the fact that you usually want to use both of them, they are completely separate features.

[sreejesh-radhakrishnan-db]

@scholzj thanks for reply, I am installing kafa-exporter as documented in your example - I am seeing only this in the kafka-exporter log

2022-07-12T22:13:33.689821732Z+ exec /usr/bin/tini -w -e 143 -- /tmp/run.sh
Info
2022-07-12T22:13:33.700592012ZStarting kafka_exporter (version=1.4.2, branch=HEAD, revision=0d5d4ac4ba63948748cc2c53b35ed95c310cd6f2)
Info
2022-07-12T22:13:33.727231827ZListening on HTTP :9404

Is this how exporter suppose to behave? I cannot see anything else in log, also no call made to kafka - as no entry on KAFKA log too.

hence my original Q. am I missing any pre-req? appreciate your help

[scholzj]

I do not know exactly how much does it log. Do you see any errors? Can you get the metrics on port 9404?

[varuncobain]

can you share how to install kafka exporter in my strimzi operated kafka

[scholzj]

Kafka Exporter is enabled in the Kafka custom resource. In the section .spec.kafkaExporter. Please check the docs or examples on GitHub (it will be used int he metrics examples)

[sreejesh-radhakrishnan-db]

NO error at all on exporter . all I have is this line above

when Exporter make connection to KAFKA, will it be using port 9091 ? And will this be logged at KAFKA side at all?

[scholzj]

It will use port 9091, yes. I don't think Kafka logs new client connections by default. So it will probably not log unless you changed the logging configuration.

[sreejesh-radhakrishnan-db]

the LOG is bit misleading, LISTENING to 9404 on LOG really confused me.

I increased the KAFKA-exporter log to DEBUG and then also enabled the enableSaramaLogging , and I could see the broker connection was happening okay.

I have set up a PODMONTOR against kafka-exporter 9094 /metric end point and I can scrape out KAFKA LAG and display on GCO , Monitoring

Thanks @scholzj