You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 2, 2025. It is now read-only.
Copy file name to clipboardExpand all lines: gdi/opentelemetry/components/kafkametrics-receiver.rst
+155-2Lines changed: 155 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,8 +7,161 @@ Kafka metrics receiver
7
7
.. meta::
8
8
:description: Collects Kafka metrics such as brokers, topics, partitions, and consumer groups from Kafka server, and converts them to OTLP format.
9
9
10
-
The Splunk Distribution of the OpenTelemetry Collector supports the Kafka metrics receiver. Documentation is planned for a future release.
10
+
The Kafka metrics receiver collects Kafka metrics (such as brokers, topics, partitions, and consumer groups) from a Kafka server and converts them to OTLP format. The supported pipeline type is ``metrics``. See :ref:`otel-data-processing` for more information.
11
+
12
+
Get started
13
+
======================
14
+
15
+
Follow these steps to configure and activate the component:
16
+
17
+
1. Deploy the Splunk Distribution of the OpenTelemetry Collector to your host or container platform:
18
+
19
+
- :ref:`otel-install-linux`
20
+
- :ref:`otel-install-windows`
21
+
- :ref:`otel-install-k8s`
22
+
23
+
2. Configure the receiver as described in the next section.
24
+
3. Restart the Collector.
25
+
26
+
Sample configuration
27
+
---------------------------
28
+
29
+
To activate the Kafka metrics receiver, add ``kafkametrics`` to the ``receivers`` section of your configuration file, as shown in the following example:
30
+
31
+
.. code:: yaml
32
+
33
+
receivers:
34
+
kafkametrics:
35
+
protocol_version: 2.0.0
36
+
scrapers:
37
+
- brokers
38
+
- topics
39
+
- consumers
40
+
41
+
To complete the configuration, include the receiver in the ``metrics`` pipeline of the ``service`` section of your configuration file. For example:
42
+
43
+
.. code:: yaml
44
+
45
+
service:
46
+
pipelines:
47
+
metrics:
48
+
receivers: [kafkametrics]
49
+
50
+
Configuration options
51
+
--------------------------------------------
52
+
53
+
These settings are required:
54
+
55
+
* ``protocol_version``. No default. The Kafka protocol version, for example ``2.0.0``.
56
+
57
+
* ``scrapers``. No default. Any combination of the following scrapers can be enabled:
58
+
59
+
* ``topics``
60
+
61
+
* ``consumers``
62
+
63
+
* ``brokers``
64
+
65
+
The following settings are optional:
66
+
67
+
* ``brokers``. ``localhost:9092`` by default. The list of brokers to read from.
68
+
69
+
* ``resolve_canonical_bootstrap_servers_only``. ``false`` by default. Determines whether to resolve then reverse-lookup broker IPs during startup.
70
+
71
+
* ``topic_match``. ``^[^_].*$)`` by default. Regex pattern of topics to filter on metrics collection. The default filter excludes internal topics, which start with ``_``.
72
+
73
+
* ``group_match``. ``.*)`` by default. Regex pattern of consumer groups to filter on for metrics.
74
+
75
+
* ``client_id``. ``otel-metrics-receiver`` by default. Consumer client Id.
76
+
77
+
* ``collection_interval``. ``1m`` by default. Frequency of metric collection/scraping.
78
+
79
+
* ``initial_delay``. ``1s`` by default. Determines how long this receiver waits before starting.
80
+
81
+
* ``auth``. No default. Use one of the following:
82
+
83
+
* ``plain_text``. It has the following fields:
84
+
85
+
* ``username``. The username to use.
86
+
87
+
* ``password``. The password to use.
88
+
89
+
* ``tls``. It has the following fields:
90
+
91
+
* ``ca_file``. Use only if ``insecure`` is set to ``false``. Path to the CA cert. For a client it verifies the server certificate.
92
+
93
+
* ``cert_file``. Use only if ``insecure`` is set to ``false``. Path to the TLS cert to use for TLS required connections.
94
+
95
+
* ``key_file``. Use only if ``insecure`` is set to ``false``. Path to the TLS key to use for TLS required connections.
96
+
97
+
* ``insecure``. ``false`` by default. Disables the verification of the server's certificate chain and host name, ``InsecureSkipVerify`` in the tls configuration.
98
+
99
+
* ``server_name_override``. Indicates the name of the server requested by the client in order to support virtual hosting.
100
+
101
+
* ``kerberos``. It has the following fields:
102
+
103
+
* ``service_name``. Kerberos service name.
104
+
105
+
* ``realm``. Kerberos realm.
106
+
107
+
* ``use_keytab``. If ``true``, the keytab is used instead of the password.
108
+
109
+
* ``username``. The Kerberos username used to authenticate with KDC.
110
+
111
+
* ``password``. The Kerberos password used to authenticate with KDC.
112
+
113
+
* ``config_file``. Path to Kerberos configuration, for example ``/etc/krb5.conf``.
114
+
115
+
* ``keytab_file``. Path to the keytab file, for example ``/etc/security/kafka.keytab``.
116
+
117
+
* ``disable_fast_negotiation``. ``false`` by default. Disables the PA-FX-FAST negotiation (Pre-Authentication Framework - Fast). Some common Kerberos implementations do not support PA-FX-FAST negotiation.
118
+
119
+
Configuration example: Set authentication and the collection interval to 5 seconds
To find information about this component in the meantime, see :new-page:`Kafka metrics receiver <https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kafkametricsreceiver>` on GitHub.
0 commit comments