|
| 1 | +.. _other-ingestion-collectd: |
| 2 | + |
| 3 | +Monitor hosts with collectd and OpenTelemetry |
| 4 | +===================================================================== |
| 5 | + |
| 6 | +.. meta:: |
| 7 | + :description: Use collectd and native OpenTelemetry to monitor services in Splunk Observability Cloud. See benefits, install, configuration, and metrics. |
| 8 | + |
| 9 | +To monitor your infrastructure with collectd using native OpenTelemetry in Splunk Observability Cloud, install a collectd daemon in your host and connect it to your Collector instance as described in this document. |
| 10 | + |
| 11 | +Benefits |
| 12 | +-------- |
| 13 | + |
| 14 | +.. raw:: html |
| 15 | + |
| 16 | + <div class="include-start" id="benefits.rst"></div> |
| 17 | + |
| 18 | +.. include:: /_includes/benefits.rst |
| 19 | + |
| 20 | +.. raw:: html |
| 21 | + |
| 22 | + <div class="include-stop" id="benefits.rst"></div> |
| 23 | + |
| 24 | + |
| 25 | +Configuration |
| 26 | +---------------------------------- |
| 27 | + |
| 28 | +Install a collectd daemon in your host and connect it to an OpenTelemetry Collector with the following steps: |
| 29 | + |
| 30 | +1. Install and configure collectd |
| 31 | +2. Configure the OpenTelemetry Collector |
| 32 | +3. Build and run |
| 33 | + |
| 34 | +1. Install and configure collectd |
| 35 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 36 | + |
| 37 | +Follow these steps to install and configure the collectd daemon: |
| 38 | + |
| 39 | +#. Install collectd as a Debian or Yum package in your host |
| 40 | +#. Configure the daemon to ingest free disk related metrics through `collectd/metrics.conf` |
| 41 | +#. Configure the daemon to send data over HTTP using `collectd/http.conf` |
| 42 | + |
| 43 | +In this example, the host is represented by an Ubuntu 24.04 docker image. |
| 44 | + |
| 45 | +.. code:: |
| 46 | +
|
| 47 | + services: |
| 48 | + collectd: |
| 49 | + build: collectd |
| 50 | + container_name: collectd |
| 51 | + depends_on: |
| 52 | + - otelcollector |
| 53 | + volumes: |
| 54 | + - ./collectd/http.conf:/etc/collectd/collectd.conf.d/http.conf |
| 55 | + - ./collectd/metrics.conf:/etc/collectd/collectd.conf.d/metrics.conf |
| 56 | + |
| 57 | + # OpenTelemetry Collector |
| 58 | + otelcollector: |
| 59 | + image: quay.io/signalfx/splunk-otel-collector:latest |
| 60 | + container_name: otelcollector |
| 61 | + command: ["--config=/etc/otel-collector-config.yml", "--set=service.telemetry.logs.level=debug"] |
| 62 | + volumes: |
| 63 | + - ./otel-collector-config.yml:/etc/otel-collector-config.yml |
| 64 | +
|
| 65 | +The http and metrics configuration files look like this: |
| 66 | + |
| 67 | +.. code:: yaml |
| 68 | +
|
| 69 | + # http.conf |
| 70 | + # The minimal configuration required to have collectd send data to an OpenTelemetry Collector |
| 71 | + # with a collectdreceiver deployed on port 8081. |
| 72 | + |
| 73 | + LoadPlugin write_http |
| 74 | + <Plugin "write_http"> |
| 75 | + <Node "collector"> |
| 76 | + URL "http://otelcollector:8081" |
| 77 | + Format JSON |
| 78 | + VerifyPeer false |
| 79 | + VerifyHost false |
| 80 | + </Node> |
| 81 | + </Plugin> |
| 82 | +
|
| 83 | +.. code:: yaml |
| 84 | +
|
| 85 | + # metrics.conf |
| 86 | + # An example of collectd plugin configuration reporting free disk space on the host. |
| 87 | + |
| 88 | + <LoadPlugin df> |
| 89 | + Interval 3600 |
| 90 | + </LoadPlugin> |
| 91 | + <Plugin df> |
| 92 | + ValuesPercentage true |
| 93 | + </Plugin> |
| 94 | +
|
| 95 | +1. Configure the OpenTelemetry Collector |
| 96 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 97 | + |
| 98 | +Set up your Collector instance to listen for traffic from the collectd daemon over HTTP with the :ref:`collectd-receiver`: |
| 99 | + |
| 100 | +.. code:: yaml |
| 101 | +
|
| 102 | + receivers: |
| 103 | + collectd: |
| 104 | + endpoint: "0.0.0.0:8081" |
| 105 | +
|
| 106 | + exporters: |
| 107 | + debug: |
| 108 | + verbosity: detailed |
| 109 | +
|
| 110 | + service: |
| 111 | + pipelines: |
| 112 | + metrics: |
| 113 | + receivers: [collectd] |
| 114 | + exporters: [debug] |
| 115 | +
|
| 116 | +.. caution:: Make sure to use ``0.0.0.0`` to expose port 8081 over the Docker network interface so that both Docker containers can interact. |
| 117 | + |
| 118 | +3. Build and run |
| 119 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 120 | + |
| 121 | +Run the example with the instruction to start the docker-compose setup and build the collectd container: |
| 122 | + |
| 123 | +.. code:: bash |
| 124 | +
|
| 125 | + $> docker compose up --build |
| 126 | +
|
| 127 | +Check that the Collector is receiving metrics and logging them to ``stdout`` via the debug exporter: |
| 128 | + |
| 129 | +.. code:: bash |
| 130 | +
|
| 131 | + $> docker logs otelcollector |
| 132 | +
|
| 133 | +A typical output is: |
| 134 | + |
| 135 | +.. code:: |
| 136 | +
|
| 137 | + StartTimestamp: 1970-01-01 00:00:00 +0000 UTC |
| 138 | + Timestamp: 2024-12-20 19:55:44.006000128 +0000 UTC |
| 139 | + Value: 38.976566 |
| 140 | + Metric #17 |
| 141 | + Descriptor: |
| 142 | + -> Name: percent_bytes.reserved |
| 143 | + -> Description: |
| 144 | + -> Unit: |
| 145 | + -> DataType: Gauge |
| 146 | + NumberDataPoints #0 |
| 147 | + Data point attributes: |
| 148 | + -> plugin: Str(df) |
| 149 | + -> plugin_instance: Str(etc-hosts) |
| 150 | + -> host: Str(ea1d62c7a229) |
| 151 | + -> dsname: Str(value) |
| 152 | + StartTimestamp: 1970-01-01 00:00:00 +0000 UTC |
| 153 | + Timestamp: 2024-12-20 19:55:44.006000128 +0000 UTC |
| 154 | + Value: 5.102245 |
| 155 | + {"kind": "exporter", "data_type": "metrics", "name": "debug"} |
| 156 | +
|
| 157 | +Troubleshooting |
| 158 | +--------------- |
| 159 | + |
| 160 | +.. raw:: html |
| 161 | + |
| 162 | + <div class="include-start" id="troubleshooting-components.rst"></div> |
| 163 | + |
| 164 | +.. include:: /_includes/troubleshooting-components.rst |
| 165 | + |
| 166 | +.. raw:: html |
| 167 | + |
| 168 | + <div class="include-stop" id="troubleshooting-components.rst"></div> |
| 169 | + |
| 170 | + |
| 171 | + |
0 commit comments