Skip to content
This repository was archived by the owner on Sep 2, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
.. _kubernetes-config-logs:

*********************************************************************************
Configure logs and events for Kubernetes
Collect logs and events for the Collector for Kubernetes
*********************************************************************************

.. meta::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Get started with the Collector for Linux
linux-config-ootb.rst
Default Linux metrics <metrics-ootb-linux.rst>
linux-config.rst
linux-config-logs.rst
linux-upgrade.rst
linux-uninstall.rst
collector-configuration-tutorial/about-collector-config-tutorial.rst
Expand All @@ -30,8 +31,9 @@ To install and configure the Splunk Distribution of the OpenTelemetry Collector
See the default settings and configuration options at:

* :ref:`linux-config-ootb`
* :ref:`otel-linux-config`
* By default, you'll obtain these :ref:`metrics <ootb-metrics-windows>`
* :ref:`otel-linux-config`
* :ref:`linux-config-logs`

.. include:: /_includes/gdi/collector-common-options.rst

Expand Down
82 changes: 0 additions & 82 deletions gdi/opentelemetry/collector-linux/install-linux.rst
Original file line number Diff line number Diff line change
Expand Up @@ -107,88 +107,6 @@ If you're installing your Collector instance in a host with Docker, you need to
# or if specifying the user:group directly
$ docker run -v /var/run/docker.sock:/var/run/docker.sock:ro --user "splunk-otel-collector:$(stat -c '%g' /var/run/docker.sock)" quay.io/signalfx/splunk-otel-collector:latest <...>

Collect logs for the Collector for Linux
====================================================================

Use the Universal Forwarder to send logs to the Splunk platform. See more at :ref:`collector-with-the-uf`.

Fluentd is turned off by default. If you already installed Fluentd on a host, re-install the Collector without Fluentd using the ``--without-fluentd`` option.

.. _fluentd-manual-config-linux:

Collect Linux logs with Fluentd
---------------------------------------

If you have a Log Observer entitlement or want to collect logs for the target host with Fluentd, use the ``--with-fluentd`` option to also install Fluentd when installing the Collector. For example:

.. code-block:: bash

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-fluentd --realm $SPLUNK_REALM -- $SPLUNK_ACCESS_TOKEN

When turned on, the Fluentd service is configured by default to collect and forward log events with the ``@SPLUNK`` label to the Collector, which then sends these events to the HEC ingest endpoint determined by the ``--realm <SPLUNK_REALM>`` option. For example, ``https://ingest.<SPLUNK_REALM>.signalfx.com/v1/log``.

The following Fluentd plugins are also installed:

* ``capng_c`` for activating Linux capabilities.
* ``fluent-plugin-systemd`` for systemd journal log collection.

Additionally, the following dependencies are installed as prerequisites for the Fluentd plugins:

.. tabs::

.. tab:: Debian-based systems

* build-essential
* libcap-ng0
* libcap-ng-dev
* pkg-config

.. tab:: RPM-based systems

* Development Tools
* libcap-ng
* libcap-ng-devel
* pkgconfig

You can specify the following parameters to configure the package to send log events to a custom Splunk HTTP Event Collector (HEC) endpoint URL:

* ``--hec-url <URL>``
* ``--hec-token <TOKEN>``

HEC lets you send data and application events to a Splunk deployment over the HTTP and Secure HTTP (HTTPS) protocols. See :new-page:`Set up and use HTTP Event Collector in Splunk Web <https://docs.splunk.com/Documentation/Splunk/8.2.1/Data/UsetheHTTPEventCollector>`.

The main Fluentd configuration is installed to ``/etc/otel/collector/fluentd/fluent.conf``. Custom Fluentd source configuration files can be added to the ``/etc/otel/collector/fluentd/conf.d`` directory after installation.

Note the following:

* In this directory, all files with the .conf extension are automatically included by Fluentd.
* The td-agent user must have permissions to access the configuration files and the paths defined within.
* By default, Fluentd is configured to collect systemd journal log events from ``/var/log/journal``.

After any configuration modification, run ``sudo systemctl restart td-agent`` to restart the td-agent service.

If the td-agent package is upgraded after initial installation, you might need to set the Linux capabilities for the new version by performing the following steps for td-agent versions 4.1 or higher:

#. Check for the activated capabilities:

.. code-block:: bash

sudo /opt/td-agent/bin/fluent-cap-ctl --get -f /opt/td-agent/bin/ruby
Capabilities in `` /opt/td-agent/bin/ruby`` ,
Effective: dac_override, dac_read_search
Inheritable: dac_override, dac_read_search
Permitted: dac_override, dac_read_search

#. If the output from the previous command does not include ``dac_override`` and ``dac_read_search`` as shown above, run the following commands:

.. code-block:: bash

sudo td-agent-gem install capng_c
sudo /opt/td-agent/bin/fluent-cap-ctl --add "dac_override,dac_read_search" -f /opt/td-agent/bin/ruby
sudo systemctl daemon-reload
sudo systemctl restart td-agent

.. _configure-auto-instrumentation:

Configure automatic discovery for back-end applications
Expand Down
91 changes: 91 additions & 0 deletions gdi/opentelemetry/collector-linux/linux-config-logs.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
.. _otel-install-linux-logs:
.. _linux-config-logs:

***************************************************************
Collect logs for the Collector for Linux
***************************************************************

.. meta::

:description: Describes how to collect logs for the Splunk Distribution of OpenTelemetry Collector for Linux.

Use the Universal Forwarder to send logs to the Splunk platform. See more at :ref:`collector-with-the-uf`.

Fluentd is turned off by default. If you already installed Fluentd on a host, re-install the Collector without Fluentd using the ``--without-fluentd`` option.

.. _fluentd-manual-config-linux:

Collect Linux logs with Fluentd
===========================================================================

If you have a Log Observer entitlement or want to collect logs for the target host with Fluentd, use the ``--with-fluentd`` option to also install Fluentd when installing the Collector. For example:

.. code-block:: bash

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-fluentd --realm $SPLUNK_REALM -- $SPLUNK_ACCESS_TOKEN

When turned on, the Fluentd service is configured by default to collect and forward log events with the ``@SPLUNK`` label to the Collector, which then sends these events to the HEC ingest endpoint determined by the ``--realm <SPLUNK_REALM>`` option. For example, ``https://ingest.<SPLUNK_REALM>.signalfx.com/v1/log``.

The following Fluentd plugins are also installed:

* ``capng_c`` for activating Linux capabilities.
* ``fluent-plugin-systemd`` for systemd journal log collection.

Additionally, the following dependencies are installed as prerequisites for the Fluentd plugins:

.. tabs::

.. tab:: Debian-based systems

* build-essential
* libcap-ng0
* libcap-ng-dev
* pkg-config

.. tab:: RPM-based systems

* Development Tools
* libcap-ng
* libcap-ng-devel
* pkgconfig

You can specify the following parameters to configure the package to send log events to a custom Splunk HTTP Event Collector (HEC) endpoint URL:

* ``--hec-url <URL>``
* ``--hec-token <TOKEN>``

HEC lets you send data and application events to a Splunk deployment over the HTTP and Secure HTTP (HTTPS) protocols. See :new-page:`Set up and use HTTP Event Collector in Splunk Web <https://docs.splunk.com/Documentation/Splunk/8.2.1/Data/UsetheHTTPEventCollector>`.

The main Fluentd configuration is installed to ``/etc/otel/collector/fluentd/fluent.conf``. Custom Fluentd source configuration files can be added to the ``/etc/otel/collector/fluentd/conf.d`` directory after installation.

Note the following:

* In this directory, all files with the .conf extension are automatically included by Fluentd.
* The td-agent user must have permissions to access the configuration files and the paths defined within.
* By default, Fluentd is configured to collect systemd journal log events from ``/var/log/journal``.

After any configuration modification, run ``sudo systemctl restart td-agent`` to restart the td-agent service.

If the td-agent package is upgraded after initial installation, you might need to set the Linux capabilities for the new version by performing the following steps for td-agent versions 4.1 or higher:

#. Check for the activated capabilities:

.. code-block:: bash

sudo /opt/td-agent/bin/fluent-cap-ctl --get -f /opt/td-agent/bin/ruby
Capabilities in `` /opt/td-agent/bin/ruby`` ,
Effective: dac_override, dac_read_search
Inheritable: dac_override, dac_read_search
Permitted: dac_override, dac_read_search

#. If the output from the previous command does not include ``dac_override`` and ``dac_read_search`` as shown above, run the following commands:

.. code-block:: bash

sudo td-agent-gem install capng_c
sudo /opt/td-agent/bin/fluent-cap-ctl --add "dac_override,dac_read_search" -f /opt/td-agent/bin/ruby
sudo systemctl daemon-reload
sudo systemctl restart td-agent