Skip to content
This repository was archived by the owner on Sep 2, 2025. It is now read-only.

Commit c261b80

Browse files
authored
Merge branch 'main' into patch-4
2 parents 47b36c1 + ce92552 commit c261b80

File tree

10 files changed

+143
-155
lines changed

10 files changed

+143
-155
lines changed

_includes/metric-categories.rst

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,8 +50,7 @@
5050
| This category is not part of the report.
5151
5252
* - 13
53-
- | APM Monitoring MetricSets
54-
| This category is not part of the report.
53+
- APM Monitoring MetricSets
5554

5655
* - 14
5756
- Infrastructure Monitoring function

gdi/get-data-in/connect/aws/aws-prereqs.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -412,7 +412,7 @@ Read more at the official AWS documentation:
412412

413413
* :new-page:`AWS Organization Service Control Policies <https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html>`
414414
* :new-page:`Permissions boundaries for IAM entities <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html>`
415-
* :new-page:`Troubleshooting IAM permission access denied or unauthorized errors <https://web.archive.org/web/20231129090004/https://repost.aws/knowledge-center/troubleshoot-iam-permission-errors>`
415+
* :new-page:`Troubleshooting IAM permission access denied or unauthorized errors <https://repost.aws/knowledge-center/troubleshoot-iam-permission-errors>`
416416

417417
.. _aws-regions:
418418

gdi/get-data-in/connect/aws/aws-troubleshooting.rst

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,22 +18,22 @@ If issues persist, you can also contact :ref:`support`.
1818
Error validating your AWS connection
1919
=========================================
2020

21-
The automatic attempt to validate a connection that you just configured fails, so there is no connection between Splunk Observability Cloud and your AWS account.
21+
The automatic attempt to validate a connection that you just configured fails, so there is no connection between Splunk Observability Cloud and your AWS account. This can include failed API calls with ``400`` error codes.
2222

2323
Cause
2424
^^^^^^
2525

26-
The connection might fail due to invalid Identity Access Management (IAM) policy used by your AWS integration.
26+
The connection might fail due to your AWS integration using invalid Identity Access Management (IAM) policies, or missing some of the required permissions.
2727

28-
If you use the AWS Organizations' :new-page:`Service control policies <https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html>` or :new-page:`Permission boundaries for IAM entities <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html>`, they
28+
If you use AWS Organizations' :new-page:`Service control policies <https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html>` or :new-page:`Permission boundaries for IAM entities <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html>`, they
2929
might impact the AWS IAM policy you're using to connect to Splunk Observability Cloud.
3030

3131
Solution
3232
^^^^^^^^^
3333

34-
Ensure all :ref:`aws-required-permissions` are included in your IAM policy.
34+
Make sure to include all the required policies to connect your AWS account to Splunk Observability Cloud, as described in :ref:`aws-required-permissions`. Note that required permissions depend on your connection method (polling or Metric Streams), and that tag collection requires specific permissions.
3535

36-
Also review the AWS Organizations' policies and boundaries you're using.
36+
You also need to review the AWS Organizations' policies and boundaries you're using.
3737

3838
.. _aws-ts-cloud:
3939

gdi/opentelemetry/collector-kubernetes/install-k8s.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ Depending on your destination, you need:
7272
* ``splunkObservability.accessToken``. Your Splunk Observability Cloud org access token with ingest authorization scope. See :ref:`admin-org-tokens`.
7373
* ``splunkObservability.realm``. Splunk realm to send telemetry data to. The default is ``us0``. See :new-page:`realms <https://dev.splunk.com/observability/docs/realms_in_endpoints/>`.
7474

75-
.. note:: The Collector for Kubernetes uses ``main`` as the default Splunk platform index.
75+
.. note:: The Collector for Kubernetes uses ``main`` as the default Splunk platform index. You can use annotations to send data to different indexes. See :ref:`kubernetes-config-logs-annotations-indexes` for more information.
7676

7777
Deploy the Helm chart
7878
==============================================

gdi/opentelemetry/collector-kubernetes/kubernetes-config-logs.rst

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,12 @@
22
.. _kubernetes-config-logs:
33

44
*********************************************************************************
5-
Configure logs and events for Kubernetes
5+
Collect logs and events for the Collector for Kubernetes
66
*********************************************************************************
77

88
.. meta::
99
:description: Configure logs and events for the Splunk Distribution of OpenTelemetry Collector for Kubernetes.
1010

11-
1211
.. note:: See how to configure the Collector for Kubernetes at :ref:`otel-kubernetes-config` and :ref:`otel-kubernetes-config-advanced`.
1312

1413
Starting on version 0.86.0, the Splunk Distribution of the Collector for Kubernetes collects native OpenTelemetry logs by default.
@@ -77,17 +76,26 @@ To process multi-line logs, add the following section to your values.yaml config
7776
7877
Use :new-page:`regex101 <https://regex101.com/ >` to find a Golang regex that works for your format and specify it in the config file for the config option ``firstEntryRegex``.
7978

79+
.. _kubernetes-config-logs-annotations:
80+
8081
Manage log ingestion using annotations
8182
===========================================================================
8283

83-
Use the ``splunk.com/index`` annotation on pods or namespaces to indicate which Splunk platform indexes you want to send logs to. Pod annotation will take precedence over namespace annotation when both are annotated.
84+
.. _kubernetes-config-logs-annotations-indexes:
85+
86+
Send logs to different indexes
87+
-----------------------------------------------------
88+
89+
The Collector for Kubernetes uses ``main`` as the default Splunk platform index. Use the ``splunk.com/index`` annotation on pods or namespaces to indicate which Splunk platform indexes you want to send logs to.
8490

8591
For example, to send logs from the ``kube-system`` namespace to the ``k8s_events`` index, use the command:
8692

8793
.. code-block:: bash
8894
8995
kubectl annotate namespace kube-system splunk.com/index=k8s_events
9096
97+
.. note:: A pod annotation takes precedence over a namespace annotation when both are annotated.
98+
9199
Filter logs using pod or namespace annotations
92100
-----------------------------------------------------
93101

gdi/opentelemetry/collector-linux/collector-linux-intro.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Get started with the Collector for Linux
1717
linux-config-ootb.rst
1818
Default Linux metrics <metrics-ootb-linux.rst>
1919
linux-config.rst
20+
linux-config-logs.rst
2021
linux-upgrade.rst
2122
linux-uninstall.rst
2223
collector-configuration-tutorial/about-collector-config-tutorial.rst
@@ -30,8 +31,9 @@ To install and configure the Splunk Distribution of the OpenTelemetry Collector
3031
See the default settings and configuration options at:
3132

3233
* :ref:`linux-config-ootb`
33-
* :ref:`otel-linux-config`
3434
* By default, you'll obtain these :ref:`metrics <ootb-metrics-windows>`
35+
* :ref:`otel-linux-config`
36+
* :ref:`linux-config-logs`
3537

3638
.. include:: /_includes/gdi/collector-common-options.rst
3739

gdi/opentelemetry/collector-linux/install-linux.rst

Lines changed: 0 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -107,88 +107,6 @@ If you're installing your Collector instance in a host with Docker, you need to
107107
# or if specifying the user:group directly
108108
$ docker run -v /var/run/docker.sock:/var/run/docker.sock:ro --user "splunk-otel-collector:$(stat -c '%g' /var/run/docker.sock)" quay.io/signalfx/splunk-otel-collector:latest <...>
109109
110-
Collect logs for the Collector for Linux
111-
====================================================================
112-
113-
Use the Universal Forwarder to send logs to the Splunk platform. See more at :ref:`collector-with-the-uf`.
114-
115-
Fluentd is turned off by default. If you already installed Fluentd on a host, re-install the Collector without Fluentd using the ``--without-fluentd`` option.
116-
117-
.. _fluentd-manual-config-linux:
118-
119-
Collect Linux logs with Fluentd
120-
---------------------------------------
121-
122-
If you have a Log Observer entitlement or want to collect logs for the target host with Fluentd, use the ``--with-fluentd`` option to also install Fluentd when installing the Collector. For example:
123-
124-
.. code-block:: bash
125-
126-
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
127-
sudo sh /tmp/splunk-otel-collector.sh --with-fluentd --realm $SPLUNK_REALM -- $SPLUNK_ACCESS_TOKEN
128-
129-
When turned on, the Fluentd service is configured by default to collect and forward log events with the ``@SPLUNK`` label to the Collector, which then sends these events to the HEC ingest endpoint determined by the ``--realm <SPLUNK_REALM>`` option. For example, ``https://ingest.<SPLUNK_REALM>.signalfx.com/v1/log``.
130-
131-
The following Fluentd plugins are also installed:
132-
133-
* ``capng_c`` for activating Linux capabilities.
134-
* ``fluent-plugin-systemd`` for systemd journal log collection.
135-
136-
Additionally, the following dependencies are installed as prerequisites for the Fluentd plugins:
137-
138-
.. tabs::
139-
140-
.. tab:: Debian-based systems
141-
142-
* build-essential
143-
* libcap-ng0
144-
* libcap-ng-dev
145-
* pkg-config
146-
147-
.. tab:: RPM-based systems
148-
149-
* Development Tools
150-
* libcap-ng
151-
* libcap-ng-devel
152-
* pkgconfig
153-
154-
You can specify the following parameters to configure the package to send log events to a custom Splunk HTTP Event Collector (HEC) endpoint URL:
155-
156-
* ``--hec-url <URL>``
157-
* ``--hec-token <TOKEN>``
158-
159-
HEC lets you send data and application events to a Splunk deployment over the HTTP and Secure HTTP (HTTPS) protocols. See :new-page:`Set up and use HTTP Event Collector in Splunk Web <https://docs.splunk.com/Documentation/Splunk/8.2.1/Data/UsetheHTTPEventCollector>`.
160-
161-
The main Fluentd configuration is installed to ``/etc/otel/collector/fluentd/fluent.conf``. Custom Fluentd source configuration files can be added to the ``/etc/otel/collector/fluentd/conf.d`` directory after installation.
162-
163-
Note the following:
164-
165-
* In this directory, all files with the .conf extension are automatically included by Fluentd.
166-
* The td-agent user must have permissions to access the configuration files and the paths defined within.
167-
* By default, Fluentd is configured to collect systemd journal log events from ``/var/log/journal``.
168-
169-
After any configuration modification, run ``sudo systemctl restart td-agent`` to restart the td-agent service.
170-
171-
If the td-agent package is upgraded after initial installation, you might need to set the Linux capabilities for the new version by performing the following steps for td-agent versions 4.1 or higher:
172-
173-
#. Check for the activated capabilities:
174-
175-
.. code-block:: bash
176-
177-
sudo /opt/td-agent/bin/fluent-cap-ctl --get -f /opt/td-agent/bin/ruby
178-
Capabilities in `` /opt/td-agent/bin/ruby`` ,
179-
Effective: dac_override, dac_read_search
180-
Inheritable: dac_override, dac_read_search
181-
Permitted: dac_override, dac_read_search
182-
183-
#. If the output from the previous command does not include ``dac_override`` and ``dac_read_search`` as shown above, run the following commands:
184-
185-
.. code-block:: bash
186-
187-
sudo td-agent-gem install capng_c
188-
sudo /opt/td-agent/bin/fluent-cap-ctl --add "dac_override,dac_read_search" -f /opt/td-agent/bin/ruby
189-
sudo systemctl daemon-reload
190-
sudo systemctl restart td-agent
191-
192110
.. _configure-auto-instrumentation:
193111

194112
Configure automatic discovery for back-end applications
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
.. _otel-install-linux-logs:
2+
.. _linux-config-logs:
3+
4+
***************************************************************
5+
Collect logs for the Collector for Linux
6+
***************************************************************
7+
8+
.. meta::
9+
10+
:description: Describes how to collect logs for the Splunk Distribution of OpenTelemetry Collector for Linux.
11+
12+
Use the Universal Forwarder to send logs to the Splunk platform. See more at :ref:`collector-with-the-uf`.
13+
14+
Fluentd is turned off by default. If you already installed Fluentd on a host, re-install the Collector without Fluentd using the ``--without-fluentd`` option.
15+
16+
.. _fluentd-manual-config-linux:
17+
18+
Collect Linux logs with Fluentd
19+
===========================================================================
20+
21+
If you have a Log Observer entitlement or want to collect logs for the target host with Fluentd, use the ``--with-fluentd`` option to also install Fluentd when installing the Collector. For example:
22+
23+
.. code-block:: bash
24+
25+
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
26+
sudo sh /tmp/splunk-otel-collector.sh --with-fluentd --realm $SPLUNK_REALM -- $SPLUNK_ACCESS_TOKEN
27+
28+
When turned on, the Fluentd service is configured by default to collect and forward log events with the ``@SPLUNK`` label to the Collector, which then sends these events to the HEC ingest endpoint determined by the ``--realm <SPLUNK_REALM>`` option. For example, ``https://ingest.<SPLUNK_REALM>.signalfx.com/v1/log``.
29+
30+
The following Fluentd plugins are also installed:
31+
32+
* ``capng_c`` for activating Linux capabilities.
33+
* ``fluent-plugin-systemd`` for systemd journal log collection.
34+
35+
Additionally, the following dependencies are installed as prerequisites for the Fluentd plugins:
36+
37+
.. tabs::
38+
39+
.. tab:: Debian-based systems
40+
41+
* build-essential
42+
* libcap-ng0
43+
* libcap-ng-dev
44+
* pkg-config
45+
46+
.. tab:: RPM-based systems
47+
48+
* Development Tools
49+
* libcap-ng
50+
* libcap-ng-devel
51+
* pkgconfig
52+
53+
You can specify the following parameters to configure the package to send log events to a custom Splunk HTTP Event Collector (HEC) endpoint URL:
54+
55+
* ``--hec-url <URL>``
56+
* ``--hec-token <TOKEN>``
57+
58+
HEC lets you send data and application events to a Splunk deployment over the HTTP and Secure HTTP (HTTPS) protocols. See :new-page:`Set up and use HTTP Event Collector in Splunk Web <https://docs.splunk.com/Documentation/Splunk/8.2.1/Data/UsetheHTTPEventCollector>`.
59+
60+
The main Fluentd configuration is installed to ``/etc/otel/collector/fluentd/fluent.conf``. Custom Fluentd source configuration files can be added to the ``/etc/otel/collector/fluentd/conf.d`` directory after installation.
61+
62+
Note the following:
63+
64+
* In this directory, all files with the .conf extension are automatically included by Fluentd.
65+
* The td-agent user must have permissions to access the configuration files and the paths defined within.
66+
* By default, Fluentd is configured to collect systemd journal log events from ``/var/log/journal``.
67+
68+
After any configuration modification, run ``sudo systemctl restart td-agent`` to restart the td-agent service.
69+
70+
If the td-agent package is upgraded after initial installation, you might need to set the Linux capabilities for the new version by performing the following steps for td-agent versions 4.1 or higher:
71+
72+
#. Check for the activated capabilities:
73+
74+
.. code-block:: bash
75+
76+
sudo /opt/td-agent/bin/fluent-cap-ctl --get -f /opt/td-agent/bin/ruby
77+
Capabilities in `` /opt/td-agent/bin/ruby`` ,
78+
Effective: dac_override, dac_read_search
79+
Inheritable: dac_override, dac_read_search
80+
Permitted: dac_override, dac_read_search
81+
82+
#. If the output from the previous command does not include ``dac_override`` and ``dac_read_search`` as shown above, run the following commands:
83+
84+
.. code-block:: bash
85+
86+
sudo td-agent-gem install capng_c
87+
sudo /opt/td-agent/bin/fluent-cap-ctl --add "dac_override,dac_read_search" -f /opt/td-agent/bin/ruby
88+
sudo systemctl daemon-reload
89+
sudo systemctl restart td-agent
90+
91+

logs/processors.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,9 @@ Going forward after the transition to Log Observer Connect, you can process data
4141
* - Data Stream Processor
4242
- See :new-page:`Use the Data Stream Processor <https://docs.splunk.com/Documentation/DSP/1.4.2/User/About>`.
4343

44+
* - Ingest Processor
45+
- See :new-page:`About Ingest Processor <https://docs.splunk.com/Documentation/SplunkCloud/latest/IngestProcessor/AboutIngestProcessorSolution>`.
46+
4447

4548
Prepackaged processing rules
4649
=============================================================================

0 commit comments

Comments
 (0)