Skip to content
This repository was archived by the owner on Sep 2, 2025. It is now read-only.

Commit 47b36c1

Browse files
authored
Merge branch 'main' into patch-4
2 parents 26040b3 + 60e3590 commit 47b36c1

File tree

7 files changed

+84
-34
lines changed

7 files changed

+84
-34
lines changed

_images/logs/WorkloadMgmt.png

149 KB
Loading

gdi/get-data-in/connect/aws/aws-troubleshooting.rst

Lines changed: 5 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -23,37 +23,17 @@ The automatic attempt to validate a connection that you just configured fails, s
2323
Cause
2424
^^^^^^
2525

26-
The connection might fail due to mismatched Identity Access Management (IAM) policies. To diagnose connection failure, check the permissions or policies you set up and compare them to the permissions that AWS requires.
26+
The connection might fail due to invalid Identity Access Management (IAM) policy used by your AWS integration.
2727

28-
Verify whether your error message looks similar to this example:
29-
30-
.. code-block:: none
31-
32-
Error validating AWS / Cloudwatch credentials
33-
Validation failed for following region(s):
34-
us-east-1
35-
[ec2] software.amazon.awssdk.services.ec2.model.Ec2Exception: You are not authorized to perform this operation.
36-
37-
If you receive a similar error message, then the IAM policy that you created to connect AWS to Splunk Observability Cloud does not match the policy already in your AWS account.
38-
39-
Similarly, if your AWS account uses a service control policy (SCP) or administrative features such as ``PermissionsBoundary``, then there might be limits on which calls can be made in your organization, even if those calls are covered by your AWS IAM policy.
28+
If you use the AWS Organizations' :new-page:`Service control policies <https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html>` or :new-page:`Permission boundaries for IAM entities <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html>`, they
29+
might impact the AWS IAM policy you're using to connect to Splunk Observability Cloud.
4030

4131
Solution
4232
^^^^^^^^^
4333

44-
Splunk Observability Cloud uses the following calls to validate whether it can accept data from the AWS Compute Optimizer tool to support CloudWatch metric streams:
45-
46-
.. code-block:: none
47-
48-
client.describeInstanceStatus(),
49-
client.describeTags(),
50-
client.describeReservedInstances(),
51-
client.describeReservedInstancesModifications()
52-
client.describeOrganization()
53-
54-
To ensure that your AWS integration works as expected, revisit your configuration choices in Splunk Observability Cloud to verify that they match the permissions policy in your AWS management console.
34+
Ensure all :ref:`aws-required-permissions` are included in your IAM policy.
5535

56-
A match ensures that conflicting permissions do not cause your AWS environment to block integrations. See the "Amazon CloudWatch permissions reference" in the Amazon documentation for details about the available permissions.
36+
Also review the AWS Organizations' policies and boundaries you're using.
5737

5838
.. _aws-ts-cloud:
5939

gdi/opentelemetry/collector-kubernetes/kubernetes-config-advanced.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ For example:
3838
- key: k8s.pod.name
3939
value: '^(podNameX)$'
4040
# Define the logs pipeline with the default values as well as your new processor component
41-
service:
41+
service:
4242
pipelines:
4343
logs:
4444
processors:

gdi/opentelemetry/collector-linux/install-linux.rst

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,29 @@ To skip these steps and use configured repos on the target system that provide t
8484
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
8585
sudo sh /tmp/splunk-otel-collector.sh --realm $SPLUNK_REALM --skip-collector-repo --skip-fluentd-repo \
8686
-- $SPLUNK_ACCESS_TOKEN
87+
88+
.. _collector-linux-with-docker:
89+
90+
Use the Collector in a host with Docker
91+
====================================================================
92+
93+
If you're installing your Collector instance in a host with Docker, you need to configure a client to establish a connection with the daemon. Depending on your Docker installation and Collector deployment method, try one of these options:
94+
95+
1. If your daemon is listening to a domain socket (for example ``/var/run/docker.sock``), your Collector service or executable needs appropriate permissions and access. Add the ``splunk-otel-collector`` user to the Docker group as configured on your system:
96+
97+
.. code-block:: bash
98+
99+
$ usermod -aG docker splunk-otel-collector
100+
101+
2. When using the :new-page:`quay.io/signalfx/splunk-otel-collector <https://quay.io/repository/signalfx/splunk-otel-collector>` image, add the default container user to the required group as configured on your system, and the bind and mount the domain socket:
102+
103+
.. code-block:: bash
104+
105+
$ docker run -v /var/run/docker.sock:/var/run/docker.sock:ro --group-add $(stat -c '%g' /var/run/docker.sock) quay.io/signalfx/splunk-otel-collector:latest <...>
87106
107+
# or if specifying the user:group directly
108+
$ docker run -v /var/run/docker.sock:/var/run/docker.sock:ro --user "splunk-otel-collector:$(stat -c '%g' /var/run/docker.sock)" quay.io/signalfx/splunk-otel-collector:latest <...>
109+
88110
Collect logs for the Collector for Linux
89111
====================================================================
90112

gdi/opentelemetry/components/host-metrics-receiver.rst

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -205,7 +205,26 @@ Process
205205
mute_process_io_error: <true|false>
206206
scrape_process_delay: <time>
207207
208-
If you keep getting errors related to process reading, consider setting ``mute_process_name_error``, ``mute_process_exe_error``, or ``mute_process_io_error`` to ``true``.
208+
The following example demonstrates how to configure a process scraper that collects two metrics, in addition to the defaults, and uses a resource attribute to include the process owner in the collected data:
209+
210+
.. code:: yaml
211+
212+
receivers:
213+
hostmetrics:
214+
scrapers:
215+
process:
216+
resource_attributes:
217+
process.owner:
218+
enabled: true
219+
metrics:
220+
process.memory.usage:
221+
enabled: true
222+
process.disk.io:
223+
enabled: true
224+
225+
For more information about enabling and disabling metrics and resource attributes using the process scraper, see :new-page:`hostmetricsreceiver/process <https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/internal/scraper/processscraper/documentation.md>` in the OpenTelemetry documentation.
226+
227+
If you continuously see errors related to process reading, consider setting ``mute_process_name_error``, ``mute_process_exe_error``, or ``mute_process_io_error`` to ``true``.
209228

210229
Filtering
211230
----------------------

logs/scp.rst

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -101,19 +101,34 @@ In Splunk Cloud Platform, follow the instructions in the guided setup for the in
101101
:width: 100%
102102
:alt: The Create user page in Splunk Cloud Platform where you can assign a user to the service account role.
103103

104+
105+
8. Add a Workload Rule in Splunk Cloud Platform to limit Log Observer Connect searches to 5 minutes. This limit maintains a responsive experience for Log Observer users and reduces the chances that Log Observer Connect searches are queued. Follow the guidance in :new-page:`Create a Workload Rule in Splunk Web <https://docs.splunk.com/Documentation/SplunkCloud/9.2.2403/Admin/CreateWLMRules#Create_a_workload_rule_in_Splunk_Web>` and configure the rule as follows:
106+
107+
.. code-block:: none
108+
109+
Predicate: user=[your_Log_Observer_Connect_service-account_name] AND runtime>5m
110+
Schedule: Always on
111+
Action: Abort search
112+
113+
After creating the Workload Rule, it appears in Workload Management on the Workload Rules tab as follows:
114+
115+
.. image:: /_images/logs/WorkloadMgmt.png
116+
:width: 90%
117+
:alt: This screenshot shows the configuration of the Workload Rule limiting Log Observer Connect searches to 5 minutes.
118+
104119
.. _download-certificate:
105120

106-
8. Secure a connection to your Splunk Cloud Platform instance in Splunk Observability Cloud. See :ref:`logs-scp-prereqs` for more information on the IPs to allow.
121+
9. Secure a connection to your Splunk Cloud Platform instance in Splunk Observability Cloud. See :ref:`logs-scp-prereqs` for more information on the IPs to allow.
107122

108123
* To get help from Splunk Support, :ref:`Submit a support ticket <support-ticket>`.
109124

110125
* To do it yourself, add your public IPv4 address to your Splunk Cloud Platform allow list by following instructions in :new-page:`Add subnets to IP allow lists <https://docs.splunk.com/Documentation/SplunkCloud/latest/Admin/ConfigureIPAllowList#Add_subnets_to_IP_allow_lists>`.
111126

112-
9. Go back to the Log Observer Connect guided setup and select :guilabel:`Next`. Enter your service account username, password, and Splunk platform URL ``https://<stackname>.splunkcloud.com:8089`` to complete the guided setup.
127+
10. Go back to the Log Observer Connect guided setup and select :guilabel:`Next`. Enter your service account username, password, and Splunk platform URL ``https://<stackname>.splunkcloud.com:8089`` to complete the guided setup.
113128

114-
10. Remove your IPv4 address from the IP allowlist that you added in step 8. If you are in a GCP environment, do not remove the additional GCP IP addresses that you added in step 8.
129+
11. Remove your IPv4 address from the IP allowlist that you added in step 9. If you are in a GCP environment, do not remove the additional GCP IP addresses that you added in step 8.
115130

116-
11. Make sure to give each connection a unique name on the final page of the Log Observer Connect guided setup.
131+
12. Make sure to give each connection a unique name on the final page of the Log Observer Connect guided setup.
117132

118133
.. note:: Manage concurrent search limits using your current strategy in Splunk Cloud Platform. All searches initiated by Log Observer Connect users go through the service account you create in Splunk Cloud Platform. For each active Log Observer Connect user, four back-end searches occur when a user performs a search in Log Observer Connect. For example, if there are three users accessing Log Observer Connect at the same time, the service account for Log Observer Connect initiates approximately 12 searches in Splunk Cloud Platform.
119134

@@ -122,7 +137,7 @@ In Splunk Cloud Platform, follow the instructions in the guided setup for the in
122137
Submit a support ticket
123138
===================================================================
124139

125-
If you were not able to independently secure a connection to your Splunk Cloud Platform instance in step 8 in the previous section, you may submit a support ticket from your Splunk Cloud Platform instance to do this on your behalf. Submit a ticket to Splunk Support to configure your Splunk Cloud Platform instance's IP allow list. Configuring your allow list properly opens your Splunk Cloud Platform instance management port to Log Observer Connect, which can then search your Splunk Cloud Platform instance log data. After Splunk Support prepares your Splunk Cloud Platform instance, you can securely create a connection to Log Observer Connect.
140+
If you were not able to independently secure a connection to your Splunk Cloud Platform instance in step 9 in the previous section, you may submit a support ticket from your Splunk Cloud Platform instance to do this on your behalf. Submit a ticket to Splunk Support to configure your Splunk Cloud Platform instance's IP allow list. Configuring your allow list properly opens your Splunk Cloud Platform instance management port to Log Observer Connect, which can then search your Splunk Cloud Platform instance log data. After Splunk Support prepares your Splunk Cloud Platform instance, you can securely create a connection to Log Observer Connect.
126141

127142
To submit a support ticket, follow these steps:
128143

logs/set-up-logconnect.rst

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,9 +100,23 @@ In your Splunk Enterprise search head, follow the instructions in the guided set
100100
:width: 100%
101101
:alt: This screenshot shows the Create user page in Splunk Enterprise where you can assign a user to the service account role.
102102

103-
8. Obtain certificates for securing inter-Splunk communication. See :new-page:`Configure and install certificates in Splunk Enterprise for Splunk Log Observer Connect <https://quickdraw.splunk.com/redirect/?product=Observability&location=splunk.integration.third.party&version=current>` to learn how. Copy only the first certificate in the chain and paste it on the next page of the guided setup to securely connect Log Observer Connect and your Splunk Enterprise instance.
103+
8. Add a Workload Rule in Splunk Enterprise to limit Log Observer Connect searches to 5 minutes. This limit maintains a responsive experience for Log Observer users and reduces the chances that Log Observer Connect searches are queued. Follow the guidance in :new-page:`Create a Workload Rule in Splunk Web <https://docs.splunk.com/Documentation/SplunkCloud/9.2.2403/Admin/CreateWLMRules#Create_a_workload_rule_in_Splunk_Web>` and configure the rule as follows:
104104

105-
9. Make sure to give each connection a unique name on the final page of the Log Observer Connect guided setup.
105+
.. code-block:: none
106+
107+
Predicate: user=[your_Log_Observer_Connect_service-account_name] AND runtime>5m
108+
Schedule: Always on
109+
Action: Abort search
110+
111+
After creating the Workload Rule, it appears in Workload Management on the Workload Rules tab as follows:
112+
113+
.. image:: /_images/logs/WorkloadMgmt.png
114+
:width: 90%
115+
:alt: This screenshot shows the configuration of the Workload Rule limiting Log Observer Connect searches to 5 minutes.
116+
117+
9. Obtain certificates for securing inter-Splunk communication. See :new-page:`Configure and install certificates in Splunk Enterprise for Splunk Log Observer Connect <https://quickdraw.splunk.com/redirect/?product=Observability&location=splunk.integration.third.party&version=current>` to learn how. Copy only the first certificate in the chain and paste it on the next page of the guided setup to securely connect Log Observer Connect and your Splunk Enterprise instance.
118+
119+
10. Make sure to give each connection a unique name on the final page of the Log Observer Connect guided setup.
106120

107121
.. note:: Manage concurrent search limits using your current strategy in Splunk Enterprise. All searches initiated by Log Observer Connect users go through the service account you create in Splunk Enterprise. For each active Log Observer Connect user, four back-end searches occur when a user performs a search in the Log Observer Connect UI. For example, if there are three concurrent users accessing the Log Observer Connect UI at the same time, the service account for Log Observer Connect initiates approximately 12 searches in Splunk Enterprise.
108122

0 commit comments

Comments
 (0)