Merge pull request splunk#1607 from splunk/repo-sync

aurbiztondo-splunk · web-flow · commit 1e5bd6329bc2 · 2024-11-20T13:52:10.000+01:00
Pulling refs/heads/main into main
diff --git a/_images/gdi/aws-ts-awsmanaged1.png b/_images/gdi/aws-ts-awsmanaged1.png
diff --git a/_images/gdi/aws-ts-awsmanaged2.png b/_images/gdi/aws-ts-awsmanaged2.png
diff --git a/_images/gdi/aws-ts-awsmanaged3.png b/_images/gdi/aws-ts-awsmanaged3.png
diff --git a/_images/gdi/aws-ts-awsmanaged4.png b/_images/gdi/aws-ts-awsmanaged4.png
diff --git a/gdi/get-data-in/connect/aws/aws-troubleshooting.rst b/gdi/get-data-in/connect/aws/aws-troubleshooting.rst
@@ -7,7 +7,12 @@ Troubleshoot your AWS integration
 .. meta::
    :description: Resolve AWS policy, permission conflicts, and other issues in Splunk Observability Cloud.
 
-If you experience difficulties when connecting Splunk Observability Cloud to your Amazon Web Services (AWS) account or using the platform, read on to troubleshoot common issues. See :ref:`aws-ts-metric-streams` for issues specific to Metric Streams. 
+If you experience difficulties when connecting Splunk Observability Cloud to your Amazon Web Services (AWS) account or using the platform, read on to troubleshoot common issues. 
+
+See also the following docs:
+
+* :ref:`aws-ts-metric-streams` for issues specific to Splunk-managed Metric Streams. 
+* :ref:`aws-ts-ms-aws` for issues specific to AWS-managed Metric Streams.
 
 If issues persist, you can also contact :ref:`support`.  
 
diff --git a/gdi/get-data-in/connect/aws/aws-ts-metric-streams.rst b/gdi/get-data-in/connect/aws/aws-ts-metric-streams.rst
@@ -1,17 +1,15 @@
 .. _aws-ts-metric-streams:
 
 ******************************************************
-Troubleshoot AWS Metric Streams
+Troubleshoot Splunk-managed Metric Streams
 ******************************************************
 
 .. meta::
   :description: Troubleshoot AWS Metric Streams related issues.
 
 See the following topics when experiencing AWS Metric Streams related issues.
 
-.. note::
-
-  See also :ref:`aws-troubleshooting`.
+.. note:: See also :ref:`aws-troubleshooting` and :ref:`aws-ts-ms-aws`.
 
 I've enabled Splunk-managed Metric Streams in my AWS integration but I do not see any metrics streaming in
 ==========================================================================================================
diff --git a/gdi/get-data-in/connect/aws/aws-ts-ms-aws.rst b/gdi/get-data-in/connect/aws/aws-ts-ms-aws.rst
@@ -0,0 +1,113 @@
+.. _aws-ts-ms-aws:
+
+******************************************************
+Troubleshoot AWS-managed Metric Streams
+******************************************************
+
+.. meta::
+  :description: Troubleshoot AWS-managed Metric Streams related issues.
+
+See the following topics when experiencing AWS-managed Metric Streams related issues.
+
+.. note:: See also :ref:`aws-troubleshooting` and :ref:`aws-ts-metric-streams`.
+
+I've enabled AWS-managed Metric Streams in my AWS integration but I do not see any metrics streaming in
+==========================================================================================================
+
+Log in to your AWS console and review the following:
+
+Check the Metric Stream integration settings 
+----------------------------------------------------------------------
+
+In your CloudWatch console check the following:
+
+* The :strong:`Errors` chart must show zero errors.
+
+* The destination and output format must look similar to what is visible on the screenshot below. Note that your destination URL might use a different realm, for example https://ingest.us0.signalfx.com/v1/cloudwatch_metric_stream.
+
+.. image:: /_images/gdi/aws-ts-awsmanaged1.png
+    :width: 70%
+    :alt: Metric Stream settings
+
+Check the associated Firehose stream configuration
+----------------------------------------------------------------------
+
+Go to your AWS console and check your Firehose stream configuration:
+
+.. image:: /_images/gdi/aws-ts-awsmanaged2.png
+      :width: 70%
+      :alt: Firehose configuration
+
+Check the associated Firehose stream metrics 
+----------------------------------------------------------------------
+
+Make sure that:
+
+* :strong:`Incoming records` must have non-zero values. 
+
+  * Note that if there's a low volume of traffic going through the stream, the value of ``IncomingRecords (per second average)`` can be significantly lower than ``RecordsPerSecondLimit``, and therefore it might look like there are no incoming records. 
+
+* :strong:`HTTP endpoint delivery success` must be 100%. 
+
+.. image:: /_images/gdi/aws-ts-awsmanaged3.png
+      :width: 70%
+      :alt: Firehose stream metrics
+
+If there are any errors visible on the HTTP endpoint delivery success chart, update the Firehose stream destination settings to ensure that the HTTP endpoint is correct and use a new access token with INGEST scope.  
+
+  * Note that your destination URL might use a different realm, for example https://ingest.us0.signalfx.com/v1/cloudwatch_metric_stream.
+
+.. image:: /_images/gdi/aws-ts-awsmanaged4.png
+      :width: 70%
+      :alt: Destination settings
+
+Review roles and permissions 
+-----------------------------------
+
+Check the following:
+
+* Review the AWS role used by the Metric Streams. 
+
+* Make sure all required permissions are included, and that the CloudWatch metric stream region and the region listed in the AWS policy are the same.
+
+  * To find the AWS role used by the Metric Stream, go to the Stream overview page under the :strong:`Service Role to write to Amazon Data Firehose` label.
+
+This is a sample AWS policy:
+
+.. code-block:: none
+
+  {
+      "Version": "2012-10-17",
+      "Statement": [
+          {
+              "Effect": "Allow",
+              "Action": [
+                  "firehose:PutRecord",
+                  "firehose:PutRecordBatch"
+              ],
+              "Resource": [
+                  "arn:aws:firehose:eu-west-2:906383545488:deliverystream/PUT-HTP-7pH7O"
+              ]
+          }
+      ]
+  }
+
+These are the trust relationships:
+
+.. code-block:: none
+
+  {
+      "Version": "2012-10-17",
+      "Statement": [
+          {
+              "Effect": "Allow",
+              "Principal": {
+                  "Service": "streams.metrics.cloudwatch.amazonaws.com"
+              },
+              "Action": "sts:AssumeRole"
+          }
+      ]
+  }
+
+
+
diff --git a/gdi/get-data-in/connect/aws/get-awstoc.rst b/gdi/get-data-in/connect/aws/get-awstoc.rst
@@ -23,12 +23,13 @@ Connect AWS to Splunk Observability Cloud
   Send AWS logs to Splunk Platform <aws-logs>
   Next steps <aws-post-install>
   Troubleshoot your AWS integration <aws-troubleshooting>
-  Troubleshoot Metric Streams <aws-ts-metric-streams>
+  Troubleshoot Splunk-managed Metric Streams <aws-ts-metric-streams>
+  Troubleshoot AWS-managed Metric Streams <aws-ts-ms-aws>
   aws-recommended-stats.rst
   GetMetricStatistics API deprecation notice <aws-api-notice>
   aws-tutorial/about-aws-tutorial.rst
 
-You have several data ingestion and connection methods when it comes to monitoring your Amazon Web Services (AWS) data in Splunk Observability Cloud. 
+Splunk Observability Cloud offers you several data ingestion and connection methods to monitor your Amazon Web Services (AWS) data. 
 
 .. note:: If you want to send AWS data to the Splunk platform, use the Splunk add-on. Learn more at :new-page:`Splunk Add-on for AWS <https://docs.splunk.com/Documentation/AddOns/released/AWS/Description>`.
 
diff --git a/gdi/opentelemetry/collector-kubernetes/kubernetes-config-logs.rst b/gdi/opentelemetry/collector-kubernetes/kubernetes-config-logs.rst
@@ -2,7 +2,7 @@
 .. _kubernetes-config-logs:
 
 *********************************************************************************
-Collect logs and events for the Collector for Kubernetes
+Collect logs and events with the Collector for Kubernetes
 *********************************************************************************
 
 .. meta::
diff --git a/gdi/opentelemetry/collector-linux/linux-config-logs.rst b/gdi/opentelemetry/collector-linux/linux-config-logs.rst
@@ -2,7 +2,7 @@
 .. _linux-config-logs:
 
 ***************************************************************
-Collect logs for the Collector for Linux
+Collect logs with the Collector for Linux
 ***************************************************************
 
 .. meta::
diff --git a/gdi/opentelemetry/collector-windows/collector-windows-intro.rst b/gdi/opentelemetry/collector-windows/collector-windows-intro.rst
@@ -17,6 +17,7 @@ Get started with the Collector for Windows
    Install the Collector for Windows (manually) <install-windows-manual.rst>       
    windows-config-ootb.rst
    windows-config.rst
+   windows-config-logs.rst
    metrics-ootb-windows.rst   
    windows-upgrade.rst   
    windows-uninstall.rst
@@ -31,8 +32,9 @@ To install the Splunk Distribution of the OpenTelemetry Collector for Windows, f
 See the default settings and configuration options at:
 
 * :ref:`windows-config-ootb`
-* :ref:`otel-windows-config`
 * By default, you'll obtain these :ref:`metrics <ootb-metrics-windows>` 
+* :ref:`otel-windows-config`
+* :ref:`windows-config-logs`
 
 .. include:: /_includes/gdi/collector-common-options.rst
 
diff --git a/gdi/opentelemetry/collector-windows/install-windows.rst b/gdi/opentelemetry/collector-windows/install-windows.rst
@@ -167,55 +167,6 @@ The Windows installer script supports the following options:
      - Specify public MSI properties to be used when installing the Splunk OpenTelemetry Collector MSI package.
      -
 
-.. _windows-config-logs:
-
-Collect logs for the Collector for Windows
-====================================================================
-
-Use the Universal Forwarder to send logs to the Splunk platform. See more at :ref:`collector-with-the-uf`.
-
-.. _fluentd-manual-config-windows:
-
-Collect Windows logs with Fluentd
----------------------------------------
-
-If you wish to collect logs for the target host with Fluentd, use the ``with_fluentd = 1`` option to install and enable Fluentd when installing the Collector. For example:
-
-.. code-block:: PowerShell
-
-  & {Set-ExecutionPolicy Bypass -Scope Process -Force; $script = ((New-Object System.Net.WebClient).DownloadString('https://dl.signalfx.com/splunk-otel-collector.ps1')); $params = @{access_token = "<SPLUNK_ACCESS_TOKEN>"; realm = "<SPLUNK_REALM>"; with_fluentd = 1}; Invoke-Command -ScriptBlock ([scriptblock]::Create(". {$script} $(&{$args} @params)"))}
-
-When activated, the Fluentd service is configured by default to collect and forward log events with the ``@SPLUNK`` label to the Collector, which then send these events to the HEC ingest endpoint determined by the ``realm = "<SPLUNK_REALM>"`` option.
-For example, ``https://ingest.<SPLUNK_REALM>.signalfx.com/v1/log``.
-
-To configure the package to send log events to a custom HTTP Event Collector (HEC) endpoint URL with a token different than ``<SPLUNK_ACCESS_TOKEN>``, you can specify the following parameters for the installer script:
-
-* ``hec_url = "<SPLUNK_HEC_URL>"``
-* ``hec_token = "<SPLUNK_HEC_TOKEN>"``
-
-For example (replace the ``<SPLUNK...>`` values in the command for your configuration):
-
-.. code-block:: PowerShell
-
-  & {Set-ExecutionPolicy Bypass -Scope Process -Force; $script = ((New-Object System.Net.WebClient).DownloadString('https://dl.signalfx.com/splunk-otel-collector.ps1')); $params = @{access_token = "<SPLUNK_ACCESS_TOKEN>"; realm = "<SPLUNK_REALM>"; hec_url = "<SPLUNK_HEC_URL>"; hec_token = "<SPLUNK_HEC_TOKEN>"}; Invoke-Command -ScriptBlock ([scriptblock]::Create(". {$script} $(&{$args} @params)"))}
-
-The installation creates the main fluentd configuration file  ``<drive>\opt\td-agent\etc\td-agent\td-agent.conf``, where ``<drive>`` is the drive letter for the fluentd installation directory.
-
-You can add custom fluentd source configuration files to the ``<drive>\opt\td-agent\etc\td-agent\conf.d``
-directory after installation.
-
-Note the following:
-
-* In this directory, fluentd includes all files with the .conf extension.
-* By default, fluentd collects from the Windows Event Log. See ``<drive>\opt\td-agent\etc\td-agent\conf.d\eventlog.conf`` for the default configuration.
-
-After any configuration modification, apply the changes by restarting the system or running the following PowerShell commands:
-
-.. code-block:: PowerShell
-
-  Stop-Service fluentdwinsvc
-  Start-Service fluentdwinsvc
-
 Next steps
 ==================================
 
diff --git a/gdi/opentelemetry/collector-windows/windows-config-logs.rst b/gdi/opentelemetry/collector-windows/windows-config-logs.rst
@@ -0,0 +1,59 @@
+.. _windows-config-logs:
+
+***************************************************************
+Collect logs with the Collector for Windows
+***************************************************************
+
+.. meta::
+  
+  :description: Describes how to collect logs for the Splunk Distribution of OpenTelemetry Collector for Linux.
+
+Use the Universal Forwarder to send logs to the Splunk platform. See more at :ref:`collector-with-the-uf`.
+
+.. _fluentd-manual-config-windows:
+
+Collect Windows logs with Fluentd
+===========================================================================
+
+Fluentd is turned off by default. 
+
+If you wish to collect logs for the target host with Fluentd, use the ``with_fluentd = 1`` option to install and enable Fluentd when installing the Collector. 
+
+For example:
+
+.. code-block:: PowerShell
+
+  & {Set-ExecutionPolicy Bypass -Scope Process -Force; $script = ((New-Object System.Net.WebClient).DownloadString('https://dl.signalfx.com/splunk-otel-collector.ps1')); $params = @{access_token = "<SPLUNK_ACCESS_TOKEN>"; realm = "<SPLUNK_REALM>"; with_fluentd = 1}; Invoke-Command -ScriptBlock ([scriptblock]::Create(". {$script} $(&{$args} @params)"))}
+
+When activated, the Fluentd service is configured by default to collect and forward log events with the ``@SPLUNK`` label to the Collector, which then send these events to the HEC ingest endpoint determined by the ``realm = "<SPLUNK_REALM>"`` option.
+For example, ``https://ingest.<SPLUNK_REALM>.signalfx.com/v1/log``.
+
+To configure the package to send log events to a custom HTTP Event Collector (HEC) endpoint URL with a token different than ``<SPLUNK_ACCESS_TOKEN>``, you can specify the following parameters for the installer script:
+
+* ``hec_url = "<SPLUNK_HEC_URL>"``
+* ``hec_token = "<SPLUNK_HEC_TOKEN>"``
+
+For example (replace the ``<SPLUNK...>`` values in the command for your configuration):
+
+.. code-block:: PowerShell
+
+  & {Set-ExecutionPolicy Bypass -Scope Process -Force; $script = ((New-Object System.Net.WebClient).DownloadString('https://dl.signalfx.com/splunk-otel-collector.ps1')); $params = @{access_token = "<SPLUNK_ACCESS_TOKEN>"; realm = "<SPLUNK_REALM>"; hec_url = "<SPLUNK_HEC_URL>"; hec_token = "<SPLUNK_HEC_TOKEN>"}; Invoke-Command -ScriptBlock ([scriptblock]::Create(". {$script} $(&{$args} @params)"))}
+
+The installation creates the main Fluentd configuration file  ``<drive>\opt\td-agent\etc\td-agent\td-agent.conf``, where ``<drive>`` is the drive letter for the fluentd installation directory.
+
+You can add custom Fluentd source configuration files to the ``<drive>\opt\td-agent\etc\td-agent\conf.d``
+directory after installation.
+
+Note the following:
+
+* In this directory, Fluentd includes all files with the .conf extension.
+* By default, fluentd collects from the Windows Event Log. See ``<drive>\opt\td-agent\etc\td-agent\conf.d\eventlog.conf`` for the default configuration.
+
+After any configuration modification, apply the changes by restarting the system or running the following PowerShell commands:
+
+.. code-block:: PowerShell
+
+  Stop-Service fluentdwinsvc
+  Start-Service fluentdwinsvc
+
+
diff --git a/gdi/opentelemetry/troubleshoot-logs.rst b/gdi/opentelemetry/troubleshoot-logs.rst
@@ -9,6 +9,14 @@ Troubleshoot log collection
 
 This document describes common issues related to log collection with the Collector.
 
+.. note:: 
+  
+  To collect logs see:
+
+  * :ref:`kubernetes-config-logs`
+  * :ref:`linux-config-logs`
+  * :ref:`windows-config-logs`
+  
 To troubleshoot the health and performance of the Collector see the :new-page:`OpenTelemetry Project troublehooting docs <https://opentelemetry.io/docs/collector/troubleshooting>`. It includes information about troubleshooting tools and debugging.
 
 My source isn't generating logs
@@ -69,9 +77,7 @@ You can manually generate logs. By default, Fluentd monitors journald and /var/l
   echo "2021-03-17 02:14:44 +0000 [debug]: test" >>/var/log/syslog.log
   echo "2021-03-17 02:14:44 +0000 [debug]: test" | systemd-cat
 
-.. note::
-
-  Properly structured syslog is required for Fluentd to properly pick up the log line.
+.. caution:: Fluentd requires properly structured syslog to pick up the log line.
 
 .. _unwanted_profiling_logs:
 
diff --git a/gdi/requirements.rst b/gdi/requirements.rst
@@ -17,8 +17,16 @@ The Splunk Distribution of OpenTelemetry Collector supports the following operat
 Linux
 ------------------------------------------------------------
 
+.. raw:: html
+
+   <div class="include-start" id="requirements/collector-linux.rst"></div>
+
 .. include:: /_includes/requirements/collector-linux.rst
 
+.. raw:: html
+
+   <div class="include-stop" id="requirements/collector-linux.rst"></div>
+
 See :ref:`get-started-linux` for more information.
 
 Windows
