Skip to content
This repository was archived by the owner on Sep 2, 2025. It is now read-only.

Commit 16197f6

Browse files
aurbiztondo-splunkaurbiztondo-splunk
authored
Merge pull request #2460 from splunk/urbiz-OD6504-ec2-details
[6504]: Collector ECS doc improvements
2 parents 91a3628 + b88f0c0 commit 16197f6

File tree

1 file changed

+71
-54
lines changed

1 file changed

+71
-54
lines changed

‎gdi/opentelemetry/deployments/deployments-ecs-ec2.rst‎

Lines changed: 71 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -7,32 +7,28 @@ Deploy the Collector with Amazon ECS EC2
77
.. meta::
88
:description: Deploy the Splunk Observability Cloud OpenTelemetry Collector as a Sidecar in an Amazon ECS EC2 cluster.
99

10-
Use the guided setup to deploy the Collector as a Sidecar in an Amazon ECS EC2 cluster. The guided setup provides a JSON task definition for the Collector.
11-
12-
Choose one of the following Collector configuration options:
13-
14-
- **Default:** The file /etc/otel/collector/ecs_ec2_config.yaml in the Collector image is used for the Collector configuration.
15-
- **File:** Specify the file to use for the Collector configuration. See :ref:`ecs-ec2-custom-config`.
16-
- **AWS Parameter Store:** Specify the AWS Parameter Store key or ARN to use for the Collector configuration. See :ref:`ecs-ec2-custom-config`.
10+
Use the guided setup to deploy the Collector as a sidecar in an Amazon ECS EC2 cluster.
1711

1812
To access the guided setup for AWS integration, perform the following steps:
1913

2014
#. Log in to Splunk Observability Cloud.
2115
#. On the navigation menu, select :guilabel:`Data Management`.
2216
#. Go to the :guilabel:`Available integrations` tab, or select :guilabel:`Add Integration` in the :guilabel:`Deployed integrations` tab.
2317
#. Select the tile for :guilabel:`Amazon ECS EC2`.
24-
#. Follow the steps provided in the guided setup.
18+
#. Follow the steps provided in the guided setup, which provides a JSON task definition for the Collector.
2519

26-
Getting started
27-
=================================
20+
Choose one of the following Collector configuration options:
2821

29-
The following sections describe how to create a task definition and launch the Collector. A task definition is required to run Docker containers in Amazon ECS. After creating the task definition, you need to launch the Collector.
22+
- **Default:** The file /etc/otel/collector/ecs_ec2_config.yaml in the Collector image is used for the Collector configuration.
23+
- **File:** Specify the file to use for the Collector configuration. See :ref:`ecs-ec2-custom-config`.
24+
- **AWS Parameter Store:** Specify the AWS Parameter Store key or ARN to use for the Collector configuration. See :ref:`ecs-ec2-aws-store`.
3025

31-
Add the Collector as a Sidecar
32-
---------------------------------
33-
.. note:: To use this option you need to be familiar with Amazon ECS EC2 launch type. See :new-page:`Getting started with the classic console using Amazon EC2 <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/getting-started-ecs-ec2.html>` for further reading.
26+
Add the Collector as a sidecar
27+
==================================================================
3428

35-
Open the ECS task definition to which the Collector Sidecar is going to be added:
29+
.. note:: To use this option you need to be familiar with Amazon ECS EC2 launch. See :new-page:`Getting started with the classic console using Amazon EC2 <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/getting-started-ecs-ec2.html>` for further reading.
30+
31+
Open the ECS task definition in which you'll add the Collector sidecar:
3632

3733
1. Locate the task definition for the Collector from the :new-page:`repository <https://github.com/signalfx/splunk-otel-collector/blob/main/deployments/ecs/ec2/splunk-otel-collector.json>`.
3834
2. Merge the definitions of the Collector with the existing ECS task definition.
@@ -50,28 +46,65 @@ Notes:
5046

5147
.. _ecs-ec2-custom-config:
5248

53-
Use a custom configuration
54-
==============================
49+
Deploy the Collector using a custom configuration
50+
============================================================
5551

5652
To use a custom configuration file, replace the value of the ``SPLUNK_CONFIG`` environment variable with the file path of the custom configuration file in the Collector task definition.
5753

5854
Alternatively, you can specify the custom configuration YAML directly using the ``SPLUNK_CONFIG_YAML`` environment variable, as described in :ref:`ecs-observer-config`.
5955

6056
.. _ecs-observer-config:
6157

62-
Configure ``ecs_observer``
63-
--------------------------------
58+
Configure the Amazon Elastic Container Service Observer extension
59+
=======================================================================
60+
61+
Use the Amazon Elastic Container Service Observer (``ecs_observer``) extension in your custom configuration to discover metric targets in running tasks, filtered by service names, task definitions, and container labels.
62+
63+
Prerequisites
64+
----------------------------------------------------------------
65+
66+
The following applies:
6467

65-
Use extension Amazon Elastic Container Service Observer (``ecs_observer``) in your custom configuration to discover metrics targets in running tasks, filtered by service names, task definitions, and container labels. ``ecs_observer`` is currently limited to Prometheus targets and requires the read-only permissions below. The Collector should be configured to run as an ECS Daemon. You can add the permissions to the task role by adding them to a customer-managed policy that is attached to the task role.
68+
* The Collector must run as an ECS daemon. See :ref:`ecs-observer-launch`.
69+
* The ECS Observer is currently limited to Prometheus targets.
70+
* The ECS Observer requires the read-only permissions below. Add them to the customer-managed policy that is attached to the task role.
6671

6772
.. code-block:: yaml
6873
69-
ecs:List*
70-
ecs:Describe*
74+
ecs:List*
75+
ecs:Describe*
7176
72-
The following custom configuration examples show the ``ecs_observer`` configured to find Prometheus targets in the ``lorem-ipsum-cluster`` cluster and ``us-west-2`` region, where the task ARN pattern is ``^arn:aws:ecs:us-west-2:906383545488:task-definition/lorem-ipsum-task:[0-9]+$``.
77+
.. _ecs-observer-launch:
78+
79+
Launch the Collector as a daemon from the ECS console
80+
----------------------------------------------------------------
81+
82+
To launch the Collector from the Amazon ECS console:
7383

74-
The results are written to ``/etc/ecs_sd_targets.yaml``. The ``prometheus`` receiver is configured to read targets from the results file. The values for ``access_token`` and ``realm`` are read from the ``SPLUNK_ACCESS_TOKEN`` and ``SPLUNK_REALM`` environment variables , which must be specified in your container definition.
84+
1. Go to your cluster in the console and select :guilabel:`Services`.
85+
86+
2. Select :guilabel:`Create` and define the following options:
87+
88+
* Launch Type: EC2
89+
90+
* Task Definition (Family): splunk-otel-collector
91+
92+
* Task Definition (Revision): 1 (or whatever the latest is in your case)
93+
94+
* Service Name: splunk-otel-collector
95+
96+
* Service type: DAEMON
97+
98+
3. Leave everything else as default and proceed to :guilabel:`Next step` until you're required to create the service.
99+
100+
4. Select :guilabel:`Create Service` to deploy the Collector onto each node in the ECS cluster. You should see infrastructure and docker metrics flowing soon.
101+
102+
Configuration example
103+
----------------------------------------------------------------
104+
105+
The following example configures the ``ecs_observer`` to find Prometheus targets in the ``lorem-ipsum-cluster`` cluster and ``us-west-2`` region, where the task ARN pattern is ``^arn:aws:ecs:us-west-2:906383545488:task-definition/lorem-ipsum-task:[0-9]+$``. The results are written to ``/etc/ecs_sd_targets.yaml``.
106+
107+
The ``prometheus`` receiver is configured to read targets from the results file. The values for ``access_token`` and ``realm`` are read from the ``SPLUNK_ACCESS_TOKEN`` and ``SPLUNK_REALM`` environment variables, which you must specify in your container definition.
75108

76109
.. code-block:: yaml
77110
@@ -111,42 +144,26 @@ The results are written to ``/etc/ecs_sd_targets.yaml``. The ``prometheus`` rece
111144
processors: [batch, resourcedetection]
112145
exporters: [signalfx]
113146
114-
.. _aws-parameter-store:
147+
.. _ecs-ec2-aws-store:
115148

116-
Launch the Collector as a Daemon
117-
--------------------------------------------
149+
Deploy the Collector using the AWS Parameter Store
150+
=========================================================================================
118151

119-
To launch the Collector from the Amazon ECS console:
120-
121-
#. Go to your cluster in the console.
122-
#. Select :guilabel:`Services`.
123-
#. Select :guilabel:`Create`.
124-
#. Select the following options:
125-
#. Launch Type: EC2
126-
#. Task Definition (Family): splunk-otel-collector
127-
#. Task Definition (Revision): 1 (or whatever the latest is in your case)
128-
#. Service Name: splunk-otel-collector
129-
#. Service type: DAEMON
130-
#. Leave everything else at default.
131-
#. Select :guilabel:`Next step`.
132-
#. Leave everything on this next page at their defaults and select :guilabel:`Next step`.
133-
#. Leave everything on this next page at their defaults and select :guilabel:`Next step`.
134-
#. Select :guilabel:`Create Service` to deploy the Collector onto each node in the ECS cluster. You should see infrastructure and docker metrics flowing soon.
152+
To use the AWS Parameter Store, specify the config YAML directly in the ``SPLUNK_CONFIG_YAML`` environment variable instead of using ``SPLUNK_CONFIG``.
135153

136-
Use the AWS Parameter Store
137-
----------------------------
154+
.. note:: To have read access to the Parameter Store add the policy ``AmazonSSMReadOnlyAccess`` to the task role. See :new-page:`Systems manager parameter store <https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html>` for more information.
138155

139-
Use the ``SPLUNK_CONFIG_YAML`` environment variable to specify the configuration YAML directly. Use ``SPLUNK_CONFIG_YAML`` in place of ``SPLUNK_CONFIG``.
156+
Follow these steps:
140157

141-
For example, first, store the custom configuration for the :ref:`ecs-observer-config` in a parameter called ``splunk-otel-collector-config`` in the AWS Systems Manager Parameter Store.Next, assign the parameter to ``SPLUNK_CONFIG_YAML`` using the ``valueFrom`` option, as shown in the following example:
158+
#. Store the custom configuration for the :ref:`ecs-observer-config` in the parameter ``splunk-otel-collector-config`` in the AWS Systems Manager Parameter Store.
159+
#. Next, assign the ``splunk-otel-collector-config`` parameter to ``SPLUNK_CONFIG_YAML`` using the ``valueFrom`` option, as shown in the following example:
142160

143161
.. code-block:: none
144162
145-
{
146-
"name": "lorem-ipsum-cluster",
147-
"valueFrom": "^arn:aws:ecs:us-west-2:906383545488:task-definition/lorem-ipsum-task:[0-9]+$""
148-
}
163+
{
164+
"name": "lorem-ipsum-cluster",
165+
"valueFrom": "^arn:aws:ecs:us-west-2:906383545488:task-definition/lorem-ipsum-task:[0-9]+$""
166+
}
167+
168+
.. caution:: The AWS Parameter Store limits the size of the config file to 4096 bytes.
149169

150-
.. note::
151-
152-
You should add policy ``AmazonSSMReadOnlyAccess`` to the task role for the task to have read access to the Parameter Store. See :new-page:`Systems manager parameter store <https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html>` for more information.

0 commit comments

Comments
 (0)