Skip to content

Commit b9689cb

Browse files
JSticklerJStickler
authored
Merge pull request #48001 from libander/RHDEVDOCS-3551
RHDEVDOCS-3551 - Cloudwatch Forwarding with STS
2 parents 7c6fb0f + 95443ba commit b9689cb

File tree

2 files changed

+135
-1
lines changed

2 files changed

+135
-1
lines changed

‎logging/cluster-logging-external.adoc

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,140 @@ include::modules/cluster-logging-collector-log-forward-syslog.adoc[leveloffset=+
4343

4444
include::modules/cluster-logging-collector-log-forward-cloudwatch.adoc[leveloffset=+1]
4545

46+
=== Forwarding logs to Amazon CloudWatch from STS enabled clusters
47+
48+
For clusters with AWS Security Token Service (STS) enabled, you can create an AWS service account manually or create a credentials request using the xref:../authentication/managing_cloud_provider_credentials/about-cloud-credential-operator.adoc[Cloud Credential Operator(CCO)] utility `ccoctl`.
49+
50+
.Prerequisites
51+
52+
* {logging-title-uc}: 5.5 and later
53+
54+
[NOTE]
55+
====
56+
This feature is not supported by the vector collector.
57+
====
58+
59+
.Creating an AWS credentials request
60+
. Create a `CredentialsRequest` Custom Resource YAML using the template below:
61+
+
62+
.CloudWatch Credentials Request Template
63+
[source,yaml]
64+
----
65+
apiVersion: cloudcredential.openshift.io/v1
66+
kind: CredentialsRequest
67+
metadata:
68+
name: <your_role_name>-credrequest
69+
namespace: openshift-cloud-credential-operator
70+
spec:
71+
providerSpec:
72+
apiVersion: cloudcredential.openshift.io/v1
73+
kind: AWSProviderSpec
74+
statementEntries:
75+
- action:
76+
- logs:PutLogEvents
77+
- logs:CreateLogGroup
78+
- logs:PutRetentionPolicy
79+
- logs:CreateLogStream
80+
- logs:DescribeLogGroups
81+
- logs:DescribeLogStreams
82+
effect: Allow
83+
resource: arn:aws:logs:*:*:*
84+
secretRef:
85+
name: <your_role_name>
86+
namespace: openshift-logging
87+
serviceAccountNames:
88+
- logcollector
89+
----
90+
+
91+
. Use the `ccoctl` command to to create a role for AWS using your `CredentialsRequest` CR. With the `CredentialsRequest` object, this `ccoctl` command creates an IAM role with a trust policy that is tied to the specified OIDC identity provider, and a permissions policy that grants permissions to perform operations on CloudWatch resources. This command also creates a YAML configuration file in ``/<path_to_ccoctl_output_dir>/manifests/openshift-logging-<your_role_name>-credentials.yaml`. This secret file contains the `role_arn` key/value used during authentication with the AWS IAM identity provider.
92+
+
93+
[source,terminal]
94+
----
95+
ccoctl aws create-iam-roles \
96+
--name=<name> \
97+
--region=<aws_region> \
98+
--credentials-requests-dir=<path_to_directory_with_list_of_credentials_requests>/credrequests \
99+
--identity-provider-arn=arn:aws:iam::<aws_account_id>:oidc-provider/<name>-oidc.s3.<aws_region>.amazonaws.com <1>
100+
----
101+
<1> <name> is the name used to tag your cloud resources and should match the name used during your STS cluster install
102+
+
103+
. Apply the secret created:
104+
[source,terminal]
105+
+
106+
----
107+
oc apply -f output/manifests/openshift-logging-<your_role_name>-credentials.yaml
108+
----
109+
+
110+
. Create or edit a `ClusterLogForwarder` custom resource:
111+
+
112+
[source,yaml]
113+
----
114+
apiVersion: "logging.openshift.io/v1"
115+
kind: ClusterLogForwarder
116+
metadata:
117+
name: instance <1>
118+
namespace: openshift-logging <2>
119+
spec:
120+
outputs:
121+
- name: cw <3>
122+
type: cloudwatch <4>
123+
cloudwatch:
124+
groupBy: logType <5>
125+
groupPrefix: <group prefix> <6>
126+
region: us-east-2 <7>
127+
secret:
128+
name: <your_role_name> <8>
129+
pipelines:
130+
- name: to-cloudwatch <9>
131+
inputRefs: <10>
132+
- infrastructure
133+
- audit
134+
- application
135+
outputRefs:
136+
- cw <11>
137+
----
138+
<1> The name of the `ClusterLogForwarder` CR must be `instance`.
139+
<2> The namespace for the `ClusterLogForwarder` CR must be `openshift-logging`.
140+
<3> Specify a name for the output.
141+
<4> Specify the `cloudwatch` type.
142+
<5> Optional: Specify how to group the logs:
143+
+
144+
* `logType` creates log groups for each log type
145+
* `namespaceName` creates a log group for each application name space. Infrastructure and audit logs are unaffected, remaining grouped by `logType`.
146+
* `namespaceUUID` creates a new log groups for each application namespace UUID. It also creates separate log groups for infrastructure and audit logs.
147+
<6> Optional: Specify a string to replace the default `infrastructureName` prefix in the names of the log groups.
148+
<7> Specify the AWS region.
149+
<8> Specify the name of the secret that contains your AWS credentials.
150+
<9> Optional: Specify a name for the pipeline.
151+
<10> Specify which log types to forward by using the pipeline: `application,` `infrastructure`, or `audit`.
152+
<11> Specify the name of the output to use when forwarding logs with this pipeline.
153+
154+
155+
[role="_additional-resources"]
156+
.Additional resources
157+
* link:https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html[AWS STS API Reference]
158+
159+
160+
==== Creating a secret for AWS CloudWatch with an existing AWS role
161+
If you have an existing role for AWS, you can create a secret for AWS with STS using the `oc create secret --from-literal` command.
162+
163+
[source,terminal]
164+
----
165+
oc create secret generic cw-sts-secret -n openshift-logging --from-literal=role_arn=arn:aws:iam::123456789012:role/my-role_with-permissions
166+
----
167+
168+
.Example Secret
169+
[source,yaml]
170+
----
171+
apiVersion: v1
172+
kind: Secret
173+
metadata:
174+
namespace: openshift-logging
175+
name: my-secret-name
176+
stringData:
177+
role_arn: arn:aws:iam::123456789012:role/my-role_with-permissions
178+
----
179+
46180
include::modules/cluster-logging-collector-log-forward-loki.adoc[leveloffset=+1]
47181

48182
include::modules/cluster-logging-troubleshooting-loki-entry-out-of-order-errors.adoc[leveloffset=+2]

‎modules/cluster-logging-collector-log-forward-cloudwatch.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
[id="cluster-logging-collector-log-forward-cloudwatch_{context}"]
33
= Forwarding logs to Amazon CloudWatch
44

5-
You can forward logs to Amazon CloudWatch, a monitoring and log storage service hosted by Amazon Web Services (AWS). You can forward logs to CloudWatch in addition to, or instead of, the default {logging} managed Elasticsearch log store.
5+
You can forward logs to Amazon CloudWatch, a monitoring and log storage service hosted by Amazon Web Services (AWS). You can forward logs to CloudWatch in addition to, or instead of, the default log store.
66

77
To configure log forwarding to CloudWatch, you must create a `ClusterLogForwarder` custom resource (CR) with an output for CloudWatch, and a pipeline that uses the output.
88

0 commit comments

Comments
 (0)