Skip to content

Commit ec3ef0b

Browse files
embano1embano1
authored
Add Eventbridge Tutorial (#1745)
Issue #, if available: Description of changes: Add `Eventbridge` tutorial and minor fixes in `Pipes` tutorial. By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
1 parent aca4f01 commit ec3ef0b

File tree

2 files changed

+331
-3
lines changed

2 files changed

+331
-3
lines changed

‎docs/content/docs/tutorials/eventbridge-example.md

Lines changed: 308 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,308 @@
1+
---
2+
title: "Manage EventBridge event buses and rules with the ACK EventBridge Controller"
3+
description: "Send filtered events on a custom bus to SQS."
4+
lead: "Create and manage EventBridge event buses and rules directly from Kubernetes"
5+
draft: false
6+
menu:
7+
docs:
8+
parent: "tutorials"
9+
weight: 45
10+
toc: true
11+
---
12+
13+
EventBridge is a serverless service that uses events to connect application components together, making it easier for
14+
you to build scalable event-driven applications. Use it to route events from sources such as home-grown applications,
15+
AWS services, and third-party software to consumer applications across your organization. EventBridge provides a simple
16+
and consistent way to ingest, filter, transform, and deliver events so you can build new applications quickly.
17+
18+
In this tutorial you will learn how to create and manage a custom EventBridge [event
19+
bus](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-bus.html) and
20+
[rule](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-rules.html) to filter and forward messages to an SQS
21+
[target](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-targets.html) from an Amazon Elastic Kubernetes
22+
(EKS) deployment.
23+
24+
## Setup
25+
26+
Although it is not necessary to use Amazon Elastic Kubernetes Service (Amazon EKS) with ACK, this guide assumes that you
27+
have access to an Amazon EKS cluster. If this is your first time creating an Amazon EKS cluster, see [Amazon EKS
28+
Setup](https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-eks-setup.html).
29+
For automated cluster creation using `eksctl`, see [Getting started with Amazon EKS -
30+
`eksctl`](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html) and create your cluster with
31+
Amazon EC2 Linux managed nodes.
32+
33+
### Prerequisites
34+
35+
This guide assumes that you have:
36+
37+
- Created an EKS cluster with Kubernetes version 1.24 or higher.
38+
- AWS IAM permissions to create roles and attach policies to roles.
39+
- AWS IAM permissions to manages queues and send messages to a queue.
40+
- Installed the following tools on the client machine used to access your Kubernetes cluster:
41+
- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv1.html) - A command line tool for interacting
42+
with AWS services.
43+
- [kubectl](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html) - A command line tool for working
44+
with Kubernetes clusters.
45+
- [eksctl](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html) - A command line tool for working with EKS
46+
clusters.
47+
- [Helm 3.8+](https://helm.sh/docs/intro/install/) - A tool for installing and managing Kubernetes applications.
48+
- [jq](https://stedolan.github.io/jq/download/) to parse AWS CLI JSON output
49+
50+
### Install the ACK service controller for EventBridge
51+
52+
Log into the Helm registry that stores the ACK charts:
53+
```bash
54+
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
55+
```
56+
57+
Deploy the ACK service controller for Amazon EventBridge using the [eventbridge-chart Helm chart](https://gallery.ecr.aws/aws-controllers-k8s/eventbridge-chart). Resources should be created in the `us-east-1` region:
58+
59+
```bash
60+
helm install --create-namespace -n ack-system oci://public.ecr.aws/aws-controllers-k8s/eventbridge-chart --version=v1.0.0 --generate-name --set=aws.region=us-east-1
61+
```
62+
63+
For a full list of available values to the Helm chart, please [review the values.yaml file](https://github.com/aws-controllers-k8s/eventbridge-controller/blob/main/helm/values.yaml).
64+
65+
### Configure IAM permissions
66+
67+
Once the service controller is deployed, you will need to [configure the IAM permissions][irsa-permissions] for the
68+
controller to query the EventBridge API. For full details, please review the AWS Controllers for Kubernetes documentation for
69+
[how to configure the IAM permissions][irsa-permissions]. If you follow the examples in the documentation, use the value
70+
of `eventbridge` for `SERVICE`.
71+
72+
## Create an EventBridge Custom Event Bus and Rule with an SQS Target
73+
74+
### Create the target SQS queue
75+
76+
To keep the scope of this tutorial simple, the SQS queue and IAM permissions will be created with the AWS CLI.
77+
Alternatively, the [ACK SQS
78+
Controller](https://aws-controllers-k8s.github.io/community/docs/community/services/#amazon-sqs) and [ACK IAM
79+
Controller](https://aws-controllers-k8s.github.io/community/docs/community/services/#amazon-iam) can be used to manage
80+
these resources with Kubernetes.
81+
82+
Execute the following command to define the environment variables used throughout the example.
83+
84+
{{% hint type="info" title="Make sure environment variables are set" %}}
85+
If you followed the steps in the IAM permissions section above, the required environment variables `${AWS_REGION}` and
86+
`${AWS_ACCOUNT_ID}` are already set. Otherwise please set these variables before executing the following steps. The value for `${AWS_REGION}` must also match the `--set=aws.region` value used in the `helm install` command above.
87+
{{% /hint %}}
88+
89+
```bash
90+
export EVENTBRIDGE_NAMESPACE=eventbridge-example
91+
export EVENTBUS_NAME=custom-eventbus-ack
92+
export RULE_NAME=custom-eventbus-ack-sqs-rule
93+
export TARGET_QUEUE=custom-eventbus-ack-rule-sqs-target
94+
```
95+
96+
Create the target queue.
97+
98+
```bash
99+
cat <<EOF > target-queue.json
100+
{
101+
"QueueName": "${TARGET_QUEUE}",
102+
"Attributes": {
103+
"Policy": "{\"Statement\":[{\"Sid\":\"EventBridgeToSqs\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"events.amazonaws.com\"},\"Action\":[\"sqs:SendMessage\"],\"Resource\":\"arn:aws:sqs:${AWS_REGION}:${AWS_ACCOUNT_ID}:${TARGET_QUEUE}\",\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"arn:aws:events:${AWS_REGION}:${AWS_ACCOUNT_ID}:rule/${EVENTBUS_NAME}/${RULE_NAME}\"}}}]}"
104+
}
105+
}
106+
EOF
107+
108+
aws sqs create-queue --cli-input-json file://target-queue.json
109+
```
110+
111+
The output of above commands looks like
112+
113+
```bash
114+
{
115+
"QueueUrl": "https://sqs.us-east-1.amazonaws.com/1234567890/custom-eventbus-ack-rule-sqs-target"
116+
}
117+
```
118+
119+
### Create a Custom Event Bus
120+
121+
Execute the following command to create the example namespace and a custom event bus.
122+
123+
```bash
124+
kubectl create ns ${EVENTBRIDGE_NAMESPACE}
125+
126+
cat <<EOF > bus.yaml
127+
apiVersion: eventbridge.services.k8s.aws/v1alpha1
128+
kind: EventBus
129+
metadata:
130+
name: ${EVENTBUS_NAME}
131+
spec:
132+
name: ${EVENTBUS_NAME}
133+
EOF
134+
135+
kubectl -n ${EVENTBRIDGE_NAMESPACE} create -f bus.yaml
136+
```
137+
138+
The output of above commands looks like
139+
140+
```bash
141+
namespace/eventbridge-example created
142+
eventbus.eventbridge.services.k8s.aws/custom-eventbus-ack created
143+
```
144+
145+
Verify the event bus resource is synchronized.
146+
147+
```bash
148+
kubectl -n ${EVENTBRIDGE_NAMESPACE} get eventbus ${EVENTBUS_NAME}
149+
```
150+
151+
The output of above commands looks like
152+
153+
```bash
154+
NAME SYNCED AGE
155+
custom-eventbus-ack True 64s
156+
```
157+
158+
### Create a Rule with an SQS Target
159+
160+
Execute the following command to retrieve the ARN for the SQS target created above needed for the Kubernetes manifest.
161+
162+
```bash
163+
export TARGET_QUEUE_ARN=$(aws --output json sqs get-queue-attributes --queue-url "https://sqs.${AWS_REGION}.amazonaws.com/${AWS_ACCOUNT_ID}/${TARGET_QUEUE}" --attribute-names QueueArn | jq -r '.Attributes.QueueArn')
164+
```
165+
166+
Execute the following command to create a Kubernetes manifest for a rule, forwarding events matching the specified rule
167+
filter criteria to the target queue. The EventBridge filter pattern will match any event received on the custom event
168+
bus with a `detail-type` of `event.from.ack.v0`. Alternatively, the filter pattern can be omitted to forward all events
169+
from the custom event bus.
170+
171+
```bash
172+
cat <<EOF > rule.yaml
173+
apiVersion: eventbridge.services.k8s.aws/v1alpha1
174+
kind: Rule
175+
metadata:
176+
name: $RULE_NAME
177+
spec:
178+
name: $RULE_NAME
179+
description: "ACK EventBridge Filter Rule to SQS using event bus reference"
180+
eventBusRef:
181+
from:
182+
name: $EVENTBUS_NAME
183+
eventPattern: |
184+
{
185+
"detail-type":["event.from.ack.v0"]
186+
}
187+
targets:
188+
- arn: $TARGET_QUEUE_ARN
189+
id: sqs-rule-target
190+
retryPolicy:
191+
maximumRetryAttempts: 0 # no retries
192+
EOF
193+
194+
kubectl -n ${EVENTBRIDGE_NAMESPACE} create -f rule.yaml
195+
```
196+
197+
The output of above commands looks like
198+
199+
```bash
200+
rule.eventbridge.services.k8s.aws/custom-eventbus-ack-sqs-rule created
201+
```
202+
203+
Verify the rule resource is synchronized.
204+
205+
```bash
206+
kubectl -n ${EVENTBRIDGE_NAMESPACE} get rule ${RULE_NAME}
207+
```
208+
209+
The output of above commands looks like
210+
211+
```bash
212+
NAME SYNCED AGE
213+
custom-eventbus-ack-sqs-rule True 18s
214+
```
215+
216+
### Verify the event filtering and forwarding is working
217+
218+
Execute the following command to send an event to the custom bus matching the rule filter pattern.
219+
220+
```bash
221+
cat <<EOF > event.json
222+
[
223+
{
224+
"Source": "my.aws.events.cli",
225+
"DetailType": "event.from.ack.v0",
226+
"Detail": "{\"hello-world\":\"from ACK for EventBridge\"}",
227+
"EventBusName": "${EVENTBUS_NAME}"
228+
}
229+
]
230+
EOF
231+
232+
aws events put-events --entries file://event.json
233+
```
234+
235+
The output of above commands looks like
236+
237+
```bash
238+
{
239+
"FailedEntryCount": 0,
240+
"Entries": [
241+
{
242+
"EventId": "ccd21ee8-339d-cabe-520d-b847c98ba2cb"
243+
}
244+
]
245+
}
246+
```
247+
248+
Verify the message was received by the SQS queue with
249+
250+
```bash
251+
aws sqs receive-message --queue-url https://sqs.${AWS_REGION}.amazonaws.com/${AWS_ACCOUNT_ID}/${TARGET_QUEUE}
252+
```
253+
254+
The output of above commands looks like
255+
256+
```bash
257+
{
258+
"Messages": [
259+
{
260+
"MessageId": "80cef2f3-ff25-4441-9217-665bb0217ec5",
261+
<snip>
262+
"Body": "{\"version\":\"0\",\"id\":\"def3d99b-806b-5d92-d036-9e0884bdc387\",\"detail-type\":\"event.from.ack.v0\",\"source\":\"my.aws.events.cli\",\"account\":\"1234567890\",\"time\":\"2023-03-22T11:22:34Z\",\"region\":\"us-east-1\",\"resources\":[],\"detail\":{\"hello-world\":\"from ACK for EventBridge\"}}"
263+
}
264+
]
265+
}
266+
```
267+
268+
## Next steps
269+
270+
The ACK service controller for Amazon EventBridge is based on the [Amazon EventBridge
271+
API](https://docs.aws.amazon.com/eventbridge/latest/APIReference/Welcome.html).
272+
273+
Refer to [API Reference](https://aws-controllers-k8s.github.io/community/reference/) for *EventBridge* to find all the
274+
supported Kubernetes custom resources and fields.
275+
276+
### Cleanup
277+
278+
Remove all the resource created in this tutorial using `kubectl delete` command.
279+
280+
```bash
281+
kubectl -n ${EVENTBRIDGE_NAMESPACE} delete -f rule.yaml
282+
kubectl -n ${EVENTBRIDGE_NAMESPACE} delete -f bus.yaml
283+
kubectl delete ns ${EVENTBRIDGE_NAMESPACE}
284+
```
285+
286+
The output of delete command should look like
287+
288+
```bash
289+
rule.eventbridge.services.k8s.aws "custom-eventbus-ack-sqs-rule" deleted
290+
eventbus.eventbridge.services.k8s.aws "custom-eventbus-ack" deleted
291+
namespace "eventbridge-example" deleted
292+
```
293+
294+
Remove the manually created SQS resource.
295+
296+
```bash
297+
aws sqs delete-queue --queue-url https://sqs.${AWS_REGION}.amazonaws.com/${AWS_ACCOUNT_ID}/${TARGET_QUEUE}
298+
```
299+
300+
If the command executes successfully, no output is generated.
301+
302+
To remove the EventBridge ACK service controller, related CRDs, and namespaces, see [ACK Cleanup][cleanup].
303+
304+
To delete your EKS clusters, see [Amazon EKS - Deleting a cluster][cleanup-eks].
305+
306+
[irsa-permissions]: ../../user-docs/irsa/
307+
[cleanup]: ../../user-docs/cleanup/
308+
[cleanup-eks]: https://docs.aws.amazon.com/eks/latest/userguide/delete-cluster.html

‎docs/content/docs/tutorials/pipes-example.md

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ aws ecr-public get-login-password --region us-east-1 | helm registry login --use
5454
Deploy the ACK service controller for Amazon Pipes using the [pipes-chart Helm chart](https://gallery.ecr.aws/aws-controllers-k8s/pipes-chart). Resources should be created in the `us-east-1` region:
5555

5656
```bash
57-
helm install --create-namespace -n ack-system oci://public.ecr.aws/aws-controllers-k8s/pipes-chart --version=v0.0.3 --generate-name --set=aws.region=us-east-1
57+
helm install --create-namespace -n ack-system oci://public.ecr.aws/aws-controllers-k8s/pipes-chart --version=v1.0.0 --generate-name --set=aws.region=us-east-1
5858
```
5959

6060
For a full list of available values to the Helm chart, please [review the values.yaml file](https://github.com/aws-controllers-k8s/pipes-controller/blob/main/helm/values.yaml).
@@ -327,22 +327,42 @@ supported Kubernetes custom resources and fields.
327327

328328
### Cleanup
329329

330-
Remove all the resource created in this tutorial using `kubectl delete` command.
330+
Remove all the Pipes resources created in this tutorial using `kubectl delete` command.
331331

332332
```bash
333-
kubectl -n ${QUEUE_NAMESPACE} delete -f pipe-sqs-to-sqs.yaml
333+
kubectl -n ${PIPE_NAMESPACE} delete -f pipe-sqs-to-sqs.yaml
334+
kubectl delete ns ${PIPE_NAMESPACE}
334335
```
335336

336337
The output of delete command should look like
337338

338339
```bash
339340
pipe.pipes.services.k8s.aws "pipes-sqs-to-sqs" deleted
341+
namespace "pipes-example" deleted
340342
```
341343

342344
{{% hint type="info" title="Deleting Delays" %}}
343345
It might take some time for the Pipe to be deleted as the operation is performed asynchronously in the API.
344346
{{% /hint %}}
345347

348+
Remove the manually created SQS resources.
349+
350+
```bash
351+
aws sqs delete-queue --queue-url https://sqs.${AWS_REGION}.amazonaws.com/${AWS_ACCOUNT_ID}/${SOURCE_QUEUE}
352+
aws sqs delete-queue --queue-url https://sqs.${AWS_REGION}.amazonaws.com/${AWS_ACCOUNT_ID}/${TARGET_QUEUE}
353+
```
354+
355+
If the command executes successfully, no output is generated.
356+
357+
Remove the manually created IAM resources.
358+
359+
```bash
360+
aws iam delete-role-policy --role-name ${PIPE_ROLE} --policy-name ${PIPE_POLICY}
361+
aws iam delete-role --role-name ${PIPE_ROLE}
362+
```
363+
364+
If the command executes successfully, no output is generated.
365+
346366
To remove the Pipes ACK service controller, related CRDs, and namespaces, see [ACK Cleanup][cleanup].
347367

348368
To delete your EKS clusters, see [Amazon EKS - Deleting a cluster][cleanup-eks].

0 commit comments

Comments
 (0)