Skip to content

Commit d838fef

Browse files
oli-proli-pr
committed
fix:ECS Task needs memory and CPU overrides when invoked from Pipe
1 parent 7708154 commit d838fef

File tree

6 files changed

+292
-0
lines changed

6 files changed

+292
-0
lines changed

‎examples/pipe-ecs-target/README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# EventBridge Pipes ECS Target
2+
3+
Configuration in this directory creates EventBridge resource configuration including an ECS task target for a Pipe.
4+
5+
## Usage
6+
7+
To run this example you need to execute:
8+
9+
```bash
10+
$ terraform init
11+
$ terraform plan
12+
$ terraform apply
13+
```
14+
15+
Note that this example may create resources which cost money. Run `terraform destroy` when you don't need these resources.
16+
17+
<!-- BEGIN_TF_DOCS -->
18+
## Requirements
19+
20+
| Name | Version |
21+
|------|---------|
22+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
23+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.98 |
24+
| <a name="requirement_random"></a> [random](#requirement\_random) | >= 3.0 |
25+
26+
## Providers
27+
28+
| Name | Version |
29+
|------|---------|
30+
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.100.0 |
31+
| <a name="provider_random"></a> [random](#provider\_random) | 3.7.2 |
32+
33+
## Modules
34+
35+
| Name | Source | Version |
36+
|------|--------|---------|
37+
| <a name="module_ecs_cluster"></a> [ecs\_cluster](#module\_ecs\_cluster) | terraform-aws-modules/ecs/aws | ~> 5.0 |
38+
| <a name="module_eventbridge"></a> [eventbridge](#module\_eventbridge) | ../../ | n/a |
39+
40+
## Resources
41+
42+
| Name | Type |
43+
|------|------|
44+
| [aws_iam_policy.eventbridge_pipes_ecs_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
45+
| [aws_iam_role_policy_attachment.eventbridge_pipes_ecs_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
46+
| [aws_security_group_rule.default_egress_https](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
47+
| [aws_sqs_queue.source](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue) | resource |
48+
| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
49+
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
50+
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
51+
| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/security_group) | data source |
52+
| [aws_subnets.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) | data source |
53+
| [aws_vpc.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) | data source |
54+
55+
## Inputs
56+
57+
No inputs.
58+
59+
## Outputs
60+
61+
| Name | Description |
62+
|------|-------------|
63+
| <a name="output_eventbridge_bus_arn"></a> [eventbridge\_bus\_arn](#output\_eventbridge\_bus\_arn) | The EventBridge Bus ARN |
64+
| <a name="output_eventbridge_rule_arns"></a> [eventbridge\_rule\_arns](#output\_eventbridge\_rule\_arns) | The EventBridge Rule ARNs |
65+
| <a name="output_eventbridge_rule_ids"></a> [eventbridge\_rule\_ids](#output\_eventbridge\_rule\_ids) | The EventBridge Rule IDs |
66+
<!-- END_TF_DOCS -->

‎examples/pipe-ecs-target/main.tf‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
provider "aws" {
2+
region = "eu-west-1"
3+
4+
# Make it faster by skipping something
5+
skip_metadata_api_check = true
6+
skip_region_validation = true
7+
skip_credentials_validation = true
8+
}
9+
10+
#############################################################
11+
# Data sources to get VPC and default security group details
12+
#############################################################
13+
data "aws_vpc" "default" {
14+
default = true
15+
}
16+
17+
data "aws_security_group" "default" {
18+
name = "default"
19+
vpc_id = data.aws_vpc.default.id
20+
}
21+
22+
data "aws_subnets" "default" {
23+
filter {
24+
name = "vpc-id"
25+
values = [data.aws_vpc.default.id]
26+
}
27+
}
28+
29+
data "aws_caller_identity" "current" {}
30+
data "aws_region" "current" {}
31+
32+
resource "aws_sqs_queue" "source" {
33+
name = "${random_pet.this.id}-source"
34+
}
35+
36+
####################
37+
# Actual Eventbridge
38+
####################
39+
module "eventbridge" {
40+
source = "../../"
41+
42+
# Schedules can only be created on default bus
43+
create_bus = false
44+
45+
create_role = true
46+
role_name = "ecs-eventbridge-${random_pet.this.id}"
47+
attach_ecs_policy = true
48+
ecs_target_arns = [
49+
module.ecs_cluster.services["hello-world"].task_definition_arn,
50+
"arn:aws:ecs:${data.aws_region.current.id}:${data.aws_caller_identity.current.account_id}:task/${random_pet.this.id}/*"
51+
]
52+
53+
ecs_pass_role_resources = [module.ecs_cluster.services["hello-world"].task_exec_iam_role_arn]
54+
55+
pipes = {
56+
test_ecs_pipe = {
57+
58+
attach_policies_for_integrations = true
59+
60+
source = aws_sqs_queue.source.arn
61+
target = module.ecs_cluster.cluster_arn
62+
63+
attach_policies_for_integrations = true
64+
65+
target_parameters = {
66+
ecs_task_parameters = {
67+
assign_public_ip = "ENABLED"
68+
task_count = 1
69+
launch_type = "FARGATE"
70+
task_definition_arn = module.ecs_cluster.services["hello-world"].task_definition_arn
71+
container_name = "hello-world"
72+
73+
security_groups = [module.ecs_cluster.services["hello-world"].security_group_id]
74+
subnets = data.aws_subnets.default.ids
75+
76+
memory = 512
77+
cpu = 256
78+
79+
enable_ecs_managed_tags = true
80+
}
81+
}
82+
}
83+
}
84+
}
85+
86+
resource "aws_iam_policy" "eventbridge_pipes_ecs_policy" {
87+
name = "test-pipes-ecs-policy"
88+
description = "Policy for EventBridge Pipes to run ECS tasks"
89+
90+
policy = jsonencode({
91+
Version = "2012-10-17"
92+
Statement = [
93+
{
94+
Effect = "Allow"
95+
Action = [
96+
"ecs:RunTask",
97+
"ecs:TagResource"
98+
]
99+
Resource = [module.ecs_cluster.services["hello-world"].task_definition_arn]
100+
},
101+
{
102+
Effect = "Allow"
103+
Action = [
104+
"iam:PassRole"
105+
]
106+
Resource = [
107+
module.ecs_cluster.services["hello-world"].task_exec_iam_role_arn,
108+
module.ecs_cluster.services["hello-world"].tasks_iam_role_arn
109+
]
110+
Condition = {
111+
StringLike = {
112+
"iam:PassedToService" = "ecs-tasks.amazonaws.com"
113+
}
114+
}
115+
}
116+
]
117+
})
118+
}
119+
120+
resource "aws_iam_role_policy_attachment" "eventbridge_pipes_ecs_policy" {
121+
for_each = module.eventbridge.eventbridge_pipe_role_names
122+
123+
role = each.value
124+
policy_arn = aws_iam_policy.eventbridge_pipes_ecs_policy.arn
125+
}
126+
127+
######
128+
# ECS
129+
######
130+
131+
module "ecs_cluster" {
132+
source = "terraform-aws-modules/ecs/aws"
133+
version = "~> 5.0"
134+
135+
cluster_name = random_pet.this.id
136+
137+
fargate_capacity_providers = {
138+
FARGATE = {
139+
default_capacity_provider_strategy = {
140+
weight = 100
141+
}
142+
}
143+
FARGATE_SPOT = {
144+
default_capacity_provider_strategy = {
145+
weight = 100
146+
}
147+
}
148+
}
149+
150+
services = {
151+
hello-world = {
152+
create_service = false
153+
subnet_ids = data.aws_subnets.default.ids
154+
desired_count = 1
155+
deployment_maximum_percent = 100
156+
deployment_minimum_healthy_percent = 0
157+
158+
security_group_rules = {
159+
egress = {
160+
type = "egress"
161+
description = "container-pull-egress"
162+
from_port = 443
163+
to_port = 443
164+
protocol = "tcp"
165+
cidr_blocks = ["0.0.0.0/0"]
166+
}
167+
}
168+
169+
container_definitions = {
170+
hello-world = {
171+
image = "public.ecr.aws/docker/library/hello-world:latest",
172+
cpu = 256,
173+
memory = 512
174+
}
175+
}
176+
}
177+
}
178+
}
179+
180+
##################
181+
# Extra resources
182+
##################
183+
184+
resource "random_pet" "this" {
185+
length = 2
186+
}
187+
188+
resource "aws_security_group_rule" "default_egress_https" {
189+
type = "egress"
190+
from_port = 443
191+
to_port = 443
192+
protocol = "tcp"
193+
cidr_blocks = ["0.0.0.0/0"]
194+
security_group_id = data.aws_security_group.default.id
195+
description = "Allow HTTPS outbound for ECR image pulls"
196+
}

‎examples/pipe-ecs-target/outputs.tf‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
output "eventbridge_bus_arn" {
2+
description = "The EventBridge Bus ARN"
3+
value = module.eventbridge.eventbridge_bus_arn
4+
}
5+
6+
output "eventbridge_rule_ids" {
7+
description = "The EventBridge Rule IDs"
8+
value = module.eventbridge.eventbridge_rule_ids
9+
}
10+
11+
output "eventbridge_rule_arns" {
12+
description = "The EventBridge Rule ARNs"
13+
value = module.eventbridge.eventbridge_rule_arns
14+
}

‎examples/pipe-ecs-target/variables.tf‎

Whitespace-only changes.

‎examples/pipe-ecs-target/versions.tf‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
terraform {
2+
required_version = ">= 1.0"
3+
4+
required_providers {
5+
aws = {
6+
source = "hashicorp/aws"
7+
version = ">= 5.98"
8+
}
9+
random = {
10+
source = "hashicorp/random"
11+
version = ">= 3.0"
12+
}
13+
}
14+
}

‎main.tf‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -883,6 +883,8 @@ resource "aws_pipes_pipe" "this" {
883883
container_override {
884884
command = try(ecs_task_parameters.value.command, [])
885885
name = ecs_task_parameters.value.container_name
886+
cpu = ecs_task_parameters.value.cpu
887+
memory = ecs_task_parameters.value.memory
886888

887889
dynamic "environment" {
888890
for_each = try(ecs_task_parameters.value.environment, [])

0 commit comments

Comments
 (0)