Skip to content

Commit 50565fd

Browse files
JohnPrestonJohnPreston
authored
Added the functions to lookup VPC via ARN and Identifier on top of Tags (#801)
* Added the functions to lookup VPC via ARN and Identifier on top of Tags * Raise exception if VPC not found
1 parent c1c1d6e commit 50565fd

File tree

8 files changed

+141
-38
lines changed

8 files changed

+141
-38
lines changed

‎docs/syntax/compose_x/vpc.rst‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -96,23 +96,30 @@ policy name.
9696
Lookup
9797
======
9898

99+
You can Lookup the VPC ID (Subnets must use tags) via either
100+
101+
* Identifier: vpc-abcd1234
102+
* Arn: arn:aws:ec2:eu-west-1:123456789012:vpc/vpc-0b33b2fb87c205260
103+
* Tags (recommended)
104+
105+
See the ``uses_cases/vpc`` for examples.
106+
99107
.. code-block:: yaml
100108
101109
x-vpc:
102110
Lookup:
103111
VpcId:
104112
Tags:
105-
- key: value
113+
key: value
106114
PublicSubnets:
107115
Tags:
108-
- vpc::usage: public
116+
vpc::usage: public
109117
AppSubnets:
110118
Tags:
111-
- vpc::usage: application
119+
vpc::usage: application
112120
StorageSubnets:
113121
Tags:
114-
- vpc::usage: storage0
115-
122+
vpc::usage: storage
116123
117124
118125
.. warning::

‎ecs_composex/vpc/vpc_aws.py‎

Lines changed: 53 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# SPDX-License-Identifier: MPL-2.0
22
# Copyright 2020-2025 John Mille <[email protected]>
3-
3+
import botocore.client
44
from boto3.session import Session
5-
from compose_x_common.aws.arns import ARNS_PER_TAGGINGAPI_TYPE
6-
from compose_x_common.compose_x_common import keyisset
5+
from compose_x_common.aws.arns import ARNS_PER_CFN_TYPE, ARNS_PER_TAGGINGAPI_TYPE
6+
from compose_x_common.compose_x_common import keyisset, set_else_none
77

88
from ecs_composex.common.aws import find_aws_resource_arn_from_tags_api
99
from ecs_composex.common.logging import LOG
@@ -82,6 +82,55 @@ def validate_subnets_belong_with_vpc(
8282
)
8383

8484

85+
def lookup_vpc_id(vpc_id_details: dict, lookup_session: Session) -> str:
86+
"""
87+
Function to find the VPC either by ID, Arn or Tags. Arn takes priority, then ID, then Tags
88+
"""
89+
vpc_id = set_else_none("Identifier", vpc_id_details)
90+
vpc_arn = set_else_none("Arn", vpc_id_details)
91+
vpc_tags = set_else_none(TAGS_KEY, vpc_id_details)
92+
arn_from_arn = True if vpc_arn and not vpc_id else False
93+
94+
if vpc_arn:
95+
vpc_re = ARNS_PER_CFN_TYPE["AWS::EC2::VPC"]
96+
if not vpc_re.match(vpc_arn):
97+
raise ValueError(f"{vpc_arn} is not a valid VPC ARN")
98+
vpc_id = vpc_re.match(vpc_arn).group("id")
99+
100+
if vpc_id:
101+
cloud_control_client = lookup_session.client("cloudcontrol")
102+
try:
103+
cloud_control_client.get_resource(
104+
TypeName="AWS::EC2::VPC",
105+
Identifier=vpc_id,
106+
)
107+
except botocore.client.ClientError as error:
108+
LOG.exception(error)
109+
raise ValueError(f"{vpc_id} is not a valid VPC ID")
110+
if arn_from_arn:
111+
return vpc_arn
112+
else:
113+
ec2_client = lookup_session.client("ec2")
114+
sts_client = lookup_session.client("sts")
115+
account_id = sts_client.get_caller_identity()["Account"]
116+
return (
117+
f"arn:aws:ec2:{ec2_client.meta.region_name}:{account_id}:vpc/{vpc_id}"
118+
)
119+
120+
elif vpc_tags:
121+
return find_aws_resource_arn_from_tags_api(
122+
vpc_id_details,
123+
lookup_session,
124+
"ec2:vpc",
125+
allow_multi=False,
126+
)
127+
raise LookupError(
128+
"Failed to find VPC with given details: {}".format(
129+
vpc_id or vpc_arn or vpc_tags
130+
)
131+
)
132+
133+
85134
def lookup_x_vpc_settings(vpc_resource):
86135
"""
87136
Method to set VPC settings from x-vpc
@@ -103,11 +152,9 @@ def lookup_x_vpc_settings(vpc_resource):
103152
APP_SUBNETS.title,
104153
STORAGE_SUBNETS.title,
105154
]
106-
vpc_arn = find_aws_resource_arn_from_tags_api(
155+
vpc_arn = lookup_vpc_id(
107156
vpc_resource.lookup[VPC_ID.title],
108157
vpc_resource.lookup_session,
109-
vpc_type,
110-
allow_multi=False,
111158
)
112159
vpc_re = ARNS_PER_TAGGINGAPI_TYPE[vpc_type]
113160
vpc_settings = {

‎tests/features/features/vpc.feature‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,19 @@ Feature: ecs_composex.vpc
88
Then I render the docker-compose to composex to validate
99
And I render all files to verify execution
1010
Examples:
11-
| file_path | override_file |
12-
| use-cases/blog.features.yml | use-cases/vpc/new_vpc.yml |
13-
| use-cases/blog.features.yml | use-cases/vpc/new_with_flowlogs.yml |
11+
| file_path | override_file |
12+
| use-cases/blog.features.yml | use-cases/vpc/new_vpc.yml |
13+
| use-cases/blog.features.yml | use-cases/vpc/new_with_flowlogs.yml |
14+
| use-cases/blog.features.yml | use-cases/vpc/no_nats_no_endpoints.yaml |
15+
16+
Scenario Outline: VPC Lookup
17+
Given With <file_path>
18+
And With <override_file>
19+
And I use defined files as input to define execution settings
20+
Then I render the docker-compose to composex to validate
21+
And I render all files to verify execution
22+
Examples:
23+
| file_path | override_file |
24+
| use-cases/blog.features.yml | use-cases/vpc/lookup_vpc_via_tags.yaml |
25+
| use-cases/blog.features.yml | use-cases/vpc/lookup_vpc_via_arn.yaml |
26+
| use-cases/blog.features.yml | use-cases/vpc/lookup_vpc_via_id.yaml |

‎use-cases/vpc/lookup_vpc.yml‎

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎use-cases/vpc/lookup_vpc_via_arn.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
# Blog applications
3+
4+
version: '3.8'
5+
6+
x-vpc:
7+
Lookup:
8+
VpcId:
9+
Arn: arn:aws:ec2:eu-west-1:373709687836:vpc/vpc-0b33b2fb87c205260
10+
AppSubnets:
11+
Tags:
12+
vpc::usage: application
13+
StorageSubnets:
14+
Tags:
15+
vpc::usage: storage
16+
PublicSubnets:
17+
Tags:
18+
vpc::usage: public

‎use-cases/vpc/lookup_vpc_via_id.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
# Blog applications
3+
4+
version: '3.8'
5+
6+
x-vpc:
7+
Lookup:
8+
VpcId:
9+
Identifier: vpc-0b33b2fb87c205260
10+
AppSubnets:
11+
Tags:
12+
vpc::usage: application
13+
StorageSubnets:
14+
Tags:
15+
vpc::usage: storage
16+
PublicSubnets:
17+
Tags:
18+
vpc::usage: public

‎use-cases/vpc/lookup_vpc_via_tags.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
# Blog applications
3+
4+
version: '3.8'
5+
6+
x-vpc:
7+
Lookup:
8+
VpcId:
9+
Tags:
10+
Name: e2e-lookup-vpc
11+
AppSubnets:
12+
Tags:
13+
vpc::usage: application
14+
StorageSubnets:
15+
Tags:
16+
vpc::usage: storage
17+
PublicSubnets:
18+
Tags:
19+
vpc::usage: public

‎use-cases/vpc/no_nats_no_endpoints.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
x-vpc:
2+
Properties:
3+
VpcCidr: 192.168.220.0/24 # A simple CIDR with plenty of room for the deployment.
4+
DisableNat: True # Although the Public, App and Storage subnets are created, no NAT nor route is created.
5+
Endpoints: {} # Set to {} to disable creating the default VPC endpoints Compose-X use. We won't be needing them.

0 commit comments

Comments
 (0)