Skip to content

Commit cc41448

Browse files
igorborgestigorborgest
committed
Add tags in the CloudFormation template.
1 parent 4f962d5 commit cc41448

File tree

6 files changed

+133
-98
lines changed

6 files changed

+133
-98
lines changed

‎testing/cloudformation.yaml‎

Lines changed: 58 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
AWSTemplateFormatVersion: 2010-09-09
2-
Description: |
3-
AWS Data Wrangler Development Infrastructure
2+
Description: AWS Data Wrangler Development Infrastructure
43
Parameters:
54
DatabasesPassword:
65
Type: String
@@ -10,17 +9,17 @@ Resources:
109
VPC:
1110
Type: AWS::EC2::VPC
1211
Properties:
12+
Tags:
13+
- Key: Env
14+
Value: aws-data-wrangler
1315
CidrBlock: 10.19.224.0/19
1416
EnableDnsSupport: true
1517
EnableDnsHostnames: true
16-
Tags:
17-
- Key: Name
18-
Value: aws-data-wrangler
1918
InternetGateway:
2019
Type: AWS::EC2::InternetGateway
2120
Properties:
2221
Tags:
23-
- Key: Name
22+
- Key: Env
2423
Value: aws-data-wrangler
2524
InternetGatewayAttachment:
2625
Type: AWS::EC2::VPCGatewayAttachment
@@ -32,6 +31,9 @@ Resources:
3231
PublicSubnet1:
3332
Type: AWS::EC2::Subnet
3433
Properties:
34+
Tags:
35+
- Key: Env
36+
Value: aws-data-wrangler
3537
VpcId:
3638
Ref: VPC
3739
AvailabilityZone:
@@ -40,12 +42,12 @@ Resources:
4042
- Fn::GetAZs: ''
4143
CidrBlock: 10.19.229.0/24
4244
MapPublicIpOnLaunch: true
43-
Tags:
44-
- Key: Name
45-
Value: aws-data-wrangler Public Subnet (AZ1)
4645
PublicSubnet2:
4746
Type: AWS::EC2::Subnet
4847
Properties:
48+
Tags:
49+
- Key: Env
50+
Value: aws-data-wrangler
4951
VpcId:
5052
Ref: VPC
5153
AvailabilityZone:
@@ -54,17 +56,14 @@ Resources:
5456
- Fn::GetAZs: ''
5557
CidrBlock: 10.19.230.0/24
5658
MapPublicIpOnLaunch: true
57-
Tags:
58-
- Key: Name
59-
Value: aws-data-wrangler Public Subnet (AZ2)
6059
PublicRouteTable:
6160
Type: AWS::EC2::RouteTable
6261
Properties:
62+
Tags:
63+
- Key: Env
64+
Value: aws-data-wrangler
6365
VpcId:
6466
Ref: VPC
65-
Tags:
66-
- Key: Name
67-
Value: aws-data-wrangler Public Routes
6867
DefaultPublicRoute:
6968
Type: AWS::EC2::Route
7069
DependsOn: InternetGatewayAttachment
@@ -97,6 +96,9 @@ Resources:
9796
KmsKey:
9897
Type: AWS::KMS::Key
9998
Properties:
99+
Tags:
100+
- Key: Env
101+
Value: aws-data-wrangler
100102
Description: Aws Data Wrangler Test Key.
101103
KeyPolicy:
102104
Version: '2012-10-17'
@@ -112,7 +114,8 @@ Resources:
112114
- Sid: Allow administration of the key
113115
Effect: Allow
114116
Principal:
115-
AWS: '*'
117+
AWS:
118+
Ref: AWS::AccountId
116119
Action:
117120
- kms:Create*
118121
- kms:Describe*
@@ -130,6 +133,9 @@ Resources:
130133
Bucket:
131134
Type: AWS::S3::Bucket
132135
Properties:
136+
Tags:
137+
- Key: Env
138+
Value: aws-data-wrangler
133139
PublicAccessBlockConfiguration:
134140
BlockPublicAcls: true
135141
BlockPublicPolicy: true
@@ -146,6 +152,9 @@ Resources:
146152
RedshiftRole:
147153
Type: AWS::IAM::Role
148154
Properties:
155+
Tags:
156+
- Key: Env
157+
Value: aws-data-wrangler
149158
AssumeRolePolicyDocument:
150159
Version: 2012-10-17
151160
Statement:
@@ -205,12 +214,18 @@ Resources:
205214
RedshiftSubnetGroup:
206215
Type: AWS::Redshift::ClusterSubnetGroup
207216
Properties:
217+
Tags:
218+
- Key: Env
219+
Value: aws-data-wrangler
208220
Description: AWS Data Wrangler Test Arena - Redshift Subnet Group
209221
SubnetIds:
210222
- Ref: PublicSubnet1
211223
DatabaseSecurityGroup:
212224
Type: AWS::EC2::SecurityGroup
213225
Properties:
226+
Tags:
227+
- Key: Env
228+
Value: aws-data-wrangler
214229
VpcId:
215230
Ref: VPC
216231
GroupDescription: AWS Data Wrangler Test Arena - Redshift security group
@@ -228,6 +243,9 @@ Resources:
228243
Redshift:
229244
Type: AWS::Redshift::Cluster
230245
Properties:
246+
Tags:
247+
- Key: Env
248+
Value: aws-data-wrangler
231249
DBName: test
232250
MasterUsername: test
233251
MasterUserPassword:
@@ -264,13 +282,19 @@ Resources:
264282
RdsSubnetGroup:
265283
Type: AWS::RDS::DBSubnetGroup
266284
Properties:
285+
Tags:
286+
- Key: Env
287+
Value: aws-data-wrangler
267288
DBSubnetGroupDescription: RDS Database Subnet Group
268289
SubnetIds:
269290
- Ref: PublicSubnet1
270291
- Ref: PublicSubnet2
271292
AuroraRole:
272293
Type: AWS::IAM::Role
273294
Properties:
295+
Tags:
296+
- Key: Env
297+
Value: aws-data-wrangler
274298
AssumeRolePolicyDocument:
275299
Version: 2012-10-17
276300
Statement:
@@ -297,13 +321,19 @@ Resources:
297321
PostgresqlParameterGroup:
298322
Type: AWS::RDS::DBClusterParameterGroup
299323
Properties:
324+
Tags:
325+
- Key: Env
326+
Value: aws-data-wrangler
300327
Description: Postgres 11
301328
Family: aurora-postgresql11
302329
Parameters:
303330
apg_plan_mgmt.capture_plan_baselines: 'off'
304331
AuroraClusterPostgresql:
305332
Type: AWS::RDS::DBCluster
306333
Properties:
334+
Tags:
335+
- Key: Env
336+
Value: aws-data-wrangler
307337
Engine: aurora-postgresql
308338
EngineVersion: '11.6'
309339
DBClusterIdentifier: postgresql-cluster-wrangler
@@ -326,6 +356,9 @@ Resources:
326356
AuroraInstancePostgresql:
327357
Type: AWS::RDS::DBInstance
328358
Properties:
359+
Tags:
360+
- Key: Env
361+
Value: aws-data-wrangler
329362
Engine: aurora-postgresql
330363
EngineVersion: '11.6'
331364
DBInstanceIdentifier: postgresql-instance-wrangler
@@ -338,6 +371,9 @@ Resources:
338371
MysqlParameterGroup:
339372
Type: AWS::RDS::DBClusterParameterGroup
340373
Properties:
374+
Tags:
375+
- Key: Env
376+
Value: aws-data-wrangler
341377
Description: Mysql 5.7
342378
Family: aurora-mysql5.7
343379
Parameters:
@@ -356,6 +392,9 @@ Resources:
356392
AuroraClusterMysql:
357393
Type: AWS::RDS::DBCluster
358394
Properties:
395+
Tags:
396+
- Key: Env
397+
Value: aws-data-wrangler
359398
Engine: aurora-mysql
360399
EngineVersion: '5.7'
361400
DBClusterIdentifier: mysql-cluster-wrangler
@@ -378,6 +417,9 @@ Resources:
378417
AuroraInstanceMysql:
379418
Type: AWS::RDS::DBInstance
380419
Properties:
420+
Tags:
421+
- Key: Env
422+
Value: aws-data-wrangler
381423
Engine: aurora-mysql
382424
EngineVersion: '5.7'
383425
DBInstanceIdentifier: mysql-instance-wrangler
@@ -459,21 +501,6 @@ Resources:
459501
PASSWORD:
460502
Ref: DatabasesPassword
461503
Name: aws-data-wrangler-mysql
462-
DynamoDBTable:
463-
Type: AWS::DynamoDB::Table
464-
Properties:
465-
AttributeDefinitions:
466-
- AttributeName: attr_hash
467-
AttributeType: S
468-
- AttributeName: attr_range
469-
AttributeType: S
470-
KeySchema:
471-
- AttributeName: attr_hash
472-
KeyType: HASH
473-
- AttributeName: attr_range
474-
KeyType: RANGE
475-
BillingMode: PAY_PER_REQUEST
476-
TableName: aws-data-wrangler
477504
Outputs:
478505
BucketName:
479506
Value:
@@ -542,12 +569,6 @@ Outputs:
542569
- AuroraInstanceMysql
543570
- Endpoint.Address
544571
Description: Mysql Address
545-
DynamoDbTableARN:
546-
Value:
547-
Fn::GetAtt:
548-
- DynamoDBTable
549-
- Arn
550-
Description: DynamoDB table name
551572
Region:
552573
Value:
553574
Ref: AWS::Region

‎testing/test_awswrangler/_utils.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66
ts = lambda x: datetime.strptime(x, "%Y-%m-%d %H:%M:%S.%f") # noqa
77
dt = lambda x: datetime.strptime(x, "%Y-%m-%d").date() # noqa
88

9+
CFN_VALID_STATUS = ["CREATE_COMPLETE", "ROLLBACK_COMPLETE", "UPDATE_COMPLETE", "UPDATE_ROLLBACK_COMPLETE"]
10+
911

1012
def get_df():
1113
df = pd.DataFrame(

‎testing/test_awswrangler/test_cloudwatch.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@
77
import awswrangler as wr
88
from awswrangler import exceptions
99

10+
from ._utils import CFN_VALID_STATUS
11+
1012
logging.basicConfig(level=logging.INFO, format="[%(asctime)s][%(levelname)s][%(name)s][%(funcName)s] %(message)s")
1113
logging.getLogger("awswrangler").setLevel(logging.DEBUG)
1214
logging.getLogger("botocore.credentials").setLevel(logging.CRITICAL)
@@ -15,8 +17,9 @@
1517
@pytest.fixture(scope="module")
1618
def cloudformation_outputs():
1719
response = boto3.client("cloudformation").describe_stacks(StackName="aws-data-wrangler")
20+
stack = [x for x in response.get("Stacks") if x["StackStatus"] in CFN_VALID_STATUS][0]
1821
outputs = {}
19-
for output in response.get("Stacks")[0].get("Outputs"):
22+
for output in stack.get("Outputs"):
2023
outputs[output.get("OutputKey")] = output.get("OutputValue")
2124
yield outputs
2225

0 commit comments

Comments
 (0)