Skip to content

Commit 352b98e

Browse files
author
AncaGhenade
committed
migrate create resources to terraform; add validator as submodule for ease of use & access
1 parent a707d48 commit 352b98e

File tree

8 files changed

+183
-81
lines changed

8 files changed

+183
-81
lines changed

‎.gitignore‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,4 +60,8 @@ src/main/shipment-list-frontend/yarn-error.log*
6060

6161
setup/terraform/terraform.tfstate
6262
setup/terraform/.terraform.lock.hcl
63-
setup/terraform/terraform.tfstate.backup
63+
setup/terraform/terraform.tfstate.backup
64+
65+
# lambda module
66+
shipment-picture-lambda-validator/target
67+
shipment-picture-lambda-validator/.idea

‎README.md‎

Lines changed: 43 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,15 @@ files, `application-prod.yml`, and `application-dev.yml`.
4949

5050
## Running it
5151

52-
### Production simulation
52+
## Production simulation
5353

5454
Now, we don’t have a real production environment because that’s not the point here, but most likely,
5555
an application like this runs on a container orchestration platform, and all the necessary configs
5656
are still provided. Since we’re only simulating a production instance, all the configurations are
5757
kept in the `application-prod.yml` file.
5858

59+
### User credentials
60+
5961
Before getting started, it's important to note that an IAM user, who's credentials will be used,
6062
needs to be created with the following policies:
6163

@@ -69,18 +71,31 @@ We will be using the user's credentials and export them as temporary environment
6971
$ export AWS_ACCESS_KEY_ID=[your_aws_access_key_id]
7072
$ export AWS_SECRET_ACCESS_KEY=[your_aws_secret_access_key_id]
7173
```
72-
Make sure you have Terraform [installed](https://developer.hashicorp.com/terraform/downloads).
74+
75+
### Building the validator module
76+
77+
Step into the `shipment-picture-lambda-validator` module and run `mvn clean package shade:shade`.
78+
This will create an uber-jar by packaging all its dependencies. We'll need this one in the next step.
79+
80+
### Creating resources - running Terraform
81+
82+
Make sure you have Terraform [installed](https://developer.hashicorp.com/terraform/downloads).If you're
83+
not familiar or uncomfortable with Terraform, there's also a branch that uses only AWS cli to create resources.
84+
7385
Under setup/terraform run:
7486
```
7587
$ terraform init
7688
$ terraform plan
7789
```
78-
once these 2 commands run successfully and no errors occur, it's time to run:
90+
Once these 2 commands run successfully and no errors occur, it's time to run:
7991
```
80-
$ terraform apply --auto-approve
92+
$ terraform apply
8193
```
8294

83-
This should create the needed S3 bucket, the DynamoDB `shipment` table and populate it with some sample data.
95+
This should create the needed S3 bucket, the DynamoDB `shipment` table and populate it with some
96+
sample data, and the Lambda function that will help with picture validation.
97+
98+
### Running the GUI
8499

85100
Now `cd` into `src/main/shipment-list-frontend` and run `npm install` and `npm start`.
86101
This will spin up the React app that can be accessed on `localhost:3000`.
@@ -96,31 +111,22 @@ $ mvn spring-boot:run -Dspring-boot.run.profiles=prod
96111
```
97112
Notice the `prod` profile is being set via command line arguments.
98113

114+
### Using the application
115+
99116
At `localhost:3000` you should now be able to see a list of shipments with standard icons,
100117
that means that only the database is populated, the pictures still need to be added from the
101118
`sample-pictures` folder.
119+
102120
The weight of a shipment we can perceive, but not the size, that's why we need pictures to understand,
103121
using the "banana for scale" measuring unit. How else would we know??
104122

123+
Current available actions using the GUI:
105124

106-
The Lambda function is still not up. This falls under the `shipment-list-lambda-validator` project.
107-
108-
```
109-
$ git clone https://github.com/tinyg210/shipment-list-lambda-validator.git
110-
```
111-
112-
The `create-lambda.sh` script will do everything that needs for the creation and configuration of
113-
the
114-
Lambda. (I know what you're thinking, Terraform will follow.)
115-
Run `add-notif-config-for-lambda.sh`, but before that remember to edit `notification-config.json`
116-
with
117-
your own AWS account ID. This will enable the Lambda to receive notifications every time a picture
118-
is being
119-
added to S3.
120-
121-
You should now be able to add a new picture for each shipment. Files that are not pictures will be
122-
deleted
123-
and the shipment picture will be replaced with a generic icon.
125+
- upload a new image
126+
- delete shipment from the list
127+
128+
Files that are not pictures will be deleted
129+
and the shipment picture will be replaced with a generic icon, because we don't want any trouble.
124130

125131
### Developer environment
126132

@@ -144,19 +150,28 @@ $ tflocal --help
144150
Usage: terraform [global options] <subcommand> [args]
145151
...
146152
```
147-
From here on, it's the same as before:
153+
From here on, it's smooth sailing, the same as before. You can eve use the same folder to run
154+
your commands:
155+
148156
```
149-
$ terraform plan
150-
$ terraform apply --auto-approve
157+
$ tflocal init
158+
$ tflocal plan -var 'env=dev'
159+
$ tflocal apply -var 'env=dev'
151160
```
161+
Well, almost, we're doing a little cheating and passing and environmental variable to let the Lambda
162+
know this is the `dev` environment.
152163

153164

154-
After that, the Spring Boot application needs to start using the dev profile:
165+
After that, the Spring Boot application needs to start using the dev profile (make sure you're in the
166+
root folder):
155167

156168
```
157169
$ mvn spring-boot:run -Dspring-boot.run.profiles=dev
158170
```
159171

160-
The same actions should be easily achieved again, but locally.
172+
Go back to `localhost:3000` and a new list will be available, and notice that the functionalities of
173+
the application have not changed.
174+
175+
There you have it, smooth transition from AWS to Localstack, with no code change. 👍🏻
161176

162177

‎docker-compose.yml‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,10 @@ services:
55
image: localstack/localstack
66
container_name: localstack
77
ports:
8-
- "4566:4566" # port of to where localstack can be addressed to
9-
- "9000:9000"
8+
- "127.0.0.1:4566:4566" # LocalStack Gateway
9+
- "127.0.0.1:4510-4559:4510-4559" # external services port range
1010
environment:
11-
- SERVICES=s3,dynamodb,lambda # a list of desired services you want to use.
12-
- DEFAULT_REGION=eu-central-1 # where the localstack mocks to be running
13-
- AWS_ACCESS_KEY_ID=test_access_key
14-
- AWS_SECRET_ACCESS_KEY=test_secret_access_key
11+
- DEFAULT_REGION=eu-central-1 # where the localstack mocks will be running
1512
- DATA_DIR=/tmp/localstack/data
1613
- PORT_WEB_UI=9000
1714
- LAMBDA_EXECUTOR=local

‎pom.xml‎

Lines changed: 0 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -80,37 +80,6 @@
8080
</excludes>
8181
</configuration>
8282
</plugin>
83-
84-
<!-- TBD? -->
85-
<!-- <plugin>-->
86-
<!-- <groupId>org.apache.maven.plugins</groupId>-->
87-
<!-- <artifactId>maven-checkstyle-plugin</artifactId>-->
88-
<!-- <version>${maven-checkstyle-plugin.version}</version>-->
89-
<!-- <dependencies>-->
90-
<!-- <dependency>-->
91-
<!-- <groupId>com.puppycrawl.tools</groupId>-->
92-
<!-- <artifactId>checkstyle</artifactId>-->
93-
<!-- <version>${checkstyle.version}</version>-->
94-
<!-- </dependency>-->
95-
<!-- </dependencies>-->
96-
<!-- <configuration>-->
97-
<!-- <configLocation>google_checks.xml</configLocation>-->
98-
<!-- <encoding>UTF-8</encoding>-->
99-
<!-- <consoleOutput>true</consoleOutput>-->
100-
<!-- <failsOnError>true</failsOnError>-->
101-
<!-- <linkXRef>false</linkXRef>-->
102-
<!-- <violationSeverity>warning</violationSeverity>-->
103-
<!-- </configuration>-->
104-
<!-- <executions>-->
105-
<!-- <execution>-->
106-
<!-- <id>validate</id>-->
107-
<!-- <phase>validate</phase>-->
108-
<!-- <goals>-->
109-
<!-- <goal>check</goal>-->
110-
<!-- </goals>-->
111-
<!-- </execution>-->
112-
<!-- </executions>-->
113-
<!-- </plugin>-->
11483
</plugins>
11584
</build>
11685
</project>

sample-pictures/wrong.txt renamed to sample-pictures/wrong-format-file.txt

File renamed without changes.

‎setup/terraform/main.tf‎

Lines changed: 126 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
terraform {
22
required_providers {
33
aws = {
4-
source = "hashicorp/aws"
4+
source = "hashicorp/aws"
55
version = ">= 4.52.0"
66
}
77
}
@@ -10,15 +10,18 @@ provider "aws" {
1010
region = "eu-central-1"
1111
}
1212

13-
14-
resource "aws_s3_bucket" "shipment-list-demo-bucket" {
15-
bucket = "shipment-list-demo-bucket"
13+
resource "aws_s3_bucket" "shipment_picture_bucket" {
14+
bucket = "shipment-picture-bucket"
15+
force_destroy = true
16+
lifecycle {
17+
prevent_destroy = false
18+
}
1619
}
1720

1821

19-
resource "aws_s3_bucket_acl" "shipment-list-demo-bucket-acl" {
20-
bucket = aws_s3_bucket.shipment-list-demo-bucket.id
21-
acl = "private"
22+
resource "aws_s3_bucket_acl" "shipment_picture_bucket_acl" {
23+
bucket = aws_s3_bucket.shipment_picture_bucket.id
24+
acl = "private"
2225
}
2326

2427
resource "aws_dynamodb_table" "shipment" {
@@ -32,18 +35,127 @@ resource "aws_dynamodb_table" "shipment" {
3235
}
3336
hash_key = "shipmentId"
3437
server_side_encryption {
35-
enabled = true
36-
}
38+
enabled = true
39+
}
3740

38-
stream_enabled = true
39-
stream_view_type = "NEW_AND_OLD_IMAGES"
41+
stream_enabled = true
42+
stream_view_type = "NEW_AND_OLD_IMAGES"
4043
}
4144

42-
4345
resource "aws_dynamodb_table_item" "shipment" {
44-
for_each = local.tf_data
46+
for_each = local.tf_data
4547
table_name = aws_dynamodb_table.shipment.name
4648
hash_key = "shipmentId"
47-
item = jsonencode(each.value)
49+
item = jsonencode(each.value)
50+
}
51+
52+
53+
resource "aws_s3_bucket" "lambda_code_bucket" {
54+
bucket = "shipment-picture-lambda-validator-bucket"
55+
force_destroy = true
56+
lifecycle {
57+
prevent_destroy = false
58+
}
59+
}
60+
61+
resource "aws_s3_bucket_acl" "lambda_code_bucket_acl" {
62+
bucket = aws_s3_bucket.lambda_code_bucket.id
63+
acl = "private"
64+
}
65+
66+
resource "aws_s3_bucket_object" "lambda_code" {
67+
source = "../../shipment-picture-lambda-validator/target/shipment-picture-lambda-validator-1.0-SNAPSHOT.jar"
68+
bucket = aws_s3_bucket.lambda_code_bucket.id
69+
key = "shipment-picture-lambda-validator-1.0-SNAPSHOT.jar"
70+
}
71+
72+
resource "aws_lambda_function" "shipment_picture_lambda_validator" {
73+
function_name = "shipment-picture-lambda-validator"
74+
handler = "dev.ancaghenade.shipmentpicturelambdavalidator.ServiceHandler::handleRequest"
75+
runtime = "java11"
76+
role = aws_iam_role.lambda_exec.arn
77+
s3_bucket = aws_s3_bucket.lambda_code_bucket.id
78+
s3_key = aws_s3_bucket_object.lambda_code.key
79+
memory_size = 512
80+
timeout = 15
81+
environment {
82+
variables = {
83+
ENVIRONMENT = var.env
84+
}
85+
}
4886
}
4987

88+
resource "aws_s3_bucket_notification" "demo_bucket_notification" {
89+
bucket = aws_s3_bucket.shipment_picture_bucket.id
90+
lambda_function {
91+
lambda_function_arn = aws_lambda_function.shipment_picture_lambda_validator.arn
92+
events = ["s3:ObjectCreated:*"]
93+
}
94+
}
95+
96+
resource "aws_lambda_permission" "s3_lambda_exec_permission" {
97+
statement_id = "AllowExecutionFromS3Bucket"
98+
action = "lambda:InvokeFunction"
99+
function_name = aws_lambda_function.shipment_picture_lambda_validator.function_name
100+
principal = "s3.amazonaws.com"
101+
source_arn = aws_s3_bucket.shipment_picture_bucket.arn
102+
}
103+
104+
resource "aws_iam_role" "lambda_exec" {
105+
name = "lambda_exec_role"
106+
107+
assume_role_policy = <<EOF
108+
{
109+
"Version": "2012-10-17",
110+
"Statement": [
111+
{
112+
"Effect": "Allow",
113+
"Principal": {
114+
"Service": "lambda.amazonaws.com"
115+
},
116+
"Action": "sts:AssumeRole"
117+
}
118+
]
119+
}
120+
EOF
121+
}
122+
123+
resource "aws_iam_role_policy_attachment" "lambda_exec_policy" {
124+
policy_arn = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
125+
role = aws_iam_role.lambda_exec.name
126+
}
127+
128+
resource "aws_iam_role_policy" "lambda_exec_policy" {
129+
name = "lambda_exec_policy"
130+
role = aws_iam_role.lambda_exec.id
131+
132+
policy = <<EOF
133+
{
134+
"Version": "2012-10-17",
135+
"Statement": [
136+
{
137+
"Effect": "Allow",
138+
"Action": [
139+
"logs:CreateLogGroup",
140+
"logs:CreateLogStream",
141+
"logs:PutLogEvents"
142+
],
143+
"Resource": "arn:aws:logs:*:*:*"
144+
},
145+
{
146+
"Effect": "Allow",
147+
"Action": [
148+
"s3:GetObject",
149+
"s3:PutObject"
150+
],
151+
"Resource": [
152+
"arn:aws:s3:::shipment-picture-bucket",
153+
"arn:aws:s3:::shipment-picture-bucket/*"
154+
]
155+
}
156+
]
157+
}
158+
EOF
159+
}
160+
161+

‎setup/terraform/vars.tf‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
variable "env" {
2+
type = string
3+
description = "dev env"
4+
default = ""
5+
}

‎src/main/java/dev/ancaghenade/shipmentlistdemo/buckets/BucketName.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
@Getter
66
public enum BucketName {
77

8-
SHIPMENT_PICTURE("shipment-list-demo-bucket");
8+
SHIPMENT_PICTURE("shipment-picture-bucket");
99

1010
private final String bucketName;
1111

0 commit comments

Comments
 (0)