-add better behaviour for non-compliant files

-add update on file placeholder
-tweak clients
-add new diagram
-adapt README file

Author: tinyg210

README.md

@@ -40,10 +40,12 @@ The AWS services involved are:
 
 - [S3](https://docs.localstack.cloud/user-guide/aws/s3/) for storing pictures
 - [DynamoDB](https://docs.localstack.cloud/user-guide/aws/dynamodb/) for the entities
-- [Lambda](https://docs.localstack.cloud/user-guide/aws/lambda/) function that will validate the pictures.
+- [Lambda](https://docs.localstack.cloud/user-guide/aws/lambda/) function that will validate the pictures, apply a watermark and replace non-compliant files.
+- [SNS](https://docs.localstack.cloud/user-guide/aws/sns/) that receives update notifications
+- [SQS](https://docs.localstack.cloud/user-guide/aws/lambda/) that subscribes to a topic and delivers the messages to the Spring Boot app
 
 
-#### How we will be using it
+#### How to use it
 
 We’ll be walking through a few scenarios using the application, and we expect it to maintain the
 behavior in both production and development environments. This behaviour can be "scientifically"
@@ -57,7 +59,7 @@ files, `application-prod.yml`, and  `application-dev.yml`.
 
 ## Instructions
 
-## Production simulation (Running on AWS)
+## Running on AWS
 
 Now, we don’t have a real production environment because that’s not the point here, but most likely,
 an application like this runs on a container orchestration platform, and all the necessary configs
@@ -72,6 +74,10 @@ needs to be created with the following policies:
 - AmazonS3FullAccess
 - AWSLambda_FullAccess
 - AmazonDynamoDBFullAccess
+- AmazonSNSFullAccess
+- AmazonSQSFullAccess
+- AWSLambdaExecute
+- AmazonS3ObjectLambdaExecutionRolePolicy
 
 We will be using the user's credentials and export them as temporary environment variables with the
 `export` (`set` on Windows) command:
@@ -89,10 +95,7 @@ step.
 
 ### Creating resources - running Terraform
 
-Make sure you have Terraform [installed](https://developer.hashicorp.com/terraform/downloads).If
-you're
-not familiar or uncomfortable with Terraform, there's also a branch that uses only AWS cli to create
-resources.
+Make sure you have Terraform [installed](https://developer.hashicorp.com/terraform/downloads)
 
 Under setup/terraform run:
 
@@ -108,7 +111,7 @@ $ terraform apply
 ```
 
 This should create the needed S3 bucket, the DynamoDB `shipment` table and populate it with some
-sample data, and the Lambda function that will help with picture validation.
+sample data, the Lambda function that will help with picture validation, the SQS and SNS.
 
 ### Running the GUI
 
@@ -134,8 +137,8 @@ At `localhost:3000` you should now be able to see a list of shipments with stand
 that means that only the database is populated, the pictures still need to be added from the
 `sample-pictures` folder.
 
-The weight of a shipment we can perceive, but not the size, that's why we need pictures to
-understand,
+The weight of a shipment is already given, but not the size, that's why we need pictures to
+understand it better,
 using the "banana for scale" measuring unit. How else would we know??
 
 Current available actions using the GUI:
@@ -154,6 +157,12 @@ To switch to using LocalStack instead of AWS services just run `docker compose u
 folder
 to spin up a Localstack container.
 
+Before we proceed, make sure you clean up your AWS resources by running
+
+```
+$ terraform destroy
+```
+
 To generate the exact same resources on LocalStack, we need `tflocal`, a thin wrapper script around
 the terraform command line client. `tflocal` takes care of automatically configuring the local
 service
@@ -175,14 +184,13 @@ Usage: terraform [global options] <subcommand> [args]
 ...
 ```
 
-From here on, it's smooth sailing, the same as before. Switch to `setup/tflocal` folder, the files are 
-identical to the ones in `setup/terraform`, but for the newly generated state files, it is a good idea
-to separate these "workspaces":
+From here on, it's smooth sailing, the same as before. In the `setup/terraform` folder, run the `cleanup` script
+to get rid of any files that keep track of the resources' state. Then:
 
 ```
 $ tflocal init
 $ tflocal plan -var 'env=dev
-$ tflocal apply -var 'env=dev'
+$ tflocal apply
 ```
 
 What we're doing here is just passing an environmental variable to let the Lambda

app_diagram.png

@@ -1,28 +1,18 @@
 version: "3.9"
 
 services:
-  ping:
-    image: alpine:3.14
-    container_name: ping
-    tty: true
-    ports:
-      - "127.0.0.1:8081:8081"
-    volumes:
-      - "/var/run/docker.sock:/var/run/docker.sock"
-    environment:
-      - DOCKER_HOST=unix:///var/run/docker.sock
   localstack:
     container_name: localstack
     image: localstack/localstack:latest
     ports:
       - "127.0.0.1:4566:4566"            # LocalStack Gateway
       - "127.0.0.1:4510-4559:4510-4559"  # external services port range
     environment:
-      - DEBUG=1  # enable more verbose logs
+#      - DEBUG=1  # enable more verbose logs
       - DOCKER_HOST=unix:///var/run/docker.sock #unix socket to communicate with the docker daemon
 #      - LAMBDA_KEEPALIVE_MS=0 # disable lambda keepalive
       - LOCALSTACK_HOST=localstack # where services are available from other containers
-#      - ENFORCE_IAM=1 # enforce IAM policies
+      - ENFORCE_IAM=1 # enforce IAM policies
     volumes:
       - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
       - "/var/run/docker.sock:/var/run/docker.sock"

docker-compose.yml

@@ -26,7 +26,6 @@
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
   </properties>
 
-
   <!--    Dependencies-->
   <dependencies>
     <dependency>
@@ -42,15 +41,6 @@
       <groupId>software.amazon.awssdk</groupId>
       <artifactId>s3</artifactId>
     </dependency>
-    <dependency>
-      <groupId>software.amazon.awssdk</groupId>
-      <artifactId>sqs</artifactId>
-    </dependency>
-
-    <dependency>
-      <groupId>software.amazon.awssdk</groupId>
-      <artifactId>sns</artifactId>
-    </dependency>
     <dependency>
       <groupId>software.amazon.awssdk</groupId>
       <artifactId>dynamodb-enhanced</artifactId>
@@ -69,17 +59,7 @@
     </dependency>
     <dependency>
       <groupId>io.awspring.cloud</groupId>
-      <artifactId>spring-cloud-starter-aws-messaging</artifactId>
-    </dependency>
-    <dependency>
-      <groupId>org.springframework.boot</groupId>
-      <artifactId>spring-boot-starter-websocket</artifactId>
-      <version>3.0.4</version>
-    </dependency>
-    <dependency>
-      <groupId>io.projectreactor</groupId>
-      <artifactId>reactor-core</artifactId>
-      <version>3.5.4</version>
+      <artifactId>spring-cloud-aws-starter-sqs</artifactId>
     </dependency>
 
 
@@ -109,7 +89,7 @@
       <dependency>
         <groupId>io.awspring.cloud</groupId>
         <artifactId>spring-cloud-aws-dependencies</artifactId>
-        <version>2.3.1</version>
+        <version>3.0.0-RC1</version>
         <type>pom</type>
         <scope>import</scope>
       </dependency>

pom.xml

@@ -0,0 +1,4 @@
+rm .terraform.lock.hcl
+rm -rf .terraform
+rm terraform.tfstate
+rm terraform.tfstate.backup

setup/terraform/cleanup.sh

@@ -2,7 +2,7 @@ terraform {
   required_providers {
     aws = {
       source  = "hashicorp/aws"
-      version = ">= 4.52.0"
+      version = "= 4.66.1"
     }
   }
 }
@@ -90,16 +90,6 @@ resource "aws_lambda_permission" "s3_lambda_exec_permission" {
   source_arn    = aws_s3_bucket.shipment_picture_bucket.arn
 }
 
-resource "aws_sns_topic" "update_shipment_picture_topic" {
-  name = "update_shipment_picture_topic"
-}
-
-
-resource "aws_sns_topic_subscription" "example_subscription" {
-  topic_arn = aws_sns_topic.update_shipment_picture_topic.arn
-  protocol  = "https"
-  endpoint  = var.sns_sub_endpoint
-}
 
 resource "aws_iam_role" "lambda_exec" {
   name = "lambda_exec_role"
@@ -126,7 +116,6 @@ resource "aws_iam_role_policy_attachment" "lambda_exec_policy" {
 }
 
 
-
 resource "aws_iam_role_policy" "lambda_exec_policy" {
   name = "lambda_exec_policy"
   role = aws_iam_role.lambda_exec.id
@@ -162,4 +151,56 @@ resource "aws_iam_role_policy" "lambda_exec_policy" {
           EOF
 }
 
+resource "aws_sns_topic" "update_shipment_picture_topic" {
+  name = "update_shipment_picture_topic"
+}
+
+resource "aws_sqs_queue" "update_shipment_picture_queue" {
+  name = "update_shipment_picture_queue"
+}
+
+resource "aws_sns_topic_subscription" "my_subscription" {
+  topic_arn = aws_sns_topic.update_shipment_picture_topic.arn
+  protocol  = "sqs"
+  endpoint  = aws_sqs_queue.update_shipment_picture_queue.arn
+}
+
+resource "aws_sqs_queue_policy" "my_queue_policy" {
+  queue_url = aws_sqs_queue.update_shipment_picture_queue.id
+
+  policy = <<EOF
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Sid": "AllowSNSSendMessage",
+      "Effect": "Allow",
+      "Principal": "*",
+      "Action": "sqs:SendMessage",
+      "Resource": "${aws_sqs_queue.update_shipment_picture_queue.arn}",
+      "Condition": {
+        "ArnEquals": {
+          "aws:SourceArn": "${aws_sns_topic.update_shipment_picture_topic.arn}"
+        }
+      }
+    }
+  ]
+}
+EOF
+}
+
+resource "aws_sns_topic_subscription" "my_topic_subscription" {
+  topic_arn = aws_sns_topic.update_shipment_picture_topic.arn
+  protocol  = "sqs"
+  endpoint  = aws_sqs_queue.update_shipment_picture_queue.arn
+
+  # Additional subscription attributes
+  raw_message_delivery = true
+  filter_policy        = ""
+  delivery_policy      = ""
+
+  # Ensure the subscription is confirmed automatically
+  confirmation_timeout_in_minutes = 1
+}
+