Skip to content

feat: notifications v3 [ON HOLD]#1137

Open
thiessenp-cds wants to merge 77 commits intomainfrom
feat/notifications-v3
Open

feat: notifications v3 [ON HOLD]#1137
thiessenp-cds wants to merge 77 commits intomainfrom
feat/notifications-v3

Conversation

@thiessenp-cds
Copy link
Copy Markdown
Contributor

@thiessenp-cds thiessenp-cds commented Oct 27, 2025

Summary | Résumé

This PR adds push notifications for GCForms. There are two types of notifications: immediate and deferred. An immediate notification creates a record and immediately queues up Notify to send an email. A deferred notification is first created from the client and after a process is completed the notification is queued up to send.

The utility functions can be found in the @gcforms/connector package: notification.sendImmediate(), notification.sendDeferred() and notification.enqueueDeferred().

Testing

The initial testing before merging can only be done by a dev locally.

Infra:

  • locally check out this branch
  • make build_env (will fail)
  • make lambdas
  • Open your AWS scratch account and go to Amazon ECR > Private registry > Repositories
  • find notification-lambda in the list and copy the URI
  • Update notification.tf to point to the scratch account image e.g. image_uri = "[YOUR_ACCOUNT_ID].dkr.ecr.ca-central-1.amazonaws.com/notification-lambda:latest"
  • Do the same for reliability in reliability.tf
  • make build_env
  • make connect_env

Client:

  • locally check out branch feat/notification-v3
  • yarn dev

Setup Manual test:

  • Open your AWS scratch account
  • go to Cloudwatch > Live Tail. Filter by groups: notification and reliability
  • Open GCForms

Test a Deferred Message:

  • Open a draft or published form and create 3 submissions
  • in the CloudWatch logs Only 2 notification logs should have been created

Test a Immediate Message:

  • Choose any component and wire up a button to call
    e.g.
  const handleSendImmediateNotification = async () => {
    "use server";
    notification.sendImmediate({
      emails: ["test1@cds-snc.ca", "test2@cds-snc.ca"],
      subject: "Test Notification",
      body: "This is a test notification sent from the app." 
    });
  }
  ...
  <button onClick={handleSendNotification}>Send Immediate Notification</button>
  • Click the button
  • CloudWatch logs should show a Notification sent message

Pre Release TODOs

Staging

  • add back Notify call to send emails in emails.tf

Production

  • add notification lambda to production-lambda-functions.conf

@thiessenp-cds thiessenp-cds linked an issue Oct 27, 2025 that may be closed by this pull request
@thiessenp-cds thiessenp-cds changed the title Notifications v3 Notifications v3 (work in progress) Oct 27, 2025
@thiessenp-cds thiessenp-cds changed the title Notifications v3 (work in progress) feat: notifications DB Nov 24, 2025
@thiessenp-cds thiessenp-cds changed the title feat: notifications DB feat: notifications v3 [WIP] Nov 24, 2025
@github-actions
Copy link
Copy Markdown

⚠ Terrform update available

Terraform: 1.14.8 (using 1.12.2)
Terragrunt: 0.99.5 (using 0.83.0)

@github-actions
Copy link
Copy Markdown

Staging: ecr

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 2 to add, 0 to change, 0 to destroy
Show summary
CHANGE NAME
add aws_ecr_lifecycle_policy.lambda[&quot;notification-lambda&quot;]
aws_ecr_repository.lambda[&quot;notification-lambda&quot;]
Show plan
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_ecr_lifecycle_policy.lambda["notification-lambda"] will be created
  + resource "aws_ecr_lifecycle_policy" "lambda" {
      + id          = (known after apply)
      + policy      = jsonencode(
            {
              + rules = [
                  + {
                      + action       = {
                          + type = "expire"
                        }
                      + description  = "Keep last 10 images"
                      + rulePriority = 1
                      + selection    = {
                          + countNumber = 10
                          + countType   = "imageCountMoreThan"
                          + tagStatus   = "any"
                        }
                    },
                ]
            }
        )
      + region      = "ca-central-1"
      + registry_id = (known after apply)
      + repository  = "notification-lambda"
    }

  # aws_ecr_repository.lambda["notification-lambda"] will be created
  + resource "aws_ecr_repository" "lambda" {
      + arn                  = (known after apply)
      + force_delete         = false
      + id                   = (known after apply)
      + image_tag_mutability = "MUTABLE"
      + name                 = "notification-lambda"
      + region               = "ca-central-1"
      + registry_id          = (known after apply)
      + repository_url       = (known after apply)
      + tags_all             = {
          + "CostCentre" = "forms-platform-staging"
          + "Terraform"  = "true"
        }

      + image_scanning_configuration {
          + scan_on_push = true
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  ~ ecr_repository_lambda_urls                     = {
      + notification-lambda             = (known after apply)
        # (17 unchanged attributes hidden)
    }

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.api"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.forms_app_legacy_repository"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.idp"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.idp_user_portal"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"api-end-to-end-test-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"audit-logs-archiver-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"audit-logs-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"cognito-email-sender-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"cognito-pre-sign-up-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"file-upload-cleanup-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"file-upload-processor-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"form-archiver-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"load-testing-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"nagware-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"notification-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"notify-slack-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"prisma-migration-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"reliability-dlq-consumer-lambda\"]"]
WARN - plan.json - main - Missing Common Tags: ["aws_ecr_repository.lambda[\"reliability-lambda\"]"]
WARN - plan.json - main - Missing Common Tags:...

@github-actions
Copy link
Copy Markdown

Staging: sqs

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 1 to add, 0 to change, 0 to destroy
Show summary
CHANGE NAME
add aws_sqs_queue.notification_queue
Show plan
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_sqs_queue.notification_queue will be created
  + resource "aws_sqs_queue" "notification_queue" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = 300
      + kms_master_key_id                 = "alias/aws/sqs"
      + max_message_size                  = 262144
      + message_retention_seconds         = 86400
      + name                              = "notification_queue"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + region                            = "ca-central-1"
      + sqs_managed_sse_enabled           = (known after apply)
      + tags_all                          = {
          + "CostCentre" = "forms-platform-staging"
          + "Terraform"  = "true"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 1800
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + sqs_notification_queue_arn              = (known after apply)
  + sqs_notification_queue_url              = (known after apply)

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.api_audit_log_deadletter_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.api_audit_log_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.audit_log_deadletter_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.audit_log_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.file_upload_deadletter_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.file_upload_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.notification_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.reliability_deadletter_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.reliability_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_sqs_queue.reliability_reprocessing_queue"]

29 tests, 19 passed, 10 warnings, 0 failures, 0 exceptions

@github-actions
Copy link
Copy Markdown

Staging: dynamodb

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 1 to add, 0 to change, 0 to destroy
Show summary
CHANGE NAME
add aws_dynamodb_table.notification
Show plan
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_dynamodb_table.notification will be created
  + resource "aws_dynamodb_table" "notification" {
      + arn                         = (known after apply)
      + billing_mode                = "PAY_PER_REQUEST"
      + deletion_protection_enabled = true
      + hash_key                    = "NotificationID"
      + id                          = (known after apply)
      + name                        = "Notification"
      + read_capacity               = (known after apply)
      + region                      = "ca-central-1"
      + stream_arn                  = (known after apply)
      + stream_label                = (known after apply)
      + stream_view_type            = (known after apply)
      + tags_all                    = {
          + "CostCentre" = "forms-platform-staging"
          + "Terraform"  = "true"
        }
      + write_capacity              = (known after apply)

      + attribute {
          + name = "NotificationID"
          + type = "S"
        }

      + point_in_time_recovery {
          + enabled                 = true
          + recovery_period_in_days = (known after apply)
        }

      + server_side_encryption {
          + enabled     = true
          + kms_key_arn = "arn:aws:kms:ca-central-1:687401027353:key/1f3edb85-9eac-4da9-8c7c-43a68e339ede"
        }

      + ttl {
          + attribute_name = "TTL"
          + enabled        = true
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + dynamodb_notification_table_arn    = (known after apply)
  + dynamodb_notification_table_name   = "Notification"

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.api_audit_logs"]
WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.audit_logs"]
WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.notification"]
WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.reliability_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.vault"]

24 tests, 19 passed, 5 warnings, 0 failures, 0 exceptions

@github-actions
Copy link
Copy Markdown

Staging: lambdas

✅   Terraform Init: success
✅   Terraform Validate: success
❌   Terraform Format: failed
✅   Terraform Plan: success
✅   Conftest: success

🧹   Format: run terraform fmt to fix the following:

reliability.tf
Plan: 3 to add, 2 to change, 0 to destroy
Show summary
CHANGE NAME
add aws_cloudwatch_log_group.notification
aws_lambda_event_source_mapping.notification_sqs
aws_lambda_function.notification
update aws_iam_policy.lambda_dynamodb
aws_lambda_function.reliability
Show plan
Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

Terraform will perform the following actions:

  # aws_cloudwatch_log_group.notification will be created
  + resource "aws_cloudwatch_log_group" "notification" {
      + arn               = (known after apply)
      + id                = (known after apply)
      + kms_key_id        = "arn:aws:kms:ca-central-1:687401027353:key/c5c2a1c2-c092-4fa1-8daf-3414f3511b1d"
      + log_group_class   = (known after apply)
      + name              = "/aws/lambda/Notification"
      + name_prefix       = (known after apply)
      + region            = "ca-central-1"
      + retention_in_days = 731
      + skip_destroy      = false
      + tags_all          = {
          + "CostCentre" = "forms-platform-staging"
          + "Terraform"  = "true"
        }
    }

  # aws_iam_policy.lambda_dynamodb will be updated in-place
  ~ resource "aws_iam_policy" "lambda_dynamodb" {
        id               = "arn:aws:iam::687401027353:policy/lambda_dynamobdb"
        name             = "lambda_dynamobdb"
      ~ policy           = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                      ~ Resource = [
                            # (4 unchanged elements hidden)
                            "arn:aws:dynamodb:ca-central-1:687401027353:table/ReliabilityQueue",
                          + "arn:aws:dynamodb:ca-central-1:687401027353:table/Notification/index/*",
                          + "arn:aws:dynamodb:ca-central-1:687401027353:table/Notification",
                            "arn:aws:dynamodb:ca-central-1:687401027353:table/AuditLogs/index/*",
                            # (3 unchanged elements hidden)
                        ]
                        # (2 unchanged attributes hidden)
                    },
                ]
                # (1 unchanged attribute hidden)
            }
        )
        tags             = {}
        # (7 unchanged attributes hidden)
    }

  # aws_lambda_event_source_mapping.notification_sqs will be created
  + resource "aws_lambda_event_source_mapping" "notification_sqs" {
      + arn                           = (known after apply)
      + batch_size                    = 10
      + enabled                       = true
      + event_source_arn              = "arn:aws:sqs:ca-central-1:687401027353:notification_queue"
      + function_arn                  = (known after apply)
      + function_name                 = "notification"
      + function_response_types       = [
          + "ReportBatchItemFailures",
        ]
      + id                            = (known after apply)
      + last_modified                 = (known after apply)
      + last_processing_result        = (known after apply)
      + maximum_record_age_in_seconds = (known after apply)
      + maximum_retry_attempts        = (known after apply)
      + parallelization_factor        = (known after apply)
      + region                        = "ca-central-1"
      + state                         = (known after apply)
      + state_transition_reason       = (known after apply)
      + tags_all                      = {
          + "CostCentre" = "forms-platform-staging"
          + "Terraform"  = "true"
        }
      + uuid                          = (known after apply)

      + amazon_managed_kafka_event_source_config (known after apply)

      + self_managed_kafka_event_source_config (known after apply)
    }

  # aws_lambda_function.notification will be created
  + resource "aws_lambda_function" "notification" {
      + architectures                  = (known after apply)
      + arn                            = (known after apply)
      + code_sha256                    = (known after apply)
      + function_name                  = "notification"
      + id                             = (known after apply)
      + image_uri                      = "test_url:latest"
      + invoke_arn                     = (known after apply)
      + last_modified                  = (known after apply)
      + memory_size                    = 512
      + package_type                   = "Image"
      + publish                        = false
      + qualified_arn                  = (known after apply)
      + qualified_invoke_arn           = (known after apply)
      + region                         = "ca-central-1"
      + reserved_concurrent_executions = -1
      + role                           = "arn:aws:iam::687401027353:role/iam_for_lambda"
      + signing_job_arn                = (known after apply)
      + signing_profile_version_arn    = (known after apply)
      + skip_destroy                   = false
      + source_code_hash               = (known after apply)
      + source_code_size               = (known after apply)
      + tags_all                       = {
          + "CostCentre" = "forms-platform-staging"
          + "Terraform"  = "true"
        }
      + timeout                        = 300
      + version                        = (known after apply)

      + environment {
          + variables = {
              + "DYNAMODB_NOTIFICATION_TABLE_NAME" = "Notification"
              + "NOTIFY_API_KEY"                   = (sensitive value)
              + "REGION"                           = "ca-central-1"
            }
        }

      + ephemeral_storage (known after apply)

      + logging_config {
          + log_format = "Text"
          + log_group  = "/aws/lambda/Notification"
        }

      + tracing_config {
          + mode = "Active"
        }

      + vpc_config {
          + ipv6_allowed_for_dual_stack = false
          + security_group_ids          = [
              + "sg-06651d69cd4d3c50f",
            ]
          + subnet_ids                  = [
              + "subnet-07e38df0760d389d1",
              + "subnet-07f9debd31e48ce64",
              + "subnet-0af8e6e3cf80f582d",
            ]
          + vpc_id                      = (known after apply)
        }
    }

  # aws_lambda_function.reliability will be updated in-place
  ~ resource "aws_lambda_function" "reliability" {
        id                             = "reliability"
        tags                           = {}
        # (29 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              + "NOTIFICATION_QUEUE_URL" = "https://sqs.ca-central-1.amazonaws.com/687401027353/notification_queue"
                # (5 unchanged elements hidden)
            }
        }

        # (4 unchanged blocks hidden)
    }

Plan: 3 to add, 2 to change, 0 to destroy.

Changes to Outputs:
  + lambda_notification_function_name              = "notification"
  + lambda_notification_log_group_name             = "/aws/lambda/Notification"

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.api_end_to_end_test_lambda_trigger"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.audit_logs_archiver_lambda_trigger"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.file_upload_cleanup_trigger"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.form_archiver_lambda_trigger"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.nagware_lambda_trigger"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.reliability_dlq_lambda_trigger"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.response_archiver_lambda_trigger"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.api_end_to_end_test"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.archive_form_templates"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.audit_logs"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.audit_logs_archiver"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.dead_letter_queue_consumer"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.file_upload"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.file_upload_cleanup"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.nagware"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.notification"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.prisma_migration_handler"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.reliability"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.response_archiver"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.submission"]
WARN - plan.json - main - Missing...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Notifications DB

1 participant