Skip to content

Update ses delivery and bounce notifications to handle weekly deliveries #1981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SamJamCul merged 3 commits into from
Mar 25, 2026
Merged

Update ses delivery and bounce notifications to handle weekly deliveries #1981

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d363abd
Handle weekly batch bounces and complaints
SamJamCul Mar 19, 2026
b41db06
Update rake tasks for weekly batch deliveries
SamJamCul Mar 20, 2026
07beb61
Handle weekly batch receiving submissions
SamJamCul Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions app/jobs/application_job.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,7 @@ def set_submission_batch_logging_attributes(form:, mode:, delivery:)
CurrentJobLoggingAttributes.preview = mode.preview?
CurrentJobLoggingAttributes.delivery_id = delivery.id
CurrentJobLoggingAttributes.delivery_reference = delivery.delivery_reference
CurrentJobLoggingAttributes.delivery_schedule = delivery.delivery_schedule
CurrentJobLoggingAttributes.batch_begin_at = delivery.batch_begin_at
end
end
34 changes: 30 additions & 4 deletions app/jobs/receive_submission_bounces_and_complaints_job.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
def set_logging_attributes(submission:, delivery:)
if delivery.immediate?
set_submission_logging_attributes(submission:, delivery:)
elsif delivery.daily?
elsif delivery.daily? || delivery.weekly?
set_submission_batch_logging_attributes(form: submission.form, mode: submission.mode_object, delivery:)
end
end
Expand Down Expand Up @@ -66,8 +66,14 @@
}
end

process_immediate_delivery_bounce(delivery, submission, ses_bounce, bounced_recipients) if delivery.immediate?
process_daily_delivery_bounce(delivery, submission, ses_bounce, bounced_recipients) if delivery.daily?
case delivery.delivery_schedule

Check failure on line 69 in app/jobs/receive_submission_bounces_and_complaints_job.rb

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default clause to this "case" statement. 


See more on https://sonarcloud.io/project/issues?id=alphagov_forms-runner&issues=AZ0KgTcOIpZiw5iiLnPc&open=AZ0KgTcOIpZiw5iiLnPc&pullRequest=1981
when "immediate"
process_immediate_delivery_bounce(delivery, submission, ses_bounce, bounced_recipients)
when "daily"
process_daily_delivery_bounce(delivery, submission, ses_bounce, bounced_recipients)
when "weekly"
process_weekly_delivery_bounce(delivery, submission, ses_bounce, bounced_recipients)
end
end

def process_immediate_delivery_bounce(delivery, submission, ses_bounce, bounced_recipients)
Expand All @@ -75,7 +81,7 @@

unless submission.preview?
Sentry.capture_message("Submission email bounced for form #{submission.form_id} - #{self.class.name}:",
fingerprint: ["{{ default }}", submission.form_id],

Check failure on line 84 in app/jobs/receive_submission_bounces_and_complaints_job.rb

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal "{{ default }}" 3 times. 


See more on https://sonarcloud.io/project/issues?id=alphagov_forms-runner&issues=AZ0KgTcOIpZiw5iiLnPb&open=AZ0KgTcOIpZiw5iiLnPb&pullRequest=1981
extra: {
form_id: submission.form_id,
submission_reference: submission.reference,
Expand All @@ -96,9 +102,27 @@
extra: {
form_id: submission.form_id,
delivery_reference: delivery.delivery_reference,
delivery_id: delivery.id,
delivery_schedule: delivery.delivery_schedule,
batch_begin_at: delivery.batch_begin_at,
job_id:,
ses_bounce:,
})
end
end

def process_weekly_delivery_bounce(delivery, submission, ses_bounce, bounced_recipients)
EventLogger.log_form_event("weekly_batch_email_bounced", ses_bounce: ses_bounce.merge(bounced_recipients:))

unless submission.preview?
Sentry.capture_message("Weekly submission batch email bounced for form #{submission.form_id} - #{self.class.name}:",
fingerprint: ["{{ default }}", submission.form_id],
extra: {
form_id: submission.form_id,
delivery_reference: delivery.delivery_reference,
delivery_id: delivery.id,
batch_date: submission.submission_time.to_date,
delivery_schedule: delivery.delivery_schedule,
batch_begin_at: delivery.batch_begin_at,
job_id:,
ses_bounce:,
})
Expand All @@ -110,6 +134,8 @@
EventLogger.log_form_event("submission_complaint")
elsif delivery.daily?
EventLogger.log_form_event("daily_batch_email_complaint")
elsif delivery.weekly?
EventLogger.log_form_event("weekly_batch_email_complaint")
end
end
end
2 changes: 1 addition & 1 deletion app/jobs/receive_submission_deliveries_job.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ def perform

def process_delivery(delivery, submission, **attributes)
set_submission_logging_attributes(submission:, delivery:) if delivery.immediate?
set_submission_batch_logging_attributes(form: submission.form, mode: submission.mode_object, delivery:) if delivery.daily?
set_submission_batch_logging_attributes(form: submission.form, mode: submission.mode_object, delivery:) if delivery.daily? || delivery.weekly?

delivery.update! attributes

Expand Down
4 changes: 4 additions & 0 deletions app/models/current_job_logging_attributes.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ class CurrentJobLoggingAttributes < ActiveSupport::CurrentAttributes
:preview,
:delivery_id,
:delivery_reference,
:delivery_schedule,
:batch_begin_at,
:confirmation_email_id,
:sqs_message_id,
:sns_message_timestamp
Expand All @@ -21,6 +23,8 @@ def as_hash
preview: preview.to_s,
delivery_id:,
delivery_reference:,
delivery_schedule:,
batch_begin_at:,
confirmation_email_id:,
sqs_message_id:,
sns_message_timestamp:,
Expand Down
21 changes: 14 additions & 7 deletions lib/tasks/submissions.rake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ namespace :submissions do
deliveries.daily.each do |delivery|
Rails.logger.info "Daily batch delivery - delivery_reference: #{delivery.delivery_reference}, created_at: #{delivery.created_at}, last_attempt_at: #{delivery.last_attempt_at}"
end
deliveries.weekly.each do |delivery|
Rails.logger.info "Weekly batch delivery - delivery_reference: #{delivery.delivery_reference}, created_at: #{delivery.created_at}, last_attempt_at: #{delivery.last_attempt_at}"
end
end

desc "Fetch and display all data for a specific submission given a reference"
Expand Down Expand Up @@ -52,7 +55,10 @@ namespace :submissions do
Rails.logger.info "Retrying submission with reference #{submission.reference} for form with ID: #{form_id}"
SendSubmissionJob.perform_later(submission)
elsif delivery.daily?
Rails.logger.info "Retrying daily batch delivery with delivery_id: #{delivery.id} for date: #{submission.submission_time.to_date} for form with ID: #{form_id}"
Rails.logger.info "Retrying daily batch delivery with delivery_id: #{delivery.id} for date: #{delivery.batch_begin_at.to_date} for form with ID: #{form_id}"
SendSubmissionBatchJob.perform_later(delivery:)
elsif delivery.weekly?
Rails.logger.info "Retrying weekly batch delivery with delivery_id: #{delivery.id} for week starting: #{delivery.batch_begin_at.to_date} for form with ID: #{form_id}"
SendSubmissionBatchJob.perform_later(delivery:)
end
end
Expand Down Expand Up @@ -172,15 +178,15 @@ namespace :submissions do
end
end

desc "Re-deliver daily submission batches corresponding to submissions between two timestamps for a specific form"
task :redeliver_daily_batches_by_date, %i[form_id start_timestamp end_timestamp dry_run] => :environment do |_, args|
desc "Re-deliver submission batch deliveries created between two timestamps for a specific form"
task :redeliver_batches_by_date, %i[form_id start_timestamp end_timestamp dry_run] => :environment do |_, args|
form_id = args[:form_id]
start_timestamp = args[:start_timestamp]
end_timestamp = args[:end_timestamp]
dry_run_arg = args[:dry_run]
dry_run = dry_run_arg == "true"

usage_message = "usage: rake submissions:redeliver_daily_batches_by_date[<form_id>,<start_timestamp>,<end_timestamp>,<dry_run>]".freeze
usage_message = "usage: rake submissions:redeliver_batches_by_date[<form_id>,<start_timestamp>,<end_timestamp>,<dry_run>]".freeze

if form_id.blank? || start_timestamp.blank? || end_timestamp.blank? || !dry_run_arg.in?(%w[true false])
abort usage_message
Expand All @@ -197,9 +203,10 @@ namespace :submissions do
abort "Error: Start timestamp must be before end timestamp"
end

deliveries_to_redeliver = Delivery.daily
.joins(:submissions)
.where(submissions: { form_id: form_id, created_at: start_time..end_time })
deliveries_to_redeliver = Delivery.joins(:submissions)
.where(submissions: { form_id: form_id })
.where.not(delivery_schedule: "immediate")
.where(created_at: start_time..end_time)
.distinct

Rails.logger.info "Time range: #{start_time} to #{end_time}"
Expand Down
6 changes: 6 additions & 0 deletions spec/factories/deliveries.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@

trait :daily_scheduled_delivery do
delivery_schedule { "daily" }
batch_begin_at { created_at.beginning_of_day }
end

trait :weekly_scheduled_delivery do
delivery_schedule { "weekly" }
batch_begin_at { (created_at - 7.days).beginning_of_day }
end
end
end
Loading
Loading