Skip to content

Commit 1613a85

Browse files
authored
Move from sync s3 and azure to only use azure for attachments (#8580)
## Trello card URL - https://trello.com/c/UtzBnbCV/2097-move-from-aws-s3-to-azure-storage ## Changes in this PR: - Points the attachments directly to Azure Storage accounts without going through the Mirror (S3 -> Azure). - Removes already run rake tasks for the previous setps of the S3 -> Azure move. - Backfills the existing attachments to point to the Azure Storage ## Checklists: ### Data & Schema Changes If this PR modifies data structures or validations, check the following: - [ ] Adds/removes model validations - [ ] Adds/removes database fields - [ ] Modifies Vacancy enumerables (phases, working patterns, job roles, key stages, etc.) <details> <summary>If any of the above options has changed then the author must check/resolve all of the following...</summary> ### Integration Impact Does this change affect any of these integrations? - [ ] DfE Analytics platform - [ ] Legacy imports mappings - [ ] DWP Find a Job export mappings - [ ] Publisher ATS API (may require mapping updates or API versioning) ### User Experience & Data Integrity Could this change impact: - [ ] Existing subscription alerts (will legacy subscription search filters break?) - [ ] Legacy vacancy copying (will copied vacancies fail new validations?) - [ ] In-progress drafts for Vacancies or Job Applications </details>
1 parent 097d472 commit 1613a85

10 files changed

+209
-464
lines changed

app/models/job_application.rb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ class JobApplication < ApplicationRecord
141141

142142
validates :email_address, email_address: true, if: -> { email_address_changed? } # Allows data created prior to validation to still be valid
143143

144-
has_one_attached :baptism_certificate, service: :mirror_documents
144+
has_one_attached :baptism_certificate, service: :azure_storage_documents
145145

146146
validate :status_transition, if: -> { status_changed? }
147147

app/models/organisation.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ class Organisation < ApplicationRecord
1212

1313
friendly_id :slug_candidates, use: %i[slugged history]
1414

15-
has_one_attached :logo, service: :mirror_images_and_logos
16-
has_one_attached :photo, service: :mirror_images_and_logos
15+
has_one_attached :logo, service: :azure_storage_images_and_logos
16+
has_one_attached :photo, service: :azure_storage_images_and_logos
1717

1818
has_many :organisation_vacancies, dependent: :destroy
1919
has_many :vacancies, through: :organisation_vacancies

app/models/vacancy.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,12 +87,12 @@ class Vacancy < ApplicationRecord
8787
valid_file_types: %i[PDF DOC DOCX],
8888
}.freeze
8989

90-
has_many_attached :supporting_documents, service: :mirror_documents
90+
has_many_attached :supporting_documents, service: :azure_storage_documents
9191

9292
validates :supporting_documents, content_type: DOCUMENT_CONTENT_TYPES,
9393
size: { less_than: DOCUMENT_FILE_SIZE_LIMIT }, virus_free: true, if: -> { include_additional_documents }
9494

95-
has_one_attached :application_form, service: :mirror_documents
95+
has_one_attached :application_form, service: :azure_storage_documents
9696

9797
has_many :saved_jobs, dependent: :destroy
9898
has_many :saved_by, through: :saved_jobs, source: :jobseeker

config/environments/production.rb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
# The application uses multiple services for storing files. This sets up a default value which gets overridden
77
# in every specific use case.
8-
config.active_storage.service = :mirror_documents
8+
config.active_storage.service = :azure_storage_documents
99

1010
# Configure the domains permitted to access coordinates API
1111
config.allowed_cors_origin = proc { "https://#{DOMAIN}" }

lib/tasks/backfill_azure_storage.rake

Lines changed: 0 additions & 61 deletions
This file was deleted.

lib/tasks/find_and_mirror_s3_blobs.rake

Lines changed: 0 additions & 60 deletions
This file was deleted.
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
desc "Point existing attachments from mirror services to Azure storage directly"
2+
3+
task point_existing_attachments_to_azure_storage: :environment do
4+
puts "Starting migration of attachments to Azure storage..."
5+
puts "=" * 80
6+
7+
# Migration mapping: from mirror services to direct Azure services
8+
# This bypasses the mirror service (which has S3 as primary) and points directly to Azure
9+
migrations = {
10+
"mirror_documents" => "azure_storage_documents",
11+
"mirror_images_and_logos" => "azure_storage_images_and_logos",
12+
}
13+
14+
total_updated = 0
15+
16+
migrations.each do |old_service, new_service|
17+
puts "\nMigrating blobs from '#{old_service}' to '#{new_service}'..."
18+
19+
blobs = ActiveStorage::Blob.where(service_name: old_service)
20+
count = blobs.count
21+
22+
if count.zero?
23+
puts "No blobs found for service '#{old_service}'"
24+
else
25+
puts "Found #{count} blob(s) to migrate"
26+
# Update service names in batches
27+
updated_count = blobs.update_all(service_name: new_service)
28+
total_updated += updated_count
29+
puts "✓ Updated #{updated_count} blob(s) to use '#{new_service}'"
30+
end
31+
end
32+
33+
puts "\n#{'=' * 80}"
34+
puts "Migration complete!"
35+
puts "Total blobs updated: #{total_updated}"
36+
puts "=" * 80
37+
end

spec/lib/tasks/backfill_azure_storage_rake_task_spec.rb

Lines changed: 0 additions & 132 deletions
This file was deleted.

0 commit comments

Comments
 (0)