Attachments not being automatically promoted when using shrine with tus

[sudhanshug16]

Hey,

I am able to upload my files to a cache storage via tus but shrine is not automatically promoting these to permanent storage store.
In Rails console, calling finalize or promote on the attacher does nothing and returns nil. However, calling atomic_promote promotes the file to the store.

Here is my setup:

require "shrine/storage/s3"
require "shrine/storage/tus"
require "shrine/storage/file_system"

if Rails.env.production?
  Shrine.storages[:store] = Shrine::Storage::S3.new(
    bucket: Rails.application.credentials.blackblaze&.bucket,
    region: Rails.application.credentials.blackblaze&.region,
    access_key_id: Rails.application.credentials.blackblaze&.key_id,
    secret_access_key: Rails.application.credentials.blackblaze&.key_name,
    endpoint: Rails.application.credentials.blackblaze&.endpoint
  )

  Shrine.storages[:cache] = Shrine::Storage::S3.new(
    bucket: Rails.application.credentials.shrine_cache&.bucket,
    region: Rails.application.credentials.shrine_cache&.region,
    access_key_id: Rails.application.credentials.shrine_cache&.access_key_id,
    secret_access_key: Rails.application.credentials.shrine_cache&.secret_access_key
  )
else
  Shrine.storages[:store] = Shrine::Storage::FileSystem.new(Rails.root.join("tmp"), prefix: "shrine_uploads")
  Shrine.storages[:cache] = Shrine::Storage::FileSystem.new(Rails.root.join("tmp"), prefix: "shrine_cache")
end

Shrine.storages[:tus] = Shrine::Storage::Tus.new(tus_storage: Tus::Server.opts[:storage])

Shrine.plugin :activerecord
Shrine.plugin :backgrounding
Shrine.plugin :refresh_metadata
Shrine.plugin :cached_attachment_data
Shrine.plugin :restore_cached_data
Shrine.plugin :derivatives, create_on_promote: true
Shrine.plugin :store_dimensions, analyzer: :ruby_vips

Shrine::Attacher.promote_block do
  ShrinePromoteJob.perform_async(self.class.name, record.class.name, record.id, name.to_s, file_data)
end
Shrine::Attacher.destroy_block do
  ShrineDestroyJob.perform_async(self.class.name, data)
end

require "tus/server"
require "tus/storage/s3"
require "tus/storage/filesystem"

if Rails.env.production?
  Tus::Server.opts[:storage] = Tus::Storage::S3.new(
    bucket: Rails.application.credentials.blackblaze&.bucket,
    region: Rails.application.credentials.blackblaze&.region,
    access_key_id: Rails.application.credentials.blackblaze&.key_id,
    secret_access_key: Rails.application.credentials.blackblaze&.key_name,
    endpoint: Rails.application.credentials.blackblaze&.endpoint
  )
  Tus::Server.opts[:redirect_download] = true
else
  Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new(Rails.root.join("tmp/tus_uploads"))
end

class TusUploader < Shrine
  storages[:cache] = storages[:tus]

  Attacher.derivatives do |original|
    magick = ImageProcessing::Vips.source(original)

    {
      thumbnail: magick.resize_to_limit!(130, 130)
    }
  end
end

class OrderLogAttachment < ApplicationRecord
  belongs_to :order_log
  validates :attachment, presence: true

  include TusUploader::Attachment(:attachment)
end

class Api::Scrubbers::Orders::OrderLogAttachmentsController < ApiController
  def create
    order_log = OrderLog.find_by(public_id: params[:order_log_id])
    raise NotFoundError if order_log.nil?

    order_log.order_log_attachments.create!(
      attachment_data: order_log_attachment_params[:attachment_data].merge(
        storage: "cache"
      ).to_json
    )

    render json: {
      result: "success"
    }, status: :created
  end

  private

  def order_log_attachment_params
    params.permit(attachment_data: [:id, metadata: [:filename, :mime_type, :size]])
  end

  def order
    @order ||= current_scrubber.orders.find_by(public_id: params[:order_id])
  end

  def order_log
    @order_log ||= order.order_logs.find_by(public_id: params[:order_log_id])
  end
end

Two questions I guess:

1. Is my assumption correct that shrine will automatically promote the upload? If not, do I need to call finalize or some other method to do so?
2. Why is the atomic version of promote working, while finalize and promote don't?

Please let me know if I am missing something or if things have changed and the docs weren't updated. I followed these two resources: https://github.com/shrinerb/shrine-tus-demo, https://github.com/shrinerb/shrine/wiki/Adding-Resumable-Uploads

[sudhanshug16]

@janko.
Apologies for the @ mention, but looking for some assistance here. Thanks for taking out the time to look into it in advance.

[janko]

You're assigning cached data directly to the column, which doesn't get picked up by Shrine. You need to use Shrine's attribute setter attachment=.

[sudhanshug16]

Thanks!!!!

[sudhanshug16]

Another question related to this setup.

Here is the updated controller that works

def create
    create_attrs = {
      uploadable: uploadable,
      upload_type: file_upload_params[:upload_type],
      user: current_user
    }

    upload_data = file_upload_params[:upload_data].merge(
      storage: "cache"
    ).to_json

    file_upload = FileUpload.create!(create_attrs)
    file_upload.upload_attacher.attach_cached(upload_data)

    render json: file_upload, status: :created
  end

I am now using Transloadit for uploading images instead of TUS. Therefore my upload_data looks something like this

{
  "upload_data": {
    "id": "photo-1745097552103_637e5420bd40476dbec1d1c4dda21c8e.jpg",
    "metadata": {
      "filename": "photo-1745097552103.jpg",
      "mime_type": "image/jpeg",
      "size": 2494704
    }
  },
  "uploadable_type": "OrderLog",
  "uploadable_id": "JDJWM5WAVGEG9UJA",
  "upload_type": "order_log_files"
}

When I call my create method, the attachment is attached properly however attach_cached seems to be downloading the file which adds latency to the API call which should just be saving the db record. Not seeing any mention of this behavior in https://shrinerb.com/docs/attacher#attaching-cached. I am not using the validation plugin.

Can you please help me understand why is the download happening and how can I prevent it? Thanks in advance

[sudhanshug16]

Umm, these plugins could be downloading the file

Shrine.plugin :refresh_metadata
Shrine.plugin :cached_attachment_data
Shrine.plugin :restore_cached_data
Shrine.plugin :store_dimensions, analyzer: :ruby_vips

Please let me know if this is correct or maybe I am still missing something else.