Best way to write to a file directly in cache?

[Floppy]

Does anyone have a recommended pattern for creating empty files in the cache, rather than uploading them?

My situation is that my users upload ZIP files, but then I extract them and add the extracted files to Shrine, not the original zipfile. My current approach is to create a temporary directory inside the cache directory, unzip into that, then open each file and attach it, but it feels messy, because there's an unnecessary copy, and it leaves these temp files lying around unnecessarily for the cache sweeper to clean up later on.

I'd rather do something like create a tempfile with Shrine, ready to be managed, then get the ZIP extractor to write directly to the filename that Shrine decided upon. Then when I attach it and Shrine promotes the file, there's only one copy, so there's nothing to clean up, and I'm. not worrying about race conditions with Shrine backgrounding etc.

So in summary: is there a way to get Shrine to create an UploadedFile record in cache that I could then write to "out of band" before promoting?

[janko]

Are you using file system storage? If yes, you could consider moving the originally extracted file into the cache storage. Something like:

File.open("/path/to/extracted/file", "rb") do |file|
  record.file_attacher.attach_cached(file, upload_options: { move: true })
end

I sort of get what you're asking for, but I'm not sure if that's currently possible with Shrine.

[Floppy]

Yeah, cache is filesystem storage, should have mentioned that explicitly. I'll have a look at that approach, though the main reason for extracting directly into the cache is so that I don't have to have another place for temporary storage.

[Floppy]

I found a solution that works, after realising that I do something similar elsewhere in my app:

After extracting the file manually into the cache area, I can then create an UploadedFile object and attach that, which then cleans up nicely after itself.

files.create(filename: filename, attachment: ModelFileUploader.uploaded_file(
  storage: :cache,
  id: File.join(file_path_inside_cache),
  metadata: {filename: File.basename(file_path_inside_cache)}
))

[jrochkind]

if I were doing something like that (and I have done things like that!) I would create your temporary files in local system temp location, using ruby stdlib Tempfile class and Dir.mktmpdir method. I would also clean them up in my code when I"m done -- although ruby should clean up Tempfile objects too when abandoned in say an error condition as a failsafe, it's more efficient to clean them up yourself.

Don't try to use possibly remote Shrine cache location for these tempfiles, just use local system temp locations using ruby stdlib convenience methods, and delete them yourself when you're done with htem if possible.

Then I think you've found a good way to attach the created file(s) you do want to keep at the end, yup!

[Floppy]

Thanks - that's the approach I was taking, and yes, if cache could be remote then that's absolutely the right thing to do. I was running into occasional race conditions with the attachment promotion and the file cleanup, which is what originally motivated the question; there are other ways to fix that though other than what I ended up doing, which would certainly be more appropriate in the general case.