Testing Immich using rclone (minio)

[alkmim]

Hello everyone.

In this dicussion (#3385) I asked about using a storage backed like S3 compatible. One of the responses I got was to use "rclone".

So, I did some tests using rclone.

After testing using rclone, I encontered the following issues:

• [Blocker] Some files from the upload folder would not be moved to the library. Whenever big videos ware "renamed" from the uploads folder to the library, I would get the error "File.Rename error: InvalidArgument: Range specified is not valid for source object rclone".
• [Major] Multiple threads in the rclone/FUSE caused a considerable performance issue. Trying to upload a lot of pictures/videos would take a long long time.

I performed a lot of tests and variation in the configuration and approach. Unfortunately, I decided to post my finds after I did all the tests, so I do not remember all the variations I did.

Feel free to ask questions. It might juggle my memory :)

If someone is able to make it work reliable using rclone, please let me know!

PS.: I am still using immich, but I used another approach that does not include minio.

[bo0tzz]

Unfortunately performance problems are inherent to storage 'translation' layers like this, there's not really a way around that (apart from direct S3 support in the application, of course)

[alkmim]

Yeah. I expected that as well. FUSE is not known by its outstanding performance. I just wanted to try it out before jumping into conclusions.

[AngelaDMerkel]

Is very high performance necessary for the storage of original assets? If there are adequate resolution proxies stored locally, the only time that the remote assets need to be accessed would be for transcoding or metadata extraction correct? There are flags in rclone mount which can help with handling lots of threads, but I have never found it reliable for use with databases. I use it occasionally with Lightroom and it can cause data loss more than one would like.