[Feature] Will immich support remote microservices?

[Wait845]

The feature

I've noticed that you offer support for remote machine learning. However, in my scenario, the use of microservices to generate thumbnails is more frequent than machine learning, and it consumes most of the CPU resources, causing my machine to slow down. In my view, microservices are independent services and should easily support features similar to those in machine learning services. Thank you.

Platform

• Server
• Web
• Mobile

[bo0tzz]

This is supported, microservices just needs access to a few more things than ML does. As long as it has access to postgres, redis, and the library folder, you can run the microservices anywhere.

[tanmaychimurkar]

Hi @bo0tzz, can you please share a sample implementation of how this can be done? I am also trying to do the same. I have Immich on one machine, and I want to offload all microservices computation to another machine (thumbnails, extract metdata, transcode videos). I don't understand how this can be done via docker, so I need your help in getting this set up.

Thanks for your input!

[suprPHREAK]

Hi @tanmaychimurkar , did you have any progress on this?

[tanmaychimurkar]

Hi @suprPHREAK, yes I managed to get this to work via tailscale and CIFS share. We need to run the redis service on the remote host machine, expose port from postgres service, and be able to connect to both postgres and library folders from the remote host (via CIFS mounts in docker).

I am currently using this setup to outsource all the microservices jobs to the remote host, and it works well I must say.

[sisilet]

@tanmaychimurkar do you have immich server running on a separate machine? if redis service runs on the remote machine, how does the immich server access redis and queues background jobs?

[tanmaychimurkar]

@sisilet Yes, the Immich server is running on my Raspberry Pi4, and the redis service is runnning only on my Desktop with RTX 4090. I have posted a comment here explaining the steps for my setup, maybe this helps you for also setting it up?

[dimitur2204]

Are there any docs on how to do that, I can't seem to find anything?

[dimitur2204]

Thanks. In my case it will be quite the job to set that up since I want my Desktop Windows PC to run the transcoding and ML and thus will need access to the Redis and Postgres running on my homelab and they are on different networks. Thanks a lot for the clarifications!

[tanmaychimurkar]

Hi @dezerving and @dimitur2204,

I can understand that it might seem quite the job, but it is not that hard to set this up. I have been using this setup for about 3 months now, and never had an issue offloading the transcoding jobs to my desktop while Immich is hosted on my RPi4. I will try again to outline the parts that are needed to be able to set this up so that both ML and transcoding jobs run on the remote machine.

Let's call the machine where Immich is running as host, and the powerful desktop hunk as remote.

1. For the host and the remote to be able to communicate, I am using tailscale, there is also the NordVPN Meshnet, or Wireguard (A little advanced, but doable nonetheless). The setup is very easy for host and remote, and once you have both of them in your tailnet, they will be able to communicate with each other. To check if they can indeed communicate, run the following ping from host to remote: ping <remote-tailscale-ip>. You should see something like below:

PING <remote-machine-ip> (<remote-machine-ip>) 56(84) bytes of data.
64 bytes from <remote-machine-ip>: icmp_seq=1 ttl=64 time=46.9 ms
64 bytes from <remote-machine-ip>: icmp_seq=2 ttl=64 time=9.38 ms
64 bytes from <remote-machine-ip>: icmp_seq=3 ttl=64 time=6.84 ms
--- <remote-machine-ip> ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 6.838/21.053/46.945/18.337 ms

Try the same from remote to host, and then you are sure the machines are able to communicate with each other.

2. For Immich, I am guessing you are using the docker-compose.yml file. If not, I would suggest use it, since the setup becomes far easier while using docker containers.

In the database service, you need to add the expose instruction, so that the port of the database is open from the host, so that the remote can connect to it and write data in it.

...

expose:
    - '5432'  # or whichever port you are using for your 'database' service, 5432 is the default port for `postgres` db
...

While you are in the docker-compose.yml file, comment out the redis service from host, since this service would now be running on remote.

3. Now for the tricky part, CIFS mounts. CIFS mounts are needed to be able to map the UPLOAD_LOCATION path from host to the remote, so that we are able to write data from remote to the host path, directly from remote. This is like ssh(well, not exactly), but you are using the CIFS mount to write directly to host from remote. For this, the documentation is listed here.

Essentially, you need to create 2 CIFS mounts: 1 for the UPLOAD_LOCATION (path to Immich files), and one for the DB_DATA_LOCATION (path to database files). The above are the environment variables that you set while running Immich instance for the first time. Start by creating a new docker-compose.yml file on your remote machine, and add the redis service to it. Then add 2 new volumes in the redis service, let's say cifs-volume-1 and cifs-volume-2, and then declare those volumes as follows:

...
volumes: 
  cifs-volume-1:
    driver_opts:
      type: cifs
      o: username=yoursshusername,password=yoursshusername,uid=1021,gid=100,vers=3.0
      device: //<host-tailscale-ip>/path/to/immich/files

# similarily for cifs-volume-2
...

These are the CIFS mounts that are now mapped on your remote machine, and point to the host paths.

Note: The above parameters o: username=yoursshusername,password=yoursshusername,uid=1021,gid=100,vers=3.0 must be setup as per the user permissions that you have setup for your Immich instance on host. More about this here.

Alright, so now that you have come this far, it is time to test the setup. Run the docker-compose.yml file from your remote, and then run the docker-compose.yml file from your host. Check the logs of both to see if the Immich_server is able to start correctly, and check if redis instance on your remote is also starting up correctly.

If all looks good, then start a microservices job, like thumbnail creation or something and see if the job goes through from the logs. At this point, only a little fine-tuning would be needed to get the setup correct, since much of the setup is done in the above 3 steps.

Sorry about the long post, and the delay. Hope this helps! :)

[kpatdev]

Thank you for the write up @tanmaychimurkar.

My host is Unraid and my remote is Windows. I think I've got this working (thumbnails do get created when I upload something new). Instead of CIFS mounting within the compose file I just used the existing SMB shares that I already have mounted to a network drive like this:

volumes:
	- "//a/upload/location"
	- "//a/external/library"
	- "//b/postgresql14"

[dimitur2204]

Thanks @tanmaychimurkar. I might look into it for the future. I have been using tailscale to access my homelab for a while now

[surbin004]

Hi @tanmaychimurkar,
i follow you up until step #3.

i am running immich stack on Synology NAS and want to transcode on my regular desktop as the NAS is using 100% of CPU. i am not using Tailscale just local network,

in my synology immich stack, i manage the stack/containers with portainer

• ${UPLOAD_LOCATION}:/usr/src/app/upload

•   - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
  

my synology is 192.168.1.102 my remote is 192.168.1.50

is my understanding correct, i would need an SSH user created in Synology?
then use
volume1
o: username=synologysshusername,password=synologysshpassword,uid=1021,gid=100,vers=3.0
device: //192.168.1.102/usr/src/app/upload
volume 2
o: username=synologysshusername,password=synologysshpassword,uid=1021,gid=100,vers=3.0
device: //192.168.1.102/var/lib/postgresql/data

[bo0tzz]

@tanmaychimurkar do I understand correctly that the instructions above are saying to mount the postgres data to the remote host over a network mount and run a second postgres container there? Because that is absolutely wrong and will cause data corruption. Don't do that. There should be one single postgres container, with its data folder not on a network mount, and all the Immich containers should connect to that same postgres.

[kpatdev]

That's not the way I took it. I read it as: just expose the container so that way remote services can connect to the singular Postgres container that the main Immich service also connects to.

Edit: jk, I was wrong

[bo0tzz]

That's not how I understood this part of the comment

Essentially, you need to create 2 CIFS mounts: 1 for the UPLOAD_LOCATION (path to Immich files), and one for the DB_DATA_LOCATION (path to database files).

[kpatdev]

Oh wow, yeah. That's a good point. I guess I did originally have it the way it was mentioned. Apologies.

I had changed my hardware and no longer needed to put microservices on a remote machine so I haven't been doing this for a couple months now which is probably why I haven't noticed any negative consequences. Thank you for pointing this out!

[yann117]

Immich does support remote/offloaded microservices, see:
#15085 (reply in thread)