[Feature] Remote Thumbnail Generation

[jadster]

I have searched the existing feature requests to make sure this is not a duplicate request.

• Yes

The feature

I'm new to Immich and mostly elated by how clean it is set up and how clean and satisfying the UX is. A rare experience.

One of the aspects that truly blew me away was how easy it was to use my little M2 Macbook Air as a remote Machine Learning worker for my feeble Synology NAS, where my Photo collection and Immich installation reside. My Mac processes the machine-learning stuff much much faster than my NAS despite no GPU acceleration.

Naturally, I would now like to further offload tasks from my NAS that are taxing it. Most notably the thumbnail generation task. It is extremely slow and CPU-intensive on my NAS and I would love to let my Macbook Air or other machines in the home network help out.

On a system like my NAS where both disk IO and CPU are vastly inferior to my macbook air and but my home network is decently quick - it seems to me it would be a large benefit with no significant overhead on my NAS if I could outsource most of the thumbnail generation away from my NAS. Especially when initializing a large dataset or making lots of changes to an external library that needs to be rescanned and processed...

I searched the issues and docs to see if this has been mentioned or described.

I did find mentions of microservices in the docs but it's much too vague for me.

And I found issue #14142 with a few examples of custom setups getting certain other services outsourced, and maybe I'll figure out eventually. But clearly it's not as easy as outsourcing the machine-learning tasks.

Assuming that remote thumbnail generation is already possible, my feature request would be this:

Make outsourcing thumbnail generation as easy and as similar as possible for the user as it works with machine-learning.
i.e.

• run an(other) instance of a thumbnail-generation container with a ready-to-use api. expose the container port on the remote computer.
• implement a UX in Admin Settings for thumbnail generation that is identical to "Machine Learning Settings", where I can just add and order URLs to remote thumbnail generation apis.

And then describe the process in the docs for Dummies like me :)

Platform

• Server
• Web
• Mobile

[bo0tzz]

Like you found, this is already possible, it's just an advanced setup. Making it easier isn't really feasible - the only way that could be done is by creating a separate container for only thumbnail handling (just like there is for ML), but that wouldn't really make sense architecture wise.

[yann117]

Thanks, I'm currently reading the doc and also the architecture page helps.
I have (the beginning of) an idea how to do it now.

I need to figure out what are the ports of each containers used for communication, that I need to expose on the homelab instance, to have the offloaded instance connect to Redis/Postgres.
Also I need to determine if access to the homelab instance's Filesystem is required for the remote or not (data flow could be all through the API/communication channel otherwise).
Probably more reading will help ;-)

Edit:

connect it to the [..] UPLOAD_LOCATION mount
That answer the question of accessing the "local" homelab FS

[yann117]

Checking the code, the postgres database container is on port 5432. I guess that is the one we need to export explicitly on the local instance.

Google tell me that is the default port for postgres :-)

Therefore I checked what is the port for redis and found it is 6379, and I find that one in the code too, so I assume this is the one to export for redis?

[yann117]

Ok, finally it was not so hard 👍 I got it working by doing few changes.
Below are my files/changes, let's call "local" the main instance, your NAS/Pi/Mini-PC/homelab, and "remote" your powerful other machine.

On the local:

1. open the ports 5432 and 6379 in your firewall
2. save and then edit your compose.yml, applying the changes below ## ADDITION:

services:
  immich-server:
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /data/pictures/john:/john:ro
      - /data/pictures/jane:/jane:ro
    env_file:
      - .env
    ## ADDITION: Only keep the api workers (UI) running on the 'local'
    environment:
      IMMICH_WORKERS_INCLUDE: 'api'
    ...

  immich-machine-learning:
    ... # no changes

  redis:
    ## ADDITION: Publishing Redis port to expose it for the remote
    ports:
      - '6379:6379'

  database:
    ## ADDITION: Publishing Postgres port to expose it for the remote
    ports:
      - '5432:5432'

On the remote:

1. don't forget to enable hardware acceleration (for ML+transcoding) if possible

2. if you changed the default Postgres database password (DB_PASSWORD variable) in your local Immich .env file, then set the same password in your remote .env file

3. mount (SMB/NFS) the local filesystem, which contain your Immich UPLOAD_LOCATION and your external libraries if any. In my case and example, I have two external libraries under /data/pictures. I mount that the whole parent /data which contains my Immich docker as well

4. in the .env file modify the UPLOAD_LOCATION with your mounted folder and so it is resolving/pointing to the local /library folder. In my case:

UPLOAD_LOCATION=/mnt/homelab/data/docker/immich-app/library

5. edit your compose.yml with the below changes:

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: vaapi # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      ## CHANGES: update your external libraries path to resolve to the **local** mounted location
      - /mnt/homelab/data/pictures/john:/john:ro
      - /mnt/homelab/data/pictures/jane:/jane:ro
    env_file:
      - .env
    ## ADDITION: Only run the microservices workers (excluding 'api'). Then define the DB and REDIS variables with your homelab IP
    environment:
      IMMICH_WORKERS_EXCLUDE: 'api'
      DB_HOSTNAME: '192.168.1.xx'
      REDIS_HOSTNAME: '192.168.1.xx'

Finally, restart both Immich instances with:

docker compose up -d --force-recreate

Now I can re-process my whole library in a few minutes as compared to days! Thank you !

[A1asd]

Ok, finally it was not so hard 👍 I got it working by doing few changes. Below are my files/changes, let's call "local" the main instance, your NAS/Pi/Mini-PC/homelab, and "remote" your powerful other machine.

On the local:

1. open the ports 5432 and 6379 in your firewall
2. save and then edit your compose.yml, applying the changes below ## ADDITION:

services:
  immich-server:
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /data/pictures/john:/john:ro
      - /data/pictures/jane:/jane:ro
    env_file:
      - .env
    ## ADDITION: Only keep the api workers (UI) running on the 'local'
    environment:
      IMMICH_WORKERS_INCLUDE: 'api'
    ...

  immich-machine-learning:
    ... # no changes

  redis:
    ## ADDITION: Publishing Redis port to expose it for the remote
    ports:
      - '6379:6379'

  database:
    ## ADDITION: Publishing Postgres port to expose it for the remote
    ports:
      - '5432:5432'

On the remote:

1. don't forget to enable hardware acceleration (for ML+transcoding) if possible
2. if you changed the default Postgres database password (DB_PASSWORD variable) in your local Immich .env file, then set the same password in your remote .env file
3. mount (SMB/NFS) the local filesystem, which contain your Immich UPLOAD_LOCATION and your external libraries if any. In my case and example, I have two external libraries under /data/pictures. I mount that the whole parent /data which contains my Immich docker as well
4. in the .env file modify the UPLOAD_LOCATION with your mounted folder and so it is resolving/pointing to the local /library folder. In my case:

UPLOAD_LOCATION=/mnt/homelab/data/docker/immich-app/library

5. edit your compose.yml with the below changes:

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: vaapi # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      ## CHANGES: update your external libraries path to resolve to the **local** mounted location
      - /mnt/homelab/data/pictures/john:/john:ro
      - /mnt/homelab/data/pictures/jane:/jane:ro
    env_file:
      - .env
    ## ADDITION: Only run the microservices workers (excluding 'api'). Then define the DB and REDIS variables with your homelab IP
    environment:
      IMMICH_WORKERS_EXCLUDE: 'api'
      DB_HOSTNAME: '192.168.1.xx'
      REDIS_HOSTNAME: '192.168.1.xx'

Finally, restart both Immich instances with:

docker compose up -d --force-recreate

Now I can re-process my whole library in a few minutes as compared to days! Thank you !

I followed your steps and have to say, after a little tinkering, this worked like a charm :)

[Hritik14]

@yann117 thanks! got it working. Although I didn't set IMMICH_WORKERS_INCLUDE: 'api'' on local (my pi) so that I have the microservices workers running even when my remote is down.

The jobs being picked up is 2x than defined in job settings. It would be good to be able to configure max job count for each machine (maybe via environment variables?) so that I can offload max tasks to my remote and min to local.

[mmomjian]

We plan to remove Redis soon which would make this process easier.

[ppenguin]

@yann117 Thanks!
I'm just now looking into it, and won't be able to use any of the docker compose stuff since I've deployed everything in nomad, and the external ML container with podman/systemd on NixOS.

But: to add remote transcoding to this, my challenge is not volume mounts or inter-container networking, but I need to understand the following:

1. the second container without API will get its config from postgres, correct?
2. I have to be careful to not have overlap in enabled workers between the two? -> is it possible to select this more fine-grained than "api" vs. "everything else" like explained here. <- in the rare case that my "powerful worker remote" would be offline, I'd want to still have complete view/upload/sync functionality
3. Is my understanding correct that task queuing/scheduling is coordinated via redis? (and that this is the reason why the setup without any further communication channels between two server instances works)

@mmomjian regarding (3), what will be replacing redis and how will this affect "distributed" capabilities?