[Feature] Serverless support

[ephemer]

I have searched the existing feature requests, both open and closed, to make sure this is not a duplicate request.

• Yes

The feature

Running Immich at home is great for many use-cases but – depending on internet speed and hardware availability – potentially not performant or reliable enough for a stress free setup. At the same time, self-hosting Immich on adequate cloud hardware would be more expensive than using <unnamed 3rd-party services>. Self-hosting via a serverless platform like Cloudflare Workers could solve both issues, being very reliable, but also much cheaper (essentially free for most users). Cost would scale with usage, rather than users paying a flat monthly fee forever.

It's clear that this would require dev work. To begin with, an alternative code path would be required to replace the api and microservices "workers" with separately deployable serverless functions that call each other as necessary.

To take full advantage of a serverless environment, the PostgreSQL and Redis databases would also need to be serverless. Cloudflare's D1 is cost-effective, but is compatible with SQLite and not Postgres, which may require reviving this thread.

Cloudflare also has a very competitive offering for KV storage, and WebWorker support via DurableObjects. In addition, it offers dedicated serverless message queues, which may be an even better match and more cost effective, assuming Redis is unneeded for other things (I did not dig deep into this yet, as I'm new to the project).

The remaining parts of the picture appear to be the ML models for CLIP and facial recognition. Cloudflare does not offer generic serverless GPU functionality, although it does offer Workers AI with GPU acceleration, which includes llava and uform-gen models for CLIP-like functionality. Regardless, CPU inference may be totally adequate for Immich's use case, given it could run with very high parallelism (e.g. 1000 parallel processors running at once).

I believe a reasonable migration path – broken into 4 milestones – could be:

1. Allow the server code to run in Cloudflare workers
   • Estimated: 2 weeks part time.
   • The server would continue to connect to externally-hosted DBs (not ideal due to latency).
2. Allow for Immich server usage with SQLite with FTS5 (and by extension, Cloudflare D1) instead of PostgresQL.
   • Estimated: 6 weeks part time (but there are still a lot of unknown unknowns here, at least to me)
   • PostgreSQL-specific features in use (and how they could be worked around):
     • Text search: FTS5 should match this functionality (albeit with different table management).
     • Vector search: sqlite-vec should be good enough. This is not supported on D1 however, and would need to be stored in a separate Cloudflare Vectorize table.
   • Ideally, differences between SQLite and Postgres could be abstracted away to allow either DB to be used.
3. Enable alternative message queues aside from BullMQ
   • Estimated: 2 weeks part time
   • Assuming this is the main use case for Redis, this should allow us to safely remove Redis entirely.
     • If Redis is being used elsewhere too, we would be blocked at this point without further investigation and work.
4. Enable ML inference via ONNX runtime + TS, in addition to the existing Python / armnn / rocm / cuda setup.
   • Estimated: 4 weeks part time.
   • GPU-accelerated CLIP processing via Workers-AI supported models could be added later if desirable.

Next Steps

I'd be happy to slowly tackle one or even all of those issues (they should be doable independently of one another) if no one objects. Of course, I don't want to work on this if similar work is planned, or maintainers disagree about the direction. Please let me know what you think.

Platform

• Server
• Web
• Mobile

[ephemer]

In particular I'd love to hear @alextran1502's thoughts on this potential direction

[mmomjian]

Hello,

Alex can certainly chime in if he sees this. SQLite support is not something we are considering at this time, and that makes this effectively a duplicate of #562. I don’t see serverless support as being feasible in immich future. Appreciate your thoughts.

[ephemer]

Thanks for your feedback @mmomjian. #562 is a part of the overall picture (which is why I linked to it in the OP) but I don't believe this is a duplicate per se.

Either way, I understand the issue about maintaining SQLite support in the core repo potentially being problematic so will consider an alternative approach.

[bo0tzz]

Also see #10394

[ephemer]

Thanks, I actually didn't see this, despite looking a few times.

That appears to be taking a slightly different approach though; IMO one that is not technically realistic or feasible due to the use of web sockets in the server code. The approach that I'm suggesting would allow for web sockets but would be splitting the server monolith into actual separate workers (which is the actual underlying architecture as far as I can gather).

In my initial pitch (OP) I initially wrote out a path to also enable use on AWS Lambda, but I decided that focussing on Cloudflare would be more feasible and minimal as an initial approach.

[zackpollard]

Something like this is highly unlikely to be merged into Immich core. However, if you did work on this I would personally be interested to see where it goes.

[ephemer]

@zackpollard assuming the work is modularised and abstracted to avoid affecting the existing code paths, I'm curious why you see it as a non-starter? I would love to avoid forking the project to make this happen

[zackpollard]

I didn't say it's a non-starter, I am just saying it's highly unlikely that it'll be merged. The amount of people that would be likely to use something like this compared to the effort to implement and maintain it would be quite drastic I expect. I think significant parts of this are going to be close to impossible to pull off without significant performance issues, namely the SQLite migration. Even if you could pull that off in a performant way, maintaining SQLite support on its own within the core codebase would be a significant effort on our part. This is just naming a couple of things, there are many reasons why I would expect this to never get merged into the core codebase.

As I said, I would love to see this as it sounds very cool technically, but realistically our time will be better spent elsewhere wrt maintenance effort.

[ephemer]

@zackpollard fair enough, I see where you're coming from.

It's a real shame because Postgres is great tech, but it seems the vast majority of users will be running it on the same machine as the app server, which will in turn be its only client. That suggests to me that SQLite could be a better match for the majority of users, although I am curious where you think it wouldn't be performant.

Regardless, I'm well aware that suggesting a change of DB (and not "just" an addition) is absolutely a non-starter for a mature project like Immich.

Thanks a lot for your feedback in any case. I will see if I can come up with an alternative solution, e.g. using the dependency injection mechanisms to replace the DB from code run in a parent "piggyback" module (with Immich core as a git submodule).