Horizontal scaling of Open WebUI Pipelines with multiple replicas

[dsewell]

We are running open webui with pipelines on a kubernetes cluster. We have configured multiple replicas of open webui and pipelines to scale for several thousand users with shared storage for the pipeline pods to share pipeline python and json files. However, we find that pipelines does not work well in a horizontally scaled deployment, if a new pipeline is uploaded or the valves for a pipeline are changed the other replica pipeline instances are not aware until they are restarted.

I have reviewed the pipelines code and can't see any mechanism to support clustering with multiple replicas.

What is the recommend method to deploy open webui pipelines at scale?

If deploying multiple replicas in a kubernetes environment what is the recommended method to ensure all replicas are in sync with new pipeline uploads and valve changes?

[dsewell]

In the absence of any other suggestions I've updated the pipelines server to add a file watcher, this detects changes to any pipeline or valve files and updates valves or reloads pipelines for changes that were made on the filesystem, not made by the local server. I'd be happy to create an issue and PR for the code changes if this is of interest.