[feat] support cluster and sentinel deployment for redis

[chenzikun]

• cluster已经过测试，sentinel暂未测试

feature-redis cluster & sentinel

fix #147

[Yeuoly]

Please use English comments instead

[erigo]

I suggest using redis.UniversalClient because it supports direct connections, Sentinel mode, and cluster mode, which can simplify configuration management.

I've created a PR at #276 to implement support for Sentinel mode. This will enable us to use Redis more reliably in the production environment.

[chenzikun]

@Yeuoly [done] convert chinese comment to english, pls check!

[Yeuoly]

I've reviewed #276, I prefer to use redis.UniversalClient as the client so that this is no need to maintain a interface like what you do, @chenzikun @erigo , how do you like?

[erigo]

I suggest adding support for Redis cluster based on #276.

[AkisAya]

i think sentinel mode and cluster mode of reids subpub will not work as expected. when there is a failure of a redis instance, pubsub connection of failed redis will not automatically change to another redis instance.
it's better to split the cache ability to different interfaces and carefully implement different part. #189

[erigo]

Thanks for pointing this out — I agree with your observation that Redis Sentinel and Cluster modes do not automatically recover Pub/Sub connections after failover, which makes Redis Pub/Sub unsuitable for high-availability scenarios.

However, I would respectfully disagree with the idea of “splitting the cache ability into different interfaces.”
In practice, it's often better to unify Redis access through a single interface using redis.UniversalClient, which allows seamless switching between standalone, Sentinel, and Cluster modes. This approach simplifies the codebase, avoids redundant abstractions, and still gives us flexibility for future changes.

As for Pub/Sub, I agree that Redis is not the right tool for reliable messaging. For systems that depend on high-availability message delivery, dedicated message queues like Kafka or NATS are far more suitable.

So in short: use redis.UniversalClient to unify caching access, but avoid Redis Pub/Sub for HA-critical workflows.

[AkisAya]

use redis.UniversalClient to unify caching access

this will make it hard to extend different caching storages. e.g. our redis cluster implemention is not opensource RedisCluster or Sentinel

but avoid Redis Pub/Sub for HA-critical workflows

agree. and it's better to split the current responsibilities of the cache into a cache interface and pubsub

[Yeuoly]

use redis.UniversalClient to unify caching access

this will make it hard to extend different caching storages. e.g. our redis cluster implemention is not opensource RedisCluster or Sentinel

but avoid Redis Pub/Sub for HA-critical workflows

agree. and it's better to split the current responsibilities of the cache into a cache interface and pubsub

I agree splitting them into different interfaces would make cache more extendable, but just at this point, Redis is the production standard for almost all the software, adding support for other cache is not a priority I guess

[chenzikun]

@Yeuoly ok.I had change my code.redis cluster base on redis.UniversalClient, pls check!

[zhuzihao-hz]

Hi👋 , I tried UniversalClient and UniversalClient, but got the following error while running

[ERROR] PluginDaemonInternalServerError: redis: Watch requires at least one key
goroutine 1575 [running]:

Is this related to this func client.Watch(ctx, func(tx *redis.Tx) ?

Sorry, I'm not very familiar with the Redis API.

[JohnMin-ai]

@zhuzihao-hz @chenzikun
使用redis集群，本地启动dify调试，会向plugin发送请求：response = self._request_with_plugin_daemon_response("POST", f"plugin/{tenant_id}/debugging/key", Response)，然后plugin会报错：[ERROR]PluginDaemonInternalServerError: redis: Watch requires at least one key

定位到 redis.go 文件 Transaction方法里面的报错位置为：
return client.Watch(ctx, func(tx *redis.Tx) error {
原因为：
client.Watch(ctx, fn) //缺少 keys 参数，导致 Redis 报错。

高版本的redis，比如：go-redis v9.5.x ，Watch() 方法源码中仍然保留以下判断逻辑
func (c *Client) Watch(ctx context.Context, fn func(*Tx) error, keys ...string) error {
if len(keys) == 0 {
return errors.New("redis: Watch requires at least one key")
}
...
}

[chenzikun]

@zhuzihao-hz @chenzikun 使用redis集群，本地启动dify调试，会向plugin发送请求：response = self._request_with_plugin_daemon_response("POST", f"plugin/{tenant_id}/debugging/key", Response)，然后plugin会报错：[ERROR]PluginDaemonInternalServerError: redis: Watch requires at least one key

定位到 redis.go 文件 Transaction方法里面的报错位置为： return client.Watch(ctx, func(tx *redis.Tx) error { 原因为： client.Watch(ctx, fn) //缺少 keys 参数，导致 Redis 报错。

高版本的redis，比如：go-redis v9.5.x ，Watch() 方法源码中仍然保留以下判断逻辑 func (c *Client) Watch(ctx context.Context, fn func(*Tx) error, keys ...string) error { if len(keys) == 0 { return errors.New("redis: Watch requires at least one key") } ... }

请问你能提供更多的上下文吗，我这边未出现你说情况，这种是集群模式导致的，还是redis本身存在这种错误；我这边可以修改Transaction这个函数，但是太确认是否会引起其他问题，因为我没有办法在所有的环境中验证。

@JohnMin-ai

[chenzikun]

@zhuzihao-hz @chenzikun 使用redis集群，本地启动dify debug，会向plugin发送请求：response = self._request_with_plugin_daemon_response("POST", f"plugin/{tenant_id}/debugging/key", Response)，然后plugin会报错：[ERROR]PluginDaemonInternalServerError: redis: Watch require at least one key

定位到redis.go文件Transaction方法里面的报错位置为： return client.Watch(ctx, func(tx *redis.Tx) error { 原因为： client.Watch(ctx, fn) // 缺少keys参数，导致Redis报错。

高版本的redis，比如：go-redis v9.5.x ，Watch()方法源码中仍然保留以下判断逻辑 func (c *Client) Watch(ctx context.Context, fn func(*Tx) error,keys ...string) error { if len(keys) == 0 { return error.New("redis: Watch需要至少一个key") } ... }

fixed this error!

[chenzikun]

@Yeuoly pls review this feature