High load when many persistent Redis clients reconnect simultaneously to Swoole\Redis\Server

[mirataollahi]

I am running a custom Redis-compatible server using Swoole\Redis\Server.
This server is used as a replacement for a real Redis server. Clients connect using the standard PHP Redis extension with persistent connections (PHP-FPM child processes keep the connections alive).

Under normal conditions, the server works correctly and responds very fast.

However, when the number of persistent client connections grows to around ~20000 connections, and at a certain peak moment many clients disconnect and reconnect at the same time, the Swoole server experiences very heavy load and connection pressure.
Now my server handle all persist connection commmands like PING , ECHO ,CLIENT and ....

My goals are:

Prevent massive simultaneous reconnect storms.

Keep client connections stable instead of having all clients reconnect at the same time.

Reduce server load during reconnect spikes.

I would like to know:

What is the recommended way to handle a large number of persistent Redis clients with Swoole\Redis\Server?

Are there Swoole configuration options or OS tuning parameters that can reduce reconnect spikes?

Is there a way to perform or smooth client reconnect behavior at the server level?

$this->server = new Swoole\Redis\Server($this->host, $this->port, SWOOLE_PROCESS, SWOOLE_SOCK_TCP);

$configs = [
            Constant::OPTION_WORKER_NUM       => 1,
            Constant::OPTION_REACTOR_NUM       => 8,
            Constant::OPTION_HOOK_FLAGS       => SWOOLE_HOOK_TCP,
            Constant::OPTION_BACKLOG          => 65_535,
            Constant::OPTION_DISPATCH_MODE    => 2, // IP dispatch for better connection distribute
            Constant::OPTION_ENABLE_COROUTINE => true,
            Constant::OPTION_OPEN_CPU_AFFINITY => true,
            Constant::OPTION_DISPLAY_ERRORS => true,
            Constant::OPTION_TRACE_FLAGS => SWOOLE_TRACE_ALL,
            Constant::OPTION_LOG_LEVEL => SWOOLE_LOG_DEBUG,
            /** Keep alive clients configs */
            Constant::OPTION_OPEN_TCP_KEEPALIVE => true,
            Constant::OPTION_TCP_KEEPIDLE     => 120,
            Constant::OPTION_TCP_KEEPINTERVAL => 10 ,
            Constant::OPTION_TCP_KEEPCOUNT    => 3 ,
            Constant::OPTION_TCP_FASTOPEN    => true ,
            Constant::OPTION_OPEN_TCP_NODELAY => true,
            Constant::OPTION_ENABLE_REUSE_PORT => true,
            /** Heartbeat IDLE configs */
            Constant::OPTION_HEARTBEAT_IDLE_TIME => 150,
            Constant::OPTION_HEARTBEAT_CHECK_INTERVAL => 10,
            Constant::OPTION_MAX_CONN => 1_000_000,
            Constant::OPTION_MAX_CONNECTION => 1_000_000,
            Constant::OPTION_MAX_COROUTINE => 500_000,
            Constant::OPTION_RELOAD_ASYNC => true, // Use new worker process when worker process is exiting
            Constant::OPTION_MAX_WAIT_TIME => 30, // 30 second wait to kill the current process
];

Environment:

PHP version: 8.2.30
Swoole version: 6.1.4
Server type: Swoole\Redis\Server
Clients: PHP Redis extension using persistent connections under PHP-FPM
OS: Linux 6.14.0-37-generic