Fail to create GlideClusterClient

[ronenlinx]

Im using AWS Elasticache with Valkey serverless instance and im failing to create GlideClusterClient. Im getting ClosingError with timed_out exception. When i try to connect to the instance using redis-cli it works. Even tried to connect using the GlideClient and same. What am i missing?

class ValkeyCache:
    def __init__(self, host: str, port: int = 6379):
        self.__addresses = [NodeAddress(host=host, port=port)]
        self.__client: Optional[GlideClusterClient] = None

    async def connect(self) -> None:
        try:
            config = GlideClusterClientConfiguration(addresses=self.__addresses, use_tls=True)

            # Create the cluster client
            self.__client = await GlideClusterClient.create(config=config)
            await self.__client.ping()
        except ClosingError as e:
            config = GlideClientConfiguration(addresses=self.__addresses, use_tls=True)
            self.__client = await GlideClient.create(config=config)
            await self.__client.ping()

This is the code im using to create the client

[Yury-Fridlyand]

Hi @ronenlinx
This code works for me:

client = await GlideClusterClient.create(GlideClusterClientConfiguration([NodeAddress(address, port)], use_tls=True))

Your code looks valid.
Keep in mind, client checks and requires connectivity to all cluster nodes. Are you running your code in the same VPC? If no - do you use an SSH tunnel or VPN? Could you please validate that you can connect to all cluster nodes using cli tool?

[ronenlinx]

Im running the code from my Mac using VPN.
Im able to connect to all cluster nodes using redis-cli, i used the CLUSTER NODES command to check it

[ronenlinx]

Even when using standalone deployment, still not able to connect using the package. The error of time out is returned immediately

[acarbonetto]

Can you post the command line you use to connect to redis-cli? You can remove IP address/port information, I'd like just like to know other flags you're using to connect.

Note: make sure the TLS flag is correct. If not, the client will timeout failing to connect.

[ronenlinx]

This is the command im using and i successfully manage to connect to the deployment

redis-cli -h example_node_endpoint -p 6379 --tls

[ronenlinx]

Even when passinc -c flag to enable cluster mode it works

[avifenesh]

Do you have any other backtrace from the error?
Can you confirm that non-tls is working?

We had an issue with serverless with node-client, which been fix but planned to be released at the end of NOV with 1.2.
Released an RC for node users to unblock, if its the same issue we can release for py as well.

[ronenlinx]

Standalone Non-tls is working using Valkey local container

[avifenesh]

Can you provide a sample of the docker file?

[ronenlinx]

just used this one https://hub.docker.com/r/valkey/valkey/

[shohamazon]

Hi @ronenlinx
Would you like to join us in our Slack Chanel so we can discuss that together? You are more than welcome :)

[dhruvishah2312]

Hi was this ever resolved? I am facing the same issue

[Yury-Fridlyand]

I copied this from #2823

[ronenlinx]

@dhruvishah2312 I tried to increase the connectionTimeout together with @shohamazon but it didn't work for me. Still waiting for a solution :)

[avifenesh]

Still worth a try for other it helped. @ronenlinx Are you on the valkey slack?

[jakob-keller]

Setting connection timeout to 10000ms resolved the issue for me.

valkey-glide==1.3.1, CPython 3.13.2, macOS Sequoia, Apple silicon (M2 Pro)
Amazon ElastiCache for Valkey 8.0.1, encryption in transit enabled, cluster mode on and off tested
Connection via AWS ClientVPN

[Yury-Fridlyand]

I'm glad that it was resolved for you.
But 10 sec connection timeout is too high value isn't it? That confuses me.

[mjbrook]

Dropping a note to say I'm also experiencing the same issue that is resolved by increasing the default connectionTimeout to a minimum of 750ms in my setup. I've included some log output below which may be of some use.

[email protected], nodejs 22.15, macOS (M2 Pro)
Amazon Elasticache for Valkey 8.0.1, encryption in transit enabled, cluster mode on, 1 shard, 1 replica per shard
Connecting using OpenVPN client to endpoint within VPC

2025-05-01T15:23:31.516544Z DEBUG logger_core: connection - new socket listener initiated
2025-05-01T15:23:31.516659Z  INFO logger_core: Client lifetime - construct client
2025-05-01T15:23:31.517242Z  INFO logger_core: Connection configuration - 
Addresses: clustercfg.euw1.cache.amazonaws.com:6379
TLS mode: Secure
Cluster mode
Connection timeout: 250
Read from Replica mode: Prefer replica
database ID: 0
Protocol: RESP3
2025-05-01T15:23:32.103706Z  WARN redis::cluster_async::connections_logic: Failed to create management connection for node `"clustercfg.euw1.cache.amazonaws.com:6379"`. Error: `timed out`
2025-05-01T15:23:32.103737Z  INFO redis::cluster_async: Connected to initial nodes:
primary-ip:6379 - primary-ip

2025-05-01T15:23:32.117502Z  INFO redis::cluster_topology: calculate_topology found topology map:
TopologyView { hash_value: 3736776969757278209, nodes_count: 1, slots_and_count: (16383, [Slot { start: 0, end: 16383, master: "primary.euw1.cache.amazonaws.com:6379", replicas: ["replica.euw1.cache.amazonaws.com:6379"] }]) }
2025-05-01T15:23:32.190818Z  WARN redis::cluster_async::connections_logic: The management connection for node primary.euw1.cache.amazonaws.com:6379 isn't set
2025-05-01T15:23:32.206272Z DEBUG redis::aio::connection: Creating TCP with TLS connection for node: "primary.euw1.cache.amazonaws.com:6379", IP: primary-ip
2025-05-01T15:23:32.456064Z  WARN redis::cluster_async::connections_logic: Failed to create management connection for node `"primary.euw1.cache.amazonaws.com:6379"`. Error: `timed out`
2025-05-01T15:23:32.458456Z DEBUG redis::aio::connection: Creating TCP with TLS connection for node: "replica.euw1.cache.amazonaws.com:6379", IP: replica-ip
2025-05-01T15:23:32.578940Z DEBUG redis::aio::connection: Creating TCP with TLS connection for node: "replica.euw1.cache.amazonaws.com:6379", IP: replica-ip
2025-05-01T15:23:32.708411Z  INFO redis::cluster_async: refresh_slots found nodes:
primary.euw1.cache.amazonaws.com:6379 - primary-ip

2025-05-01T15:23:32.708473Z  INFO logger_core: connection - new connection started
2025-05-01T15:23:32.709292Z  INFO logger_core: Client lifetime - connected to server
2025-05-01T15:23:35.710700Z  INFO redis::cluster_async: Started refreshing connections to ["replica.euw1.cache.amazonaws.com:6379"]
2025-05-01T15:23:35.712958Z DEBUG redis::aio::connection: Creating TCP with TLS connection for node: "replica.euw1.cache.amazonaws.com:6379", IP: replica-ip
2025-05-01T15:23:35.851282Z DEBUG redis::aio::connection: Creating TCP with TLS connection for node: "replica.euw1.cache.amazonaws.com:6379", IP: replica-ip
2025-05-01T15:23:35.961874Z  WARN redis::cluster_async: Failed to refresh connection for node replica.euw1.cache.amazonaws.com:6379. Error: `Failed to refresh both connections - IoError: Node: "replica.euw1.cache.amazonaws.com:6379" received errors: `timed out`, `timed out``
2025-05-01T15:23:35.961887Z DEBUG redis::cluster_async: refresh connections completed
2025-05-01T15:23:38.962952Z  INFO redis::cluster_async: Started refreshing connections to ["replica.euw1.cache.amazonaws.com:6379"]
2025-05-01T15:23:38.965307Z DEBUG redis::aio::connection: Creating TCP with TLS connection for node: "replica.euw1.cache.amazonaws.com:6379", IP: replica-ip
2025-05-01T15:23:39.108640Z DEBUG redis::aio::connection: Creating TCP with TLS connection for node: "replica.euw1.cache.amazonaws.com:6379", IP: replica-ip
2025-05-01T15:23:39.216173Z  WARN redis::cluster_async: Failed to refresh connection for node replica.euw1.cache.amazonaws.com:6379. Error: `Failed to refresh both connections - IoError: Node: "replica.euw1.cache.amazonaws.com:6379" received errors: `timed out`, `timed out``
2025-05-01T15:23:39.216182Z DEBUG redis::cluster_async: refresh connections completed

Initially, it's unable to establish a connection to the primary node. Once a connection is established, it remains unable to connect the replica node, repeating the refresh connections logic.

[avifenesh]

750 ms?

[avifenesh]

and you say that you are still unable to solve the issue?

[mjbrook]

A value of 750ms solves the issue for me - I see consistent client creation and connection stability in my testing.

Values lower than 750ms and I start to see errors establishing management connections on startup or consistent inability to connect to the replica node in the cluster.

[avifenesh]

I see, ok, thanks for the data! 750 is much better, some here have very high numbers. The connection does more, as you saw, than establishing tcp. So it's take some time, in addition to the VPN slowing.