Cost of creating ClusterClients and underlying rust client instance?

[marcind]

If I create 2 different instances of a cluster client (using the Go wrapper, for example) that use the same cluster endpoint but perhaps different options, will this result in the underlying rust client creating 1 set of cluster node connections or 2 sets of cluster node connections?

In other words, is it cheap to create new ClusterClients (underlying rust instance is reused) or expensive (new instance each time, with all the associated handshakes, etc)?

[avifenesh]

Hi :)
It is “expensive”.
While it is an interesting observation, when you say different options, you might talk about, for example, refresh slots.
If one client wants to refresh slots each x time, and the other each y time, it is conflicting. Or for example, timeouts, etc.
Refresh slots affect the client instance, it will make it busy doing something, while it is not the desired behavior for the specific client.

I'll take it into a more philosophic observation, but this is my observation, not an official perspective, so it might be unagreed by the team.

While it might seem like that, the rust client is not acting as a separate piece, like a server and a client, but it is just a piece of the client itself.
Assuming that the smallest piece of logic of the client, if we break it down, is a command — the majority of the lifetime of a command, after the network time, is spent in the core.
The go, while has a thinner code, acts as the client, and uses its arm, the Rust code, to perform tasks.
The “core” term, is a development term, not an end user term. The end user isn't concerned about where the logic is performed, he manages the logic in its app, using his chosen language. If he would like to use Rust, he would do it, he picked a different language, and how the developers of the library manage their codding style should not interfere with his development. The goal is very much the opposite, and the places where you pointed out that it isn't like that, are our priority to fix.
Taking a few entities and using the same arm to perform the mutual work, considering that the most of the time they are less busy taking offloading to the arm, then the arm is busy performing the tasks, will not lead to better results in the majority of the cases. It is probably the opposite, the arm might be too busy to perform the requests the center sends and you most likely want another arm.
I would argue it is like a manager and robotic arm in a factory. If the manager is not happy with the pace, he wants another arm, not another manager.

It might lead to the question of why not a connection pool, then.
The reason is the same logic basically, the most of the time, in a more wide look, the command is on the air between the app to server and back, and another connection would not speed it up, in the majority of the cases.
In some cases, less common, you do want more than one, and we will support a pool, but this is much less common, and you probably will be ok with adding one more client, and not a dynamic pool.

Please share your ideas if you see it differently.
I might be missing a use case, or seeing it wrong, and would be happy to challenge my perspective, or to discover another use case we need to take care of.

[marcind]

Thank you for the response @shohamazon and I do have more questions :)

1. Are non-atomic batch key commands required to fall into the same slot or are they split up into per-shard pipelines?

For pipelines, read commands will be executed on the replica (in a round robin strategy) while the write commands will be executed on the primary of course.

2. Are you saying that if I have a non-atomic batch that contains a mix of write and read commands, those commands might get routed to different server nodes depending on the command type? As an app developer I would expect commands in a batch to be processed by the database in the order they are issued. Otherwise batch results could be non-deterministic.

3. What's the recommendation for check-and-set transactions (WATCH)?

[shohamazon]

Hi, sorry for the delayed response

1. Not sure i understand the question, if the command is key-based, it would go into a specific shard.
2. The only order that matters, is that commands within the same slot keys would not be re-ordered, therefor, is you have 2 commands, one targeting shard a and one targeting shard b, you should not be worried about which commands are executed first. In the pipelines, we do keep this slots order, but, we can split write and read commands from the same slot to different nodes (primary and replica) so if its something that bothers you, you can simply configure you client to read from primary only, and you will be okay :)
3. Could you elaborate more? WATCH are for atomic batches ofc, the recommendation is to have a dedicated connection for it, see more in this issue : [Inquiry] Reuse of the GlideClusterClient while watching a key #3691

[marcind]

@shohamazon I'm trying to understand the programming model of using Glide, as an application developer. So let me start over.

Glide seems to offer some potentially useful features that make programming against clustered redis/valkey easier. E.g. it sounds like you can "call" MGET {foo} {bar} and get the correct result regardless of whether the keys are in the same slot, in the same shard, or different shards. That's pretty cool.

If instead I call a non-atomic batch (i.e. pipeline) to execute

GET {foo}
GET {bar}

I'm guessing I will also get the correct result because the commands will be pipelined over whichever connection is connecting to the shard that holds the slot for the given key, yes? If so, that's also nice and convenient.

If I have a client configured with ReadFromPreferReplica, it sounds like the above batch will be executed against the replica nodes for the appropriate shards, which is still fine and great for lowering app latency.

What if my non-atomic batch contains a mix of read and write operations. Let's pretend that INCR does not return the new value, so I need to do the following

INCR foo
GET foo

You wrote "For pipelines, read commands will be executed on the replica (in a round robin strategy) while the write commands will be executed on the primary of course.". Does this mean that the INCR could execute on one node and the GET on another, effectively introducing latency-dependent nondeterminism to the observed result of GET foo? My expectation would be that GET is always executed after INCR if they are in the same batch, regardless of the configuration of the client being used. If that expectation doesn't hold then it will be easy for developers to fall into a trap of writing code that happens to work in unit tests and test environments that might have simplified valkey/redis server setups but then stops working randomly in a real production environment, leading to data consistency issues that are difficult to debug.

Moving to the canonical check-and-set transaction example, let's pretend INCR doesn't exist at all. Now I need to execute the following commands

WATCH foo
val = GET foo
val = val + 1
MULTI
SET foo $val
GET foo
EXEC

with some sort of logic to retry if EXEC returns (nil). Sounds like Glide will make that possible, but require care from the developer to ensure the same client instance is not used concurrently by multiple threads while one of them is in the middle of a WATCH/EXEC. I have not seen any examples of application code using Glide for CAS transactions so perhaps I'm overstating the problem here.

Sounds from #3691 that the recommendation is for the app developer to maintain a pool of clients to use for transactions. So now the app developer needs to make sure they pick the correct client based on what the following code is going to be doing. As that developer I'd be likely to just use a client pool for everything, just so that I minimize the risk of some future developer making a change to the command logic without realizing that they need to use a different client.

[marcind]

The main point I'm trying to make is that I think that Glide has some usability "ledges" that have the potential to trip up developers as they move from "hello world" to implementing more complex functionality.

You start with a single glide client and multi-slot MGET works auto-magically. You then start using advanced features like batches and transactions and need to adopt a different client management paradigm lest you run into correctness problems. And these are problems that may not show up during application development but only once you deploy to a real environment.

I think there's ways to improve the client so that it's easier for developers to use it correctly regardless of what they are doing and makes it impossible (or at least very hard) for them to use it incorrectly ¹. If the recommendation to address these pitfalls is to simply use a differently configured client instance then why couldn't the client just use the appropriate configuration automatically?

I would love to see an example application that demonstrates how to correctly use advanced features correctly using the current design. I think that would help in driving the discussion about what can be improved.

Footnotes

1. https://blog.codinghorror.com/falling-into-the-pit-of-success/ ↩

[asafpamzn]

Thanks @marcind ,
Sorry for the late response.

Yes, you are right. Using pipeline won't guarantee any ordering on the data. For that you will have to use transaction.
I understand the confusion. But forcing it has some performance penalty.

The alternative is to avoid read from replica for the pipeline and route only to primaries. We should add a route and explain it at the documentation.
I don't think that we should strict it. I understand the tradeoffs. But, I believe that once users are configuring read from replica they are OK with eventual consistency.

@barshaul what do you think?

[barshaul]

@marcind Thanks for your point of view on all.

From what I gather, you raised three main points:

1. Command-level control for read routing – You suggest enabling control over read routing at the command level, not just the client level. This would allow users to override the default client-level read-from strategy for specific requests. If not explicitly set, the client-level strategy would still apply.
2. Routing consistency for non-atomic batches – You propose that when executing non-atomic batch, Glide should internally detect if a key has already been targeted by a write request. If so, any following read request to the same key in that batch should be routed to the primary to ensure consistency.
3. Thread safety for stateful commands (e.g., WATCH) – Our original intent (which still stands) was for Glide to manage a connection pool internally, allowing stateful or blocking requests to be handled with dedicated connections. This would let users use a single client for all types of operations. @asafpamzn We can prioritize making this more robust.
   Specifically for WATCH, I believe we can address it by extending the Batch interface. For example:

>>> transaction = Batch(is_atomic=True)
>>> transaction.set("key", "value")
>>> transaction.get("key")
>>> result = await client.exec(transaction, watch=["key"])
>>> print(result)
[OK, b"value"]

This way, we can pipeline the WATCH with the transaction, ensuring that no other client interactions interfere with the watched key.
@shohamazon – Does it make sense & sound feasible?

@marcind – Would appreciate your thoughts on all of the proposed above.

[marcind]

1. Command-level control for read routing – You suggest enabling control over read routing at the command level, not just the client level. This would allow users to override the default client-level read-from strategy for specific requests. If not explicitly set, the client-level strategy would still apply.

Yes. This scenario is coming directly from the application I work on where it benefits from backfilling certain data from replicas but
must absolutely read other data from the primary to ensure a correct view of the latest state of the application. From an client API standpoint it's impractical to have overrides that enable passing this preference selectively, so I opened #4035 to this.

2. Routing consistency for non-atomic batches – You propose that when executing non-atomic batch, Glide should internally detect if a key has already been targeted by a write request. If so, any following read request to the same key in that batch should be routed to the primary to ensure consistency.

I'm not proposing any specific mechanism. What I'm saying is that having a design where operations can be reordered (in a way that depends on the application configuration as well as the server topology) creates the potential for a class of bugs that will occur arbitrarily and may not show up until an application is deployed to more complex environments (perhaps only in production). The documentation of Redis/Valkey as well as the behavior of unsophisticated clients (i.e. just open a TCP connection to a server node and send commands via RESP) allow the app to depend on strong read-after-write consistency, regardless of whether the client is doing pipelining or not. I feel strongly that Glide should maintain that behavior in all cases, and if it does not this needs to be very explicitly disclaimed. If there's an argument about performance being impacted, I would just like to say that in most cases a correct app is preferred to a faster app.

"Non-atomic batches" increasingly appear to me like a Glide-specific concept that does not correspond to anything in the server-side documentation. The fact that they are the only way to achieve command pipelining from a single thread may lead to issues for app developers.

3. Thread safety for stateful commands (e.g., WATCH) – Our original intent (which still stands) was for Glide to manage a connection pool **internally**, allowing stateful or blocking requests to be handled with dedicated connections. This would let users use a single client for all types of operations. @asafpamzn  We can prioritize making this more robust.
   Specifically for WATCH, I believe we can address it by extending the Batch interface. For example:

>>> transaction = Batch(is_atomic=True)
>>> transaction.set("key", "value")
>>> transaction.get("key")
>>> result = await client.exec(transaction, watch=["key"])
>>> print(result)
[OK, b"value"]

This way, we can pipeline the WATCH with the transaction, ensuring that no other client interactions interfere with the watched key. @shohamazon – Does it make sense & sound feasible?

@barshaul I don't think this works for the Check-and-Set use case. The point of Watch is so that you can get some data, run it through some application logic, and then write some data based on that logic.

Regarding an internal connection pool, I opened #4059 to track that. I think without that in place the current CAS transaction support is of limited usability.