Protocol support: RESP3 (#2396)

Overall changes:
- [x] introduce `Resp2Type` and `Resp3Type` shims (`Resp2Type` has reduced types); existing code using `[Obsolete] Type` uses `Resp2Type` for minimal code impact
- [x] mark existing `Type` as `[Obsolete]`, and proxy to `Resp2Type` for compat
- [x] deal with null handling differences
- [x] deal with `Boolean`, which works very differently (`t`/`f` instead of `1`/`0`)
- [x] deal with `[+|-]{inf|nan}` when parsing doubles (explicitly called out in the RESP3 spec)
- [x] parse new tokens
- [x] `HELLO` handshake
  - [x] core message and result handling
  - [x] validation and fallback
- [x] prove all return types (see: redis/redis-specifications#15)
- [x] <strike>streamed RESP3</strike> omitting; not implemented by server; can revisit
- [x] deal with pub/sub differences
  - [x] check routing of publish etc
  - [x] check re-wire of subscription if failed
  - [x] check receive notifications
  - [x] connection management (i.e. not to spin up in resp3)
  - [x] connection fallback spinup (i.e. if we were trying resp3 but failed)
- [x] other
  - [x] [undocumented RESP3 delta](redis/redis-doc#2513)
  - [x] run core tests in both RESP2 and RESP3
  - [x] compensate for tests that expect separate subscription connections

Co-authored-by: Nick Craver <[email protected]>
Co-authored-by: Nick Craver <[email protected]>

Author: mgravell

Directory.Packages.props

@@ -23,7 +23,7 @@
     <!-- For binding redirect testing, main package gets this transitively -->
     <PackageVersion Include="System.IO.Pipelines" Version="5.0.1" />
     <PackageVersion Include="System.Runtime.Caching" Version="5.0.0" />
-    <PackageVersion Include="xunit" Version="2.4.2-pre.12" />
-    <PackageVersion Include="xunit.runner.visualstudio" Version="2.4.3" />
+    <PackageVersion Include="xunit" Version="2.5.0" />
+    <PackageVersion Include="xunit.runner.visualstudio" Version="2.5.0" />
   </ItemGroup>
 </Project>

StackExchange.Redis.sln

@@ -119,6 +119,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{153A10E4-E
 		docs\Profiling_v2.md = docs\Profiling_v2.md
 		docs\PubSubOrder.md = docs\PubSubOrder.md
 		docs\ReleaseNotes.md = docs\ReleaseNotes.md
+		docs\Resp3.md = docs\Resp3.md
 		docs\Scripting.md = docs\Scripting.md
 		docs\Server.md = docs\Server.md
 		docs\Testing.md = docs\Testing.md

docs/Configuration.md

@@ -1,4 +1,4 @@
-Configuration
+# Configuration
 ===
 
 When connecting to Redis version 6 or above with an ACL configured, your ACL user needs to at least have permissions to run the ECHO command. We run this command to verify that we have a valid connection to the Redis service.
@@ -15,7 +15,7 @@ The `configuration` here can be either:
 
 The latter is *basically* a tokenized form of the former.
 
-Basic Configuration Strings
+## Basic Configuration Strings
 -
 
 The *simplest* configuration example is just the host name:
@@ -66,7 +66,7 @@ Microsoft Azure Redis example with password
 var conn = ConnectionMultiplexer.Connect("contoso5.redis.cache.windows.net,ssl=true,password=...");
 ```
 
-Configuration Options
+## Configuration Options
 ---
 
 The `ConfigurationOptions` object has a wide range of properties, all of which are fully documented in intellisense. Some of the more common options to use include:
@@ -98,6 +98,7 @@ The `ConfigurationOptions` object has a wide range of properties, all of which a
 | version={string}       | `DefaultVersion`       | (`4.0` in Azure, else `2.0`) | Redis version level (useful when the server does not make this available)                                 |
 | tunnel={string}        | `Tunnel`               | `null`                       | Tunnel for connections (use `http:{proxy url}` for "connect"-based proxy server)                          |
 | setlib={bool}          | `SetClientLibrary`     | `true`                       | Whether to attempt to use `CLIENT SETINFO` to set the library name/version on the connection              |
+| protocol={string}      | `Protocol`             | `null`                       | Redis protocol to use; see section below                                                                  |
 
 Additional code-only options:
 - LoggerFactory (`ILoggerFactory`) - Default: `null`
@@ -123,16 +124,17 @@ Additional code-only options:
 Tokens in the configuration string are comma-separated; any without an `=` sign are assumed to be redis server endpoints. Endpoints without an explicit port will use 6379 if ssl is not enabled, and 6380 if ssl is enabled.
 Tokens starting with `$` are taken to represent command maps, for example: `$config=cfg`.
 
-Obsolete Configuration Options
+## Obsolete Configuration Options
 ---
+
 These options are parsed in connection strings for backwards compatibility (meaning they do not error as invalid), but no longer have any effect.
 
 | Configuration string   | `ConfigurationOptions` | Previous Default | Previous Meaning |
 | ---------------------- | ---------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------- |
 | responseTimeout={int} | `ResponseTimeout` | `SyncTimeout` | Time (ms) to decide whether the socket is unhealthy |
 | writeBuffer={int} | `WriteBuffer` | `4096` | Size of the output buffer |
 
-Automatic and Manual Configuration
+## Automatic and Manual Configuration
 ---
 
 In many common scenarios, StackExchange.Redis will automatically configure a lot of settings, including the server type and version, connection timeouts, and primary/replica relationships. Sometimes, though, the commands for this have been disabled on the redis server. In this case, it is useful to provide more information:
@@ -161,7 +163,8 @@ Which is equivalent to the command string:
 ```config
 redis0:6379,redis1:6380,keepAlive=180,version=2.8.8,$CLIENT=,$CLUSTER=,$CONFIG=,$ECHO=,$INFO=,$PING=
 ```
-Renaming Commands
+
+## Renaming Commands
 ---
 
 A slightly unusual feature of redis is that you can disable and/or rename individual commands. As per the previous example, this is done via the `CommandMap`, but instead of passing a `HashSet<string>` to `Create()` (to indicate the available or unavailable commands), you pass a `Dictionary<string,string>`. All commands not mentioned in the dictionary are assumed to be enabled and not renamed. A `null` or blank value records that the command is disabled. For example:
@@ -184,8 +187,9 @@ The above is equivalent to (in the connection string):
 $INFO=,$SELECT=use
 ```
 
-Redis Server Permissions
+## Redis Server Permissions
 ---
+
 If the user you're connecting to Redis with is limited, it still needs to have certain commands enabled for the StackExchange.Redis to succeed in connecting. The client uses:
 - `AUTH` to authenticate
 - `CLIENT` to set the client name
@@ -205,7 +209,7 @@ For example, a common _very_ minimal configuration ACL on the server (non-cluste
 
 Note that if you choose to disable access to the above commands, it needs to be done via the `CommandMap` and not only the ACL on the server (otherwise we'll attempt the command and fail the handshake). Also, if any of the these commands are disabled, some functionality may be diminished or broken.
 
-twemproxy
+## twemproxy
 ---
 
 [twemproxy](https://github.com/twitter/twemproxy) is a tool that allows multiple redis instances to be used as though it were a single server, with inbuilt sharding and fault tolerance (much like redis cluster, but implemented separately). The feature-set available to Twemproxy is reduced. To avoid having to configure this manually, the `Proxy` option can be used:
@@ -218,8 +222,9 @@ var options = new ConfigurationOptions
 };
 ```
 
-envoyproxy
+##envoyproxy
 ---
+
 [Envoyproxy](https://github.com/envoyproxy/envoy) is a tool that allows to front a redis cluster with a set of proxies, with inbuilt discovery and fault tolerance. The feature-set available to Envoyproxy is reduced. To avoid having to configure this manually, the `Proxy` option can be used:
 ```csharp
 var options = new ConfigurationOptions+{
@@ -229,7 +234,7 @@ var options = new ConfigurationOptions+{
 ```
 
 
-Tiebreakers and Configuration Change Announcements
+## Tiebreakers and Configuration Change Announcements
 ---
 
 Normally StackExchange.Redis will resolve primary/replica nodes automatically. However, if you are not using a management tool such as redis-sentinel or redis cluster, there is a chance that occasionally you will get multiple primary nodes (for example, while resetting a node for maintenance it may reappear on the network as a primary). To help with this, StackExchange.Redis can use the notion of a *tie-breaker* - which is only used when multiple primaries are detected (not including redis cluster, where multiple primaries are *expected*). For compatibility with BookSleeve, this defaults to the key named `"__Booksleeve_TieBreak"` (always in database 0). This is used as a crude voting mechanism to help determine the *preferred* primary, so that work is routed correctly.
@@ -240,8 +245,9 @@ Both options can be customized or disabled (set to `""`), via the `.Configuratio
 
 These settings are also used by the `IServer.MakeMaster()` method, which can set the tie-breaker in the database and broadcast the configuration change message. The configuration message can also be used separately to primary/replica changes simply to request all nodes to refresh their configurations, via the `ConnectionMultiplexer.PublishReconfigure` method.
 
-ReconnectRetryPolicy
+## ReconnectRetryPolicy
 ---
+
 StackExchange.Redis automatically tries to reconnect in the background when the connection is lost for any reason. It keeps retrying  until the connection has been restored. It would use ReconnectRetryPolicy to decide how long it should wait between the retries.
 ReconnectRetryPolicy can be exponential (default), linear or a custom retry policy.
 
@@ -266,3 +272,16 @@ config.ReconnectRetryPolicy = new LinearRetry(5000);
 //5	        5000
 //6	        5000
 ```
+
+## Redis protocol
+
+Without specific configuration, StackExchange.Redis will use the RESP2 protocol; this means that pub/sub requires a separate connection to the server. RESP3 is a newer protocol
+(usually, but not always, available on v6 servers and above) which allows (among other changes) pub/sub messages to be communicated on the *same* connection - which can be very
+desirable in servers with a large number of clients. The protocol handshake needs to happen very early in the connection, so *by default* the library does not attempt a RESP3 connection
+unless it has reason to expect it to work. 
+
+The library determines whether to use RESP3 by:
+- The `HELLO` command has been disabled: RESP2 is used
+- A protocol *other than* `resp3` or `3` is specified: RESP2 is used
+- A protocol of `resp3` or `3` is specified: RESP3 is attempted (with fallback if it fails)
+- In all other scenarios: RESP2 is used

docs/ReleaseNotes.md

@@ -8,6 +8,7 @@ Current package versions:
 
 ## Unreleased
 
+- Adds: RESP3 support ([#2396 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/2396)) - see https://stackexchange.github.io/StackExchange.Redis/Resp3
 - Fix [#2507](https://github.com/StackExchange/StackExchange.Redis/issues/2507): Pub/sub with multi-item payloads should be usable ([#2508 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/2508))
 - Add: connection-id tracking (internal only, no public API) ([#2508 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/2508))
 - Add: `ConfigurationOptions.LoggerFactory` for logging to an `ILoggerFactory` (e.g. `ILogger`) all connection and error events ([#2051 by NickCraver](https://github.com/StackExchange/StackExchange.Redis/pull/2051))

docs/Resp3.md

@@ -0,0 +1,45 @@
+# RESP3 and StackExchange.Redis
+
+RESP2 and RESP3 are evolutions of the Redis protocol, with RESP3 existing from Redis server version 6 onwards (v7.2+ for Redis Enterprise). The main differences are:
+
+1. RESP3 can carry out-of-band / "push" messages on a single connection, where-as RESP2 requires a separate connection for these messages
+2. RESP3 can (when appropriate) convey additional semantic meaning about returned payloads inside the same result structure
+3. Some commands (see [this topic](https://github.com/redis/redis-doc/issues/2511)) return different result structures in RESP3 mode; for example a flat interleaved array might become a jagged array
+
+For most people, #1 is the main reason to consider RESP3, as in high-usage servers - this can halve the number of connections required.
+This is particularly useful in hosted environments where the number of inbound connections to the server is capped as part of a service plan.
+Alternatively, where users are currently choosing to disable the out-of-band connection to achieve this, they may now be able to re-enable this
+(for example, to receive server maintenance notifications) *without* incurring any additional connection overhead.
+
+Because of the significance of #3 (and to avoid breaking your code), the library does not currently default to RESP3 mode. This must be enabled explicitly
+via `ConfigurationOptions.Protocol` or by adding `,protocol=resp3` (or `,protocol=3`) to the configuration string.
+
+---
+
+#3 is a critical one - the library *should* already handle all documented commands that have revised results in RESP3, but if you're using
+`Execute[Async]` to issue ad-hoc commands, you may need to update your processing code to compensate for this, ideally using detection to handle
+*either* format so that the same code works in both REP2 and RESP3. Since the impacted commands are handled internally by the library, in reality
+this should not usually present a difficulty.
+
+The minor (#2) and major (#3) differences to results are only visible to your code when using:
+
+- Lua scripts invoked via the `ScriptEvaluate[Async](...)` or related APIs, that either:
+  - Uses the `redis.setresp(3)` API and returns a value from `redis.[p]call(...)`
+  - Returns a value that satisfies the [LUA to RESP3 type conversion rules](https://redis.io/docs/manual/programmability/lua-api/#lua-to-resp3-type-conversion)
+- Ad-hoc commands (in particular: *modules*) that are invoked via the `Execute[Async](string command, ...)` API
+
+...both which return `RedisResult`. **If you are not using these APIs, you should not need to do anything additional.**
+
+Historically, you could use the `RedisResult.Type` property to query the type of data returned (integer, string, etc). In particular:
+
+- Two new properties are added: `RedisResult.Resp2Type` and `RedisResult.Resp3Type`
+  - The `Resp3Type` property exposes the new semantic data (when using RESP3) - for example, it can indicate that a value is a double-precision number, a boolean, a map, etc (types that did not historically exist)
+  - The `Resp2Type` property exposes the same value that *would* have been returned if this data had been returned over RESP2
+  - The `Type` property is now marked obsolete, but functions identically to `Resp2Type`, so that pre-existing code (for example, that has a `switch` on the type) is not impacted by RESP3
+- The `ResultType.MultiBulk` is superseded by `ResultType.Array` (this is a nomenclature change only; they are the same value and function identically)
+
+Possible changes required due to RESP3:
+
+1. To prevent build warnings, replace usage of `ResultType.MultiBulk` with `ResultType.Array`, and usage of `RedisResult.Type` with `RedisResult.Resp2Type`
+2. If you wish to exploit the additional semantic data when enabling RESP3, use `RedisResult.Resp3Type` where appropriate
+3. If you are enabling RESP3, you must verify whether the commands you are using can give different result shapes on RESP3 connections

docs/index.md

@@ -39,6 +39,7 @@ Documentation
 - [Transactions](Transactions) - how atomic transactions work in redis
 - [Events](Events) - the events available for logging / information purposes
 - [Pub/Sub Message Order](PubSubOrder) - advice on sequential and concurrent processing
+- [Using RESP3](Resp3) - information on using RESP3
 - [ServerMaintenanceEvent](ServerMaintenanceEvent) - how to listen and prepare for hosted server maintenance (e.g. Azure Cache for Redis)
 - [Streams](Streams) - how to use the Stream data type
 - [Where are `KEYS` / `SCAN` / `FLUSH*`?](KeysScan) - how to use server-based commands

src/StackExchange.Redis/APITypes/LatencyHistoryEntry.cs

@@ -13,7 +13,7 @@ private sealed class Processor : ArrayResultProcessor<LatencyHistoryEntry>
     {
         protected override bool TryParse(in RawResult raw, out LatencyHistoryEntry parsed)
         {
-            if (raw.Type == ResultType.MultiBulk)
+            if (raw.Resp2TypeArray == ResultType.Array)
             {
                 var items = raw.GetItems();
                 if (items.Length >= 2

src/StackExchange.Redis/APITypes/LatencyLatestEntry.cs

@@ -13,7 +13,7 @@ private sealed class Processor : ArrayResultProcessor<LatencyLatestEntry>
     {
         protected override bool TryParse(in RawResult raw, out LatencyLatestEntry parsed)
         {
-            if (raw.Type == ResultType.MultiBulk)
+            if (raw.Resp2TypeArray == ResultType.Array)
             {
                 var items = raw.GetItems();
                 if (items.Length >= 4

src/StackExchange.Redis/ChannelMessageQueue.cs

@@ -1,9 +1,11 @@
 using System;
 using System.Collections.Generic;
-using System.Reflection;
 using System.Threading;
 using System.Threading.Channels;
 using System.Threading.Tasks;
+#if NETCOREAPP3_1
+using System.Reflection;
+#endif
 
 namespace StackExchange.Redis
 {
@@ -125,6 +127,7 @@ public ValueTask<ChannelMessage> ReadAsync(CancellationToken cancellationToken =
         /// <param name="count">The (approximate) count of items in the Channel.</param>
         public bool TryGetCount(out int count)
         {
+            // This is specific to netcoreapp3.1, because full framework was out of band and the new prop is present
 #if NETCOREAPP3_1
             // get this using the reflection
             try

src/StackExchange.Redis/ClientInfo.cs

@@ -181,18 +181,23 @@ public ClientType ClientType
         }
 
         /// <summary>
-        /// Client RESP protocol version. Added in Redis 7.0
+        /// Client RESP protocol version. Added in Redis 7.0.
         /// </summary>
         public string? ProtocolVersion { get; private set; }
 
         /// <summary>
-        /// Client library name. Added in Redis 7.2
+        /// Client RESP protocol version. Added in Redis 7.0.
+        /// </summary>
+        public RedisProtocol? Protocol => ConfigurationOptions.TryParseRedisProtocol(ProtocolVersion, out var value) ? value : null;
+
+        /// <summary>
+        /// Client library name. Added in Redis 7.2.
         /// </summary>
         /// <remarks><seealso href="https://redis.io/commands/client-setinfo"/></remarks>
         public string? LibraryName { get; private set; }
 
         /// <summary>
-        /// Client library version. Added in Redis 7.2
+        /// Client library version. Added in Redis 7.2.
         /// </summary>
         /// <remarks><seealso href="https://redis.io/commands/client-setinfo"/></remarks>
         public string? LibraryVersion { get; private set; }
@@ -280,7 +285,7 @@ private class ClientInfoProcessor : ResultProcessor<ClientInfo[]>
         {
             protected override bool SetResultCore(PhysicalConnection connection, Message message, in RawResult result)
             {
-                switch(result.Type)
+                switch(result.Resp2TypeBulkString)
                 {
                     case ResultType.BulkString:
                         var raw = result.GetString();