fix: improve exponential backoff calculation

Given the default `MinRetryBackoff` of 8 ms, the previous implementation
for calculating the backoff duration was this:

1. Calculate exponential:

```
d := minBackoff << uint(retry)
// Retry 0: d = 8ms
// Retry 1: d = 16ms
// Retry 2: d = 32ms
// Retry 3: d = 64ms
```

2. Replace with linear jitter:

```
d = minBackoff + time.Duration(rand.Int63n(int64(d)))
// Retry 0: d = 8ms + random(0, 8ms)  = 8-16ms
// Retry 1: d = 8ms + random(0, 16ms) = 8-24ms
// Retry 2: d = 8ms + random(0, 32ms) = 8-40ms
// Retry 3: d = 8ms + random(0, 64ms) = 8-72ms
```

The average delays show this isn't really exponential:

```
Retry 0: avg = 8 + 4   = 12ms
Retry 1: avg = 8 + 8   = 16ms  (1.33x growth)
Retry 2: avg = 8 + 16  = 24ms  (1.5x growth)
Retry 3: avg = 8 + 32  = 40ms  (1.67x growth)
```

This is actually linear growth with exponential jitter range:

```
delay = constant_base + random(0, exponential_range)
```

As described in
https://aws.amazon.com/ko/blogs/architecture/exponential-backoff-and-jitter/,
we want:

```
d := minBackoff << uint(retry)
d += random(0, d)
// Retry 0: d = 8ms + random(0, 8ms)  = 8-16ms
// Retry 1: d = 16ms + random(0, 16ms) = 16-32ms
// Retry 2: d = 32ms + random(0, 32ms) = 32-64ms
// Retry 3: d = 64s + random(0, 64ms) = 64-128ms
```

Note that even with this change, Redis Cluster may still not have enough
breathing room for cluster discovery. `MaxRetries`, `MinRetryBackoff`,
and `MaxRetryBackoff` likely still need to be tweaked.

Relates to #2046

Author: stanhu

internal/internal.go

@@ -16,13 +16,14 @@ func RetryBackoff(retry int, minBackoff, maxBackoff time.Duration) time.Duration
 
 	d := minBackoff << uint(retry)
 	if d < minBackoff {
-		return maxBackoff
+		d = minBackoff
 	}
 
-	d = minBackoff + time.Duration(rand.Int63n(int64(d)))
+	jitter := time.Duration(rand.Int63n(int64(d)))
 
-	if d > maxBackoff || d < minBackoff {
-		d = maxBackoff
+	d = d + jitter
+	if d > maxBackoff {
+		return maxBackoff
 	}
 
 	return d

internal/internal_test.go

@@ -14,5 +14,18 @@ func TestRetryBackoff(t *testing.T) {
 		backoff := RetryBackoff(i, time.Millisecond, 512*time.Millisecond)
 		Expect(backoff >= 0).To(BeTrue())
 		Expect(backoff <= 512*time.Millisecond).To(BeTrue())
+
+		// Test that the maximum possible value grows exponentially
+		if i > 0 {
+			prevMax := time.Millisecond << uint(i-1)
+			currentMax := time.Millisecond << uint(i)
+
+			if currentMax <= 512*time.Millisecond {
+				ratio := float64(currentMax) / float64(prevMax)
+				Expect(ratio).To(Equal(2.0),
+					"Exponential growth expected: max for retry %d=%v, max for retry %d=%v, ratio=%.1f",
+					i-1, prevMax, i, currentMax, ratio)
+			}
+		}
 	}
 }