Update retry guidance and example

This updates the suggested retry strategy interface to accept info
about the operation being invoked in the initial token acquisition
method. This also updates the example strategy to match the new
behaviors that are being rolled out for the AWS standard retry
strategy, which includes checking for a long-polling trait on
the operation.

Author: JordonPhillips

.changes/next-release/documentation-67369516e990b0f6aade1d3e8df34c8537eaa21e.json

@@ -0,0 +1,5 @@
+{
+  "type": "documentation",
+  "description": "Updated the example retry strategy in client guidance and updated the initial token method to take information about the operation.",
+  "pull_requests": []
+}

docs/source-2.0/guides/client-guidance/retries.md

@@ -68,7 +68,7 @@ public interface RetryStrategy {
      *
      * @throws TokenAcquisitionFailedException if a token cannot be acquired.
      */
-    RetryToken acquireInitialToken();
+    RetryToken acquireInitialToken(ApiOperation<?, ?> operation);
 
     /**
      * Invoked before each subsequent (non-first) request attempt.
@@ -100,6 +100,13 @@ client. Be careful to ensure that access to that state is synchronized in order
 to prevent race conditions.
 :::
 
+:::{admonition} TODO - Define ApiOperation
+:class: note
+
+`ApiOperation` will be defined later in a separate document. At a minimum, it
+should contain the operation's ID.
+:::
+
 #### Using retry strategies
 
 An initial retry token should be acquired at the beginning of a request, before
@@ -129,12 +136,12 @@ The following is a simplified example of what it looks like to use the
  *
  * @return a successful result.
  */
-public Result request(SerializedRequest serializedRequest) {
+public Result request(ApiOperation<?, ?> operation, SerializedRequest serializedRequest) {
     // First acquire the initial retry token. If a token cannot be acquired,
     // make only one attempt without retries.
     RetryToken retryToken;
     try {
-        retryToken = this.retryStrategy.acquireInitialToken();
+        retryToken = this.retryStrategy.acquireInitialToken(operation);
     } catch (TokenAcquisitionFailedException e) {
         return send(serializedRequest);
     }
@@ -413,25 +420,34 @@ demonstrate some of the potential needs of a retry system.
 ### Example retry strategy
 
 The following is an example retry strategy that implements exponential backoff
-with jitter alongside a token bucket. This strategy adds extra cost for timeout
-errors since they may indicate a more degraded service.
+with jitter alongside a token bucket. This strategy has a reduced cost for
+throttling errors as they indicate that the service is actively managing
+retries.
 
 Aside from delay, the retry token also tracks the number of attempts that have
-been made. This is necessary because this strategy imposes a maximum attempt
-count, and also because the delay is calculated in part based on how many
-attempts have been made.
+been made as well as if the operation is a long-polling operation. The attempt
+count is necessary because this strategy imposes a maximum attempt count, and
+also because the delay is calculated in part based on how many attempts have
+been made.
+
+For long-polling operations, the strategy will continue to back off even if the
+token bucket is empty.
 
 ```java
-public record AwsStandardRetryToken(int attempts, Duration delay) implements RetryToken {
+public record AwsStandardRetryToken(
+        int attempts,
+        Duration delay,
+        boolean isLongPoll
+) implements RetryToken {
 }
 ```
 
 ```java
 public final class AwsStandardRetryStrategy implements RetryStrategy {
     // These values are not prescriptive. They are static in this example for the
     // sake of simplicity, but making them configurable is ideal.
-    private static final int RETRY_COST = 5;
-    private static final int TIMEOUT_COST = 10;
+    private static final int RETRY_COST = 14;
+    private static final int THROTTLE_RETRY_COST = 5;
     private static final int SUCCESS_REFUND = 1;
 
     private static final int MAX_ATTEMPTS = 5;
@@ -449,13 +465,14 @@ public final class AwsStandardRetryStrategy implements RetryStrategy {
     private final Object tokensLock = new Object();
 
     @Override
-    public RetryToken acquireInitialToken() {
+    public RetryToken acquireInitialToken(ApiOperation<?, ?> operation) {
         // This returns successfully even if the token bucket is empty. This is
         // because an initial attempt will always be performed anyway, and
         // returning successfully here will ensure that the retry strategy is
         // checked if that initial attempt fails. By that point, the token bucket
         // may no longer be empty.
-        return new AwsStandardRetryToken(0, null);
+        boolean isLongPoll = operation.schema().hasTrait(TraitKey.get(LongPollTrait.class));
+        return new AwsStandardRetryToken(0, null, isLongPoll);
     }
 
     @Override
@@ -479,16 +496,16 @@ public final class AwsStandardRetryStrategy implements RetryStrategy {
             case RetryInfo retryInfo when retryInfo.isRetrySafe() != RetrySafety.NO -> {
                 // Attempt to consume tokens from the token bucket to "pay"
                 // for the retry.
-                consumeTokens(retryInfo.isTimeout());
-                yield backoff(standardToken, retryInfo.retryAfter());
+                consumeTokens(retryInfo.isThrottle(), standardToken.isLongPoll());
+                yield backoff(standardToken, retryInfo.retryAfter(), retryInfo.isThrottle());
             }
 
             // If the exception does not have retry info, but does have more
             // general error info, that can also be used. This assumes that
             // a server error is likely retryable and that a client error
             // likely is not.
             case ErrorInfo errorInfo when errorInfo.fault() == ErrorFault.SERVER -> {
-                consumeTokens(false);
+                consumeTokens(false, standardToken.isLongPoll());
                 yield backoff(standardToken);
             }
             default -> throw new TokenAcquisitionFailedException("Exception not retryable.");
@@ -498,15 +515,24 @@ public final class AwsStandardRetryStrategy implements RetryStrategy {
     /**
      * Consumes tokens to "pay" for a retry.
      *
-     * @param isTimeout whether the retry is in response to a timeout error,
-     *     which will require more tokens.
+     * @param isThrottle whether the retry is in response to a throttling error,
+     *     which will require fewer tokens.
+     * @param isLongPoll whether the operation is a long-polling operation. If
+     *     so, a retry will always be performed even if the bucket doesn't have
+     *     enough tokens.
      *
      * @throws TokenAcquisitionFailedException if there are not enough tokens
      *     in the bucket to pay for the retry.
      */
-    private void consumeTokens(boolean isTimeout) {
+    private void consumeTokens(boolean isThrottle, boolean isLongPoll) {
         synchronized (tokensLock) {
-            int cost = isTimeout ? TIMEOUT_COST : RETRY_COST;
+            int cost = isThrottle ? THROTTLE_RETRY_COST : RETRY_COST;
+
+            // Long-polling operations will always backoff. If the bucket doesn't have
+            // enough tokens, it will just be emptied.
+            if (isLongPoll) {
+                cost = Math.min(cost, this.tokens);
+            }
 
             if (this.tokens < cost) {
                 throw new TokenAcquisitionFailedException("Token bucket exhausted.");
@@ -522,41 +548,51 @@ public final class AwsStandardRetryStrategy implements RetryStrategy {
      * @param token the previous token.
      */
     private AwsStandardRetryToken backoff(AwsStandardRetryToken token) {
-        return new AwsStandardRetryToken(token.attempts + 1, computeDelay(token.attempts));
+        return new AwsStandardRetryToken(
+            token.attempts + 1, computeDelay(token.attempts, false), token.isLongPoll);
     }
 
     /**
      * Computes a backoff with exponential backoff and jitter, capped at 20 seconds.
      *
      * @param token the previous token.
+     * @param isThrottle whether the triggering error was a throttle.
      * @param suggested the delay suggested by the service, which will serve as
      *     the minimum delay.
      */
-    private AwsStandardRetryToken backoff(AwsStandardRetryToken token, Duration suggested) {
+    private AwsStandardRetryToken backoff(AwsStandardRetryToken token, Duration suggested, boolean isThrottle) {
         // Compute the backoff as normal. If it is longer than the suggested
         // backoff from the service, use it. Otherwise, use the suggested
         // backoff.
-        Duration computedDelay = computeDelay(token.attempts);
-        Duration finalDelay = computedDelay.toMillis() < suggested.toMillis() ? suggested : computedDelay;
-        return new AwsStandardRetryToken(token.attempts + 1, finalDelay);
+        Duration finalDelay = computeDelay(token.attempts, isThrottle);
+        if (suggested != null && finalDelay.toMillis() < suggested.toMillis()) {
+            finalDelay = suggested;
+        }
+        return new AwsStandardRetryToken(token.attempts + 1, finalDelay, token.isLongPoll);
     }
 
     /**
      * Computes the delay with exponential backoff and jitter, capped at 20 seconds.
      *
      * @param attempts the number of attempts made so far.
+     * @param isThrottle whether the triggering error was a throttle.
      * @return the computed delay duration.
      */
-    private Duration computeDelay(int attempts) {
+    private Duration computeDelay(int attempts, boolean isThrottle) {
         // First compute the exponential backoff.
         double backoff = Math.pow(2, attempts);
 
+        // Try to recover faster from non-throttling errors.
+        if (!isThrottle) {
+            backoff = backoff * 0.05;
+        }
+
         // Next, cap it at 20 seconds.
         backoff = Math.min(backoff, MAX_BACKOFF);
 
         // Finally, add jitter and expand to milliseconds.
         double backoffMillis = Math.random() * backoff * 1000;
-        return Duration.ofMilliseconds((long) backoffMillis);
+        return Duration.ofMillis((long) backoffMillis);
     }
 
     @Override