HTTP Delivery Service should have retries with exponential backoff

[neekolas]

The HTTP delivery service should be able to be configured with a maxRetries configuration option and an initialRetryDelay configuration option. Both should be optional and default to maxRetries: 1, initialRetryDelay: 250

It should attempt failed deliveries up to maxRetries times, sleeping between each attempt starting with an initialRetryDelay sleep time. Use exponential backoff to increase the sleep time between attempts.

[xmtp-coder-tasks]

Task created: https://sandbox.xmtp.team/tasks/neekolas/a514cf0d-f4fd-43ad-9506-1259b2f7ff60

[xmtp-coder-agent]

Design Spec: HTTP Delivery Retry with Exponential Backoff

Requirements (EARS format)

R1 (Configuration - MaxRetries): When the HttpDeliveryOptions struct is configured, the system shall accept an optional MaxRetries integer field (CLI flag --http-max-retries, env HTTP_MAX_RETRIES) that defaults to 1.

R2 (Configuration - InitialRetryDelay): When the HttpDeliveryOptions struct is configured, the system shall accept an optional InitialRetryDelay integer field in milliseconds (CLI flag --http-initial-retry-delay, env HTTP_INITIAL_RETRY_DELAY) that defaults to 250.

R3 (Retry behavior): When the HTTP delivery Send method receives a non-200 response or a transient error, the system shall retry the request up to MaxRetries times.

R4 (Exponential backoff): When retrying, the system shall sleep for InitialRetryDelay * 2^attempt milliseconds between attempts (attempt 0 = first retry).

R5 (Context cancellation): When the context is cancelled during a retry loop, the system shall return the context error immediately without further retries.

R6 (Non-retryable errors): When json.Marshal or http.NewRequestWithContext fails, the system shall return the error immediately without retrying.

R7 (Logging): When a retry occurs, the system shall log the attempt number and error at warn level.

R8 (Final error): When all retry attempts are exhausted, the system shall return the last error encountered.

Design

• Add MaxRetries (default 1) and InitialRetryDelay (default 250ms) to HttpDeliveryOptions
• Extract single HTTP call to helper method, wrap in retry loop with exponential backoff
• Use context-aware sleep (select on time.After + ctx.Done()) for clean cancellation
• Full unit test coverage for retry, backoff, context cancellation, and non-retryable errors

Working on implementation now.