http: add support for HTTP 429 rate limit retries

Add retry logic for HTTP 429 (Too Many Requests) responses to handle
server-side rate limiting gracefully. When Git's HTTP client receives
a 429 response, it can now automatically retry the request after an
appropriate delay, respecting the server's rate limits.

The implementation supports the RFC-compliant Retry-After header in
both delay-seconds (integer) and HTTP-date (RFC 2822) formats. If a
past date is provided, Git retries immediately without waiting.

Retry behavior is controlled by three new configuration options:

  * http.maxRetries: Maximum number of retry attempts (default: 0,
    meaning retries are disabled by default). Users must explicitly
    opt-in to retry behavior.

  * http.retryAfter: Default delay in seconds when the server doesn't
    provide a Retry-After header (default: -1, meaning fail if no
    header is provided). This serves as a fallback mechanism.

  * http.maxRetryTime: Maximum delay in seconds for a single retry
    (default: 300). If the server requests a delay exceeding this
    limit, Git fails immediately rather than waiting. This prevents
    indefinite blocking on unreasonable server requests.

All three options can be overridden via environment variables:
GIT_HTTP_MAX_RETRIES, GIT_HTTP_RETRY_AFTER, and
GIT_HTTP_MAX_RETRY_TIME.

The retry logic implements a fail-fast approach: if any delay
(whether from server header or configuration) exceeds maxRetryTime,
Git fails immediately with a clear error message rather than capping
the delay. This provides better visibility into rate limiting issues.

Trace2 logging has been added to track retry attempts, delays, and
error conditions. This enables monitoring and debugging of rate limit
scenarios in production environments.

The implementation includes extensive test coverage for basic retry
behavior, Retry-After header formats (integer and HTTP-date),
configuration combinations, maxRetryTime limits, invalid header
handling, environment variable overrides, and edge cases.

Signed-off-by: Vaidas Pilkauskas <[email protected]>

Author: vaidas-shopify

Documentation/config/http.adoc

@@ -315,6 +315,30 @@ http.keepAliveCount::
 	unset, curl's default value is used. Can be overridden by the
 	`GIT_HTTP_KEEPALIVE_COUNT` environment variable.
 
+http.retryAfter::
+	Default wait time in seconds before retrying when a server returns
+	HTTP 429 (Too Many Requests) without a Retry-After header. If set
+	to -1 (the default), Git will fail immediately when encountering
+	a 429 response without a Retry-After header. When a Retry-After
+	header is present, its value takes precedence over this setting.
+	Can be overridden by the `GIT_HTTP_RETRY_AFTER` environment variable.
+	See also `http.maxRetries` and `http.maxRetryTime`.
+
+http.maxRetries::
+	Maximum number of times to retry after receiving HTTP 429 (Too Many
+	Requests) responses. Set to 0 (the default) to disable retries.
+	Can be overridden by the `GIT_HTTP_MAX_RETRIES` environment variable.
+	See also `http.retryAfter` and `http.maxRetryTime`.
+
+http.maxRetryTime::
+	Maximum time in seconds to wait for a single retry attempt when
+	handling HTTP 429 (Too Many Requests) responses. If the server
+	requests a delay (via Retry-After header) or if `http.retryAfter`
+	is configured with a value that exceeds this maximum, Git will fail
+	immediately rather than waiting. Default is 300 seconds (5 minutes).
+	Can be overridden by the `GIT_HTTP_MAX_RETRY_TIME` environment
+	variable. See also `http.retryAfter` and `http.maxRetries`.
+
 http.noEPSV::
 	A boolean which disables using of EPSV ftp command by curl.
 	This can be helpful with some "poor" ftp servers which don't

http-push.c

@@ -716,6 +716,10 @@ static int fetch_indices(void)
 	case HTTP_MISSING_TARGET:
 		ret = 0;
 		break;
+	case HTTP_RATE_LIMITED:
+		error("rate limited by '%s', please try again later", repo->url);
+		ret = -1;
+		break;
 	default:
 		ret = -1;
 	}
@@ -1548,6 +1552,10 @@ static int remote_exists(const char *path)
 	case HTTP_MISSING_TARGET:
 		ret = 0;
 		break;
+	case HTTP_RATE_LIMITED:
+		error("rate limited by '%s', please try again later", url);
+		ret = -1;
+		break;
 	case HTTP_ERROR:
 		error("unable to access '%s': %s", url, curl_errorstr);
 		/* fallthrough */

http-walker.c

@@ -414,6 +414,11 @@ static int fetch_indices(struct walker *walker, struct alt_base *repo)
 		repo->got_indices = 1;
 		ret = 0;
 		break;
+	case HTTP_RATE_LIMITED:
+		error("rate limited by '%s', please try again later", repo->base);
+		repo->got_indices = 0;
+		ret = -1;
+		break;
 	default:
 		repo->got_indices = 0;
 		ret = -1;