AI Gateway breaks SDK retry: GatewayInternalServerError not recognized by maxRetries

[shtefcs]

Description

The AI Gateway's error handling converts retryable APICallError instances into GatewayInternalServerError instances before they reach the SDK's retry logic. Since the retry function checks APICallError.isInstance(error), Gateway errors are never retried regardless of the maxRetries setting.

Proof

Ran this in Node.js with the installed packages:

const gateway = require('@ai-sdk/gateway/dist/index.js');

const gErr = new gateway.GatewayInternalServerError({ message: 'test', statusCode: 503 });
const apiCallMarker = Symbol.for('vercel.ai.error.AI_APICallError');

console.log('Has APICallError marker:', apiCallMarker in gErr);  // false
console.log('isRetryable:', gErr.isRetryable);                    // undefined
console.log('statusCode:', gErr.statusCode);                      // 503

The error has the Gateway marker (vercel.ai.gateway.error) but NOT the APICallError marker (vercel.ai.error.AI_APICallError), so maxRetries never activates.

Root cause

In @ai-sdk/gateway, the doStream() and doGenerate() methods catch errors and convert them:

// @ai-sdk/gateway/dist/index.js, line ~875
} catch (error) {
  throw await asGatewayError(error, ...);
}

asGatewayError() (line ~433) takes a retryable APICallError (statusCode 503, isRetryable: true) and converts it to a GatewayInternalServerError which extends GatewayError (NOT APICallError).

The SDK's retry function at ai/dist/internal/index.mjs:1207 then fails:

if (error instanceof Error 
    && APICallError.isInstance(error)      // FALSE for GatewayError
    && error.isRetryable === true          // undefined for GatewayError
    && tryNumber <= maxRetries) {
  // This NEVER executes for Gateway errors
}

Impact

Any application using @ai-sdk/gateway with maxRetries > 0 gets zero retries on transient errors (503, 429). The setting is silently ignored. This affects streamText, generateText, generateObject, and streamObject.

Applications using provider SDKs directly (@ai-sdk/google, @ai-sdk/openai) are not affected because their errors pass through as APICallError.

Expected behavior

Gateway errors with retryable status codes (503, 429, 408) should be retried according to maxRetries.

Possible fixes

1. Make GatewayError extend APICallError (or carry the marker)
2. Make GatewayError set isRetryable based on status code
3. Update the retry function to also check GatewayError.isInstance()
4. Don't convert retryable APICallError to GatewayError in the catch block

Environment

• ai: 6.0.141
• @ai-sdk/gateway: 3.0.83

[PiniShv]

Confirmed. Traced the full path through the source.

_retryWithExponentialBackoff (here) only checks APICallError.isInstance(error). But GatewayLanguageModel.doGenerate and doStream catch errors and re-throw through asGatewayError(). That wraps the original APICallError into GatewayInternalServerError, GatewayRateLimitError, etc. These extend GatewayError, not APICallError. No isRetryable. No responseHeaders. Retry check fails every time.

The original APICallError is sitting right there in cause. The retry logic never looks at it.

Fix is two parts:

1. Add isRetryable and responseHeaders to GatewayError base class. Have asGatewayError propagate them from the original APICallError. Retryable subclasses (GatewayInternalServerError, GatewayRateLimitError, GatewayTimeoutError) default to isRetryable = true.

2. Extend the retry check to also match GatewayError.isInstance(error) && error.isRetryable === true. packages/ai already depends on @ai-sdk/gateway so the import is clean.

No inheritance change needed. Gateway errors stay their own type. They just become visible to the retry layer.

Happy to put together a PR.

[shtefcs]

Thanks for confirming and tracing this through the source. Your proposed fix is exactly right.

We have a middleware workaround deployed on our end that catches GatewayError at the model level and retries with backoff, but the proper fix belongs in the SDK like you described.

Are you planning to put the PR together? If so we are happy to test it against our production workload (multi-step browser automation with 30+ tool call steps per session, hits 503s regularly). If you are not going to get to it in the next few days, we can take a crack at it ourselves. Just want to make sure we don't duplicate work.

[PiniShv]

PR is up: #14233

Ended up keeping it simpler than what I originally outlined. Instead of adding responseHeaders to GatewayError and modifying asGatewayError, the retry logic unwraps error.cause to get the headers from the original APICallError. Same result, fewer files touched.

@shtefcs happy to have you test it against your workload when you get a chance.