Retry prompt on HTTP error code(s)

[zikajk]

When LLM returns 429, we should repeat the request (and cancel it at any time).
A good default is to do it 10 times with the interval increasing by 2 times each time (2s, 4s, 8s, ...).

It also makes sense to have the model name readily available so we can eventually use different HTTP error codes for different models, if this is not enough.

To upvote this issue, give it a thumbs up. See this list for the most upvoted issues.

[ericdallo]

Interesting, I think one thing we can already do is mention the status code always on every errors, because I don't think I faced too many 429 errors, or I just don't know it was a 429

[zikajk]

We do mention the status code.

429 is the one that gets retry in most agents.
There might be some problems, e.g. Claude Code sub uses 429 when your limit is gone but there is no harm in retrying (10 times max) and allowing user to cancel IMHO.

It will be very useful with Gemini through proxy, Kimi-K2 and other models that returns smth like "Usage is too high, try again later" quite often.

[ericdallo]

Makes sense, maybe the status code could be configurable? dunno... with 429 as default