automatically handle rate limit

[aa755]

Do any one have a variant of openai-chat that would automatically handle rate limit: retry automatically after the time openai suggests in the error?

[jcs090218]

Not sure what to handle. The rate limit means your account is running out of quota. 🤔 Retry wouldn't help in these circumstances.

[aa755]

Here is an example response I was getting from the openai-chat function in this repo:

((error (message . Rate limit reached for gpt-4o in organization xxxxxxxxxx on tokens per min (TPM): Limit 30000, Used 25317, Requested 8450. Please try again in 7.534s. Visit https://platform.openai.com/account/rate-limits to learn more.) (type . tokens) (param) (code . rate_limit_exceeded)))

It was a long prompt (27KB text).
For now, I added a 60s sleep before all calls to openai-chat and I never get this error anymore. But this is too conservative. Ideally, openai-chat should have an option to automatically try after waiting for the time mentioned in the error message (7.534s in this case)