docs: clarify OpenAIClientAsync rate-limit note#729
docs: clarify OpenAIClientAsync rate-limit note#729Rohan5commit wants to merge 1 commit intoopenai:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Updates OpenAIClientAsync KDoc to remove the implication that the SDK performs proactive per-client rate limiting, and instead describe how behavior is scoped to the client instance while external rate limits are shared.
Changes:
- Replaces “rate limiting per client” wording with a description focused on per-client retry behavior and shared external rate limits.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * Reusing connections and threads reduces latency and saves memory. Each client also manages its | ||
| * own retries independently, so reusing one client keeps that retry behavior, connection pool, and | ||
| * thread pools in one place while multiple clients may compete separately against the same external | ||
| * rate limits. |
There was a problem hiding this comment.
The new KDoc still reads a bit like “client-level” retry/rate-limit behavior ("manages its own retries independently"), but retries are implemented per request (reactive on e.g. 429) and there’s no cross-request/cross-client coordination. Consider wording this explicitly (e.g., retries are per request and the SDK does not proactively throttle or coordinate rate limiting across client instances) to avoid repeating the confusion from #644.
| * Reusing connections and threads reduces latency and saves memory. Each client also manages its | |
| * own retries independently, so reusing one client keeps that retry behavior, connection pool, and | |
| * thread pools in one place while multiple clients may compete separately against the same external | |
| * rate limits. | |
| * Reusing connections and threads reduces latency and saves memory. Retries are applied per | |
| * request, reactively based on API responses such as rate limits, and are not coordinated across | |
| * requests or client instances. The SDK does not proactively throttle requests or coordinate rate | |
| * limiting across multiple clients, so separate clients may still compete against the same | |
| * external rate limits. |
Summary
Clarify the
OpenAIClientAsyncdocumentation so it describes per-client retry behavior without implying proactive global rate limiting.Related issue
Fixes #644
Guideline alignment
Read
CONTRIBUTING.mdand kept this to a single-file documentation comment change.Validation
git diff --check