Token Limiting

[SethTurin]

For my use case, I needed to set limits to how many tokens can be use in a timeframe, so I wrote some code for it. It ended up being a bigger edit that I anticipated, but I think the integration is clean and reusable.

If others feel this would be a useful feature, I can put it into a pull request.

[danny-avila]

If you feel that it’s easy to toggle or adjust as needed, please do so! Though the current implementation won’t give you the most accurate token usage. I am working on that through a PR to a dependency library

[jhcao23]

tokens per minute, request per minute, and the number of *OPENAI_API_KEY accounts the corporate owns ==> they together should be considered together. @danny-avila can you pls share with us, which lib you are currently interested into?