Does litellm handles rate limit?

[windoliver]

Does litellm handles rate limit?

[krrishdholakia]

Hey @OliverFeng LiteLLM provides router functionality to manage rate limits and rotate between different deployments - https://litellm.vercel.app/docs/routing

[solyarisoftware]

So, in LiteLLM must I use router functionality to manage rate limits having a single deployment?
Following the example here: https://litellm.vercel.app/docs/routing#advanced ?

[ishaan-jaff]

yes, are you hitting rate limit errors @solyarisoftware ?

[solyarisoftware]

No. BTW I'm using Azure deployments where you can set the rate limit value for each specific deployment, as you maybe know. What I want to elaborate is monitoring in real-time the rate limit to be alerted when I'm reaching the threshold set.