Assistants V2 - Code Interpreter - Rate Limt

[glen-fdce]

I'm trying out 1.18.0-alpha and the new Assistants V2 capabilities with Code Interpreter.

It works well with file search but with code interpreter I'm mostly getting rate limited by AOAI (eg. Rate limit is exceeded. Try again in 9 seconds.)

I note that polling is done to the thread every second, is that configurable? (Eg. call every 5 or 10 seconds instead)?

Thanks

Edit: I note using Azure.AI.OpenAI - I can poll manually (and can do so every five seconds to avoid rate limits). Will add as issue since default polling in SK appear to be problematic.

[rajyraman]

I experienced this issue as well.

[crickman]

You should be able to configure polling frequency using the PollingOptions property:

https://github.com/microsoft/semantic-kernel/blob/feature-connectors-openai/dotnet/src/Agents/OpenAI/OpenAIAssistantAgent.cs#L56

I'd be curious to understand more clearly on the configuration of your deployment. I was under the impression that throttling was based on tokens. Are you able to share the configuration of your rate limit?

[glen-fdce]

Thanks @crickman, I've done more testing and that error was indeed related to the TPM limit on the model deployment side.