Replies: 1 comment
-
EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING is supported |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I would like to inquire about the possibility of key-based TPM & RPM control in a multi-instance environment.
After examining the code, I found that since it uses local cache by default, TPM & RPM are not shared across multiple instances, so control doesn't work as expected.
I confirmed that setting default_in_memory_ttl to 0 forces the use of TPM & RPM stored in Redis, which results in normal control as expected.
However, when setting cache=true, there's an issue where responses are also cached unintentionally.
I want to control only TPM & RPM without caching responses. Is this configuration possible?
Thank you:)
Beta Was this translation helpful? Give feedback.
All reactions