Skip to content

Commit ad9f89b

Browse files
committed
update token limit policy
1 parent 2ca2786 commit ad9f89b

File tree

3 files changed

+11
-2
lines changed

3 files changed

+11
-2
lines changed

articles/api-management/azure-openai-token-limit-policy.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,10 +72,11 @@ For more information, see [Azure OpenAI Service models](../ai-services/openai/co
7272

7373
* This policy can be used multiple times per policy definition.
7474
* This policy can optionally be configured when adding an API from the Azure OpenAI Service using the portal.
75+
* [!INCLUDE [api-management-rate-limit-key-scope](../../includes/api-management-rate-limit-key-scope.md)]
7576

7677
## Example
7778

78-
In the following example, the token limit of 5000 per minute is keyed by the caller IP address. The policy doesn't estimate the number of tokens required for a prompt. After each policy execution, the remaining tokens allowed in the time period are stored in the variable `remainingTokens`.
79+
In the following example, the token limit of 5000 per minute is keyed by the caller IP address. The policy doesn't estimate the number of tokens required for a prompt. After each policy execution, the remaining tokens allowed for that caller IP address in the time period are stored in the variable `remainingTokens`.
7980

8081
```xml
8182
<policies>

articles/api-management/rate-limit-by-key-policy.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ To understand the difference between rate limits and quotas, [see Rate limits an
6161

6262
### Usage notes
6363

64-
* API Management uses a single counter for each `counter-key` value. The counter is updated at all scopes at which the policy is configured with that key value. If you want to configure separate counters at different scopes (for example, a specific API or product), specify different key values at the different scopes. For example, append a string that identifies the scope to the key value.
64+
* [!INCLUDE [api-management-rate-limit-key-scope](../../includes/api-management-rate-limit-key-scope.md)]
6565
* [!INCLUDE [api-management-self-hosted-gateway-rate-limit](../../includes/api-management-self-hosted-gateway-rate-limit.md)] [Learn more](how-to-self-hosted-gateway-on-kubernetes-in-production.md#request-throttling)
6666

6767

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
author: dlepow
3+
ms.service: api-management
4+
ms.topic: include
5+
ms.date: 05/23/2024
6+
ms.author: danlep
7+
---
8+
API Management uses a single counter for each `counter-key` value. The counter is updated at all scopes at which the policy is configured with that key value. If you want to configure separate counters at different scopes (for example, a specific API or product), specify different key values at the different scopes. For example, append a string that identifies the scope to the value of an expression.

0 commit comments

Comments
 (0)