Skip to content

Commit 41ce31b

Browse files
committed
docs fix
1 parent fce28a4 commit 41ce31b

File tree

1 file changed

+8
-4
lines changed

1 file changed

+8
-4
lines changed

docs/my-website/docs/proxy/dynamic_rate_limit.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11

22
# Dynamic TPM/RPM Allocation
33

4-
Prevent projects from gobbling too much tpm/rpm. You should use this feature when you want to reserve tpm/rpm capacity for specific projects. For example, a realtime use case should get higher priority than a different use case.
4+
Prevent projects from gobbling too much tpm/rpm.
55

66
Dynamically allocate TPM/RPM quota to api keys, based on active keys in that minute. [**See Code**](https://github.com/BerriAI/litellm/blob/9bffa9a48e610cc6886fc2dce5c1815aeae2ad46/litellm/proxy/hooks/dynamic_rate_limiter.py#L125)
77

8+
## Quick Start Usage
9+
810
1. Setup config.yaml
911

1012
```yaml showLineNumbers title="config.yaml"
@@ -97,15 +99,17 @@ This was rate limited b/c - Error code: 429 - {'error': {'message': {'error': 'K
9799
```
98100

99101

100-
#### [BETA] Set Priority / Reserve Quota
102+
## [BETA] Set Priority / Reserve Quota
103+
104+
Reserve tpm/rpm capacity for projects in prod. You should use this feature when you want to reserve tpm/rpm capacity for specific projects. For example, a realtime use case should get higher priority than a different use case.
101105

102-
Reserve tpm/rpm capacity for projects in prod.
103106

104107
:::tip
105108

106109
Reserving tpm/rpm on keys based on priority is a premium feature. Please [get an enterprise license](./enterprise.md) for it.
107110
:::
108111

112+
### Usage
109113

110114
1. Setup config.yaml
111115

@@ -118,7 +122,7 @@ model_list:
118122
rpm: 100
119123

120124
litellm_settings:
121-
callbacks: [""]
125+
callbacks: ["dynamic_rate_limiter_v3"]
122126
priority_reservation: {"dev": 0, "prod": 1}
123127

124128
general_settings:

0 commit comments

Comments
 (0)