Skip to content

Commit c802c47

Browse files
docs(debugging.md): document new feature
Closes #13814
1 parent 48619f2 commit c802c47

File tree

1 file changed

+53
-15
lines changed

1 file changed

+53
-15
lines changed

docs/my-website/docs/proxy/debugging.md

Lines changed: 53 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -11,65 +11,103 @@ The proxy also supports json logs. [See here](#json-logs)
1111

1212
**via cli**
1313

14-
```bash
14+
```bash showLineNumbers
1515
$ litellm --debug
1616
```
1717

1818
**via env**
1919

20-
```python
20+
```python showLineNumbers
2121
os.environ["LITELLM_LOG"] = "INFO"
2222
```
2323

2424
## `detailed debug`
2525

2626
**via cli**
2727

28-
```bash
28+
```bash showLineNumbers
2929
$ litellm --detailed_debug
3030
```
3131

3232
**via env**
3333

34-
```python
34+
```python showLineNumbers
3535
os.environ["LITELLM_LOG"] = "DEBUG"
3636
```
3737

3838
### Debug Logs
3939

4040
Run the proxy with `--detailed_debug` to view detailed debug logs
41-
```shell
41+
```shell showLineNumbers
4242
litellm --config /path/to/config.yaml --detailed_debug
4343
```
4444

4545
When making requests you should see the POST request sent by LiteLLM to the LLM on the Terminal output
46-
```shell
46+
```shell showLineNumbers
4747
POST Request Sent from LiteLLM:
4848
curl -X POST \
4949
https://api.openai.com/v1/chat/completions \
5050
-H 'content-type: application/json' -H 'Authorization: Bearer sk-qnWGUIW9****************************************' \
5151
-d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "this is a test request, write a short poem"}]}'
5252
```
5353

54+
## Debug single request
55+
56+
Pass in `litellm_request_debug=True` in the request body
57+
58+
```bash showLineNumbers
59+
curl -L -X POST 'http://0.0.0.0:4000/chat/completions' \
60+
-H 'Content-Type: application/json' \
61+
-H 'Authorization: Bearer sk-1234' \
62+
-d '{
63+
"model":"fake-openai-endpoint",
64+
"messages": [{"role": "user","content": "How many r in the word strawberry?"}],
65+
"litellm_request_debug": true
66+
}'
67+
```
68+
69+
This will emit the raw request sent by LiteLLM to the API Provider and raw response received from the API Provider for **just** this request in the logs.
70+
71+
72+
```bash showLineNumbers
73+
INFO: Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)
74+
20:14:06 - LiteLLM:WARNING: litellm_logging.py:938 -
75+
76+
POST Request Sent from LiteLLM:
77+
curl -X POST \
78+
https://exampleopenaiendpoint-production.up.railway.app/chat/completions \
79+
-H 'Authorization: Be****ey' -H 'Content-Type: application/json' \
80+
-d '{'model': 'fake', 'messages': [{'role': 'user', 'content': 'How many r in the word strawberry?'}], 'stream': False}'
81+
82+
83+
20:14:06 - LiteLLM:WARNING: litellm_logging.py:1015 - RAW RESPONSE:
84+
{"id":"chatcmpl-817fc08f0d6c451485d571dab39b26a1","object":"chat.completion","created":1677652288,"model":"gpt-3.5-turbo-0301","system_fingerprint":"fp_44709d6fcb","choices":[{"index":0,"message":{"role":"assistant","content":"\n\nHello there, how may I assist you today?"},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":9,"completion_tokens":12,"total_tokens":21}}
85+
86+
87+
INFO: 127.0.0.1:56155 - "POST /chat/completions HTTP/1.1" 200 OK
88+
89+
```
90+
91+
5492
## JSON LOGS
5593

5694
Set `JSON_LOGS="True"` in your env:
5795

58-
```bash
96+
```bash showLineNumbers
5997
export JSON_LOGS="True"
6098
```
6199
**OR**
62100

63101
Set `json_logs: true` in your yaml:
64102

65-
```yaml
103+
```yaml showLineNumbers
66104
litellm_settings:
67105
json_logs: true
68106
```
69107
70108
Start proxy
71109
72-
```bash
110+
```bash showLineNumbers
73111
$ litellm
74112
```
75113

@@ -80,7 +118,7 @@ The proxy will now all logs in json format.
80118
Turn off fastapi's default 'INFO' logs
81119

82120
1. Turn on 'json logs'
83-
```yaml
121+
```yaml showLineNumbers
84122
litellm_settings:
85123
json_logs: true
86124
```
@@ -89,20 +127,20 @@ litellm_settings:
89127

90128
Only get logs if an error occurs.
91129

92-
```bash
130+
```bash showLineNumbers
93131
LITELLM_LOG="ERROR"
94132
```
95133

96134
3. Start proxy
97135

98136

99-
```bash
137+
```bash showLineNumbers
100138
$ litellm
101139
```
102140

103141
Expected Output:
104142

105-
```bash
143+
```bash showLineNumbers
106144
# no info statements
107145
```
108146

@@ -119,14 +157,14 @@ This can be caused due to all your models hitting rate limit errors, causing the
119157
How to control this?
120158
- Adjust the cooldown time
121159
122-
```yaml
160+
```yaml showLineNumbers
123161
router_settings:
124162
cooldown_time: 0 # 👈 KEY CHANGE
125163
```
126164

127165
- Disable Cooldowns [NOT RECOMMENDED]
128166

129-
```yaml
167+
```yaml showLineNumbers
130168
router_settings:
131169
disable_cooldowns: True
132170
```

0 commit comments

Comments
 (0)