Skip to content

Commit 0c0c201

Browse files
Update openai_compatible_server.md (#11536)
Co-authored-by: Simon Mo <[email protected]>
1 parent 82d24f7 commit 0c0c201

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

docs/source/serving/openai_compatible_server.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,13 @@ completion = client.chat.completions.create(
112112

113113
## Extra HTTP Headers
114114

115-
Only `X-Request-Id` HTTP request header is supported for now.
115+
Only `X-Request-Id` HTTP request header is supported for now. It can be enabled
116+
with `--enable-request-id-headers`.
117+
118+
> Note that enablement of the headers can impact performance significantly at high QPS
119+
> rates. We recommend implementing HTTP headers at the router level (e.g. via Istio),
120+
> rather than within the vLLM layer for this reason.
121+
> See https://github.com/vllm-project/vllm/pull/11529 for more details.
116122
117123
```python
118124
completion = client.chat.completions.create(

0 commit comments

Comments
 (0)