Skip to content

Commit 76c43f3

Browse files
toslali-ibmmarkurtz
authored andcommitted
Add vllm id to the response
Signed-off-by: Mert Toslali <[email protected]>
1 parent f6175cd commit 76c43f3

File tree

2 files changed

+5
-0
lines changed

2 files changed

+5
-0
lines changed

src/guidellm/backends/response_handlers.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,7 @@ def compile_non_streaming(
145145
request_args=str(
146146
request.arguments.model_dump() if request.arguments else None
147147
),
148+
response_id=response.get("id"), # use vLLM ID if available
148149
text=text,
149150
input_metrics=input_metrics,
150151
output_metrics=output_metrics,

src/guidellm/schemas/response.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,10 @@ class GenerationResponse(StandardBaseModel):
4141
request_id: str = Field(
4242
description="Unique identifier matching the original GenerationRequest."
4343
)
44+
response_id: str | None = Field(
45+
default=None,
46+
description="Unique identifier matching the original vLLM Response ID."
47+
)
4448
request_args: str | None = Field(
4549
description="Arguments passed to the backend for request processing."
4650
)

0 commit comments

Comments
 (0)