[Bugfix] Fix AttributeError when passing StructuredOutputsParams to CompletionRequest#35234
[Bugfix] Fix AttributeError when passing StructuredOutputsParams to CompletionRequest#35234pks wants to merge 1 commit intovllm-project:mainfrom
Conversation
…ompletionRequest
The `check_structured_outputs_count` model validator uses `mode="before"`,
so it receives raw input data. It calls `.get()` on the `structured_outputs`
value, which only works for dicts. However the field is typed as
`StructuredOutputsParams | None`, so passing a dataclass instance is valid
but crashes with:
AttributeError: 'StructuredOutputsParams' object has no attribute 'get'
Handle both dict and StructuredOutputsParams inputs by checking the type
and using `getattr` for dataclass instances.
Fixes both `CompletionRequest` and `ChatCompletionRequest`.
Signed-off-by: Patrick Simianer <patrick@lilt.com>
There was a problem hiding this comment.
Code Review
This pull request addresses an AttributeError in check_structured_outputs_count for CompletionRequest and ChatCompletionRequest. The error occurred when structured_outputs was a StructuredOutputsParams object instead of a dictionary. The submitted change corrects this by checking the type of structured_outputs and using either getattr for objects or .get() for dictionaries to access its properties. The fix is applied consistently in both chat_completion/protocol.py and completion/protocol.py. The changes correctly resolve the described bug.
|
Hi @pks, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
Summary
The
check_structured_outputs_countmodel validator onCompletionRequestandChatCompletionRequestusesmode="before"and calls.get()on thestructured_outputsvalue, assuming it is always a dict. However, the field is typed asStructuredOutputsParams | None, so passing a dataclass instance is a valid usage but crashes with:This PR handles both dict and
StructuredOutputsParamsinputs by checking the type and usinggetattrfor dataclass instances.Reproduction
Test plan
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.