-
Notifications
You must be signed in to change notification settings - Fork 67
Open
Labels
internalfiled by core contributor or associatefiled by core contributor or associate
Milestone
Description
Describe the bug
Guidellm adds the max_completion_tokens
field to requests. As a result we see this log on vllm:
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
Expected behavior
- it should not add this feild
Environment
Include all relevant environment information:
- OS [e.g. Ubuntu 20.04]:
- Python version [e.g. 3.12.2]:
To Reproduce
Exact steps to reproduce the behavior:
Errors
If applicable, add a full print-out of any errors or exceptions that are raised or include screenshots to help explain your problem.
Additional context
Add any other context about the problem here. Also include any relevant files.
Metadata
Metadata
Assignees
Labels
internalfiled by core contributor or associatefiled by core contributor or associate