Skip to content

Guidellm adds unexpected field to requests #269

@robertgshaw2-redhat

Description

@robertgshaw2-redhat

Describe the bug
Guidellm adds the max_completion_tokens field to requests. As a result we see this log on vllm:

WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}
WARNING 08-09 20:50:05 [protocol.py:58] The following fields were present in the request but ignored: {'max_completion_tokens'}

Expected behavior

  • it should not add this feild

Environment
Include all relevant environment information:

  1. OS [e.g. Ubuntu 20.04]:
  2. Python version [e.g. 3.12.2]:

To Reproduce
Exact steps to reproduce the behavior:

Errors
If applicable, add a full print-out of any errors or exceptions that are raised or include screenshots to help explain your problem.

Additional context
Add any other context about the problem here. Also include any relevant files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    internalfiled by core contributor or associate

    Type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions