-
Notifications
You must be signed in to change notification settings - Fork 86
Open
Labels
Description
Describe the bug
I'm trying to test the openai model in openrouter. but an error occurs when checking the availability of the model due to specifying max_tokens:1
Expected behavior
The check will be successful.
Environment
Include all relevant environment information:
- OS [e.g. Ubuntu 20.04]:
- Python version [e.g. 3.12.2]:
To Reproduce
Exact steps to reproduce the behavior:
try sending a request with max_tokens:1, which is used by guidellm to verify the correct input, to openai/gpt-4.1-mini
Errors
Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 1 instead.
traceback
Creating backend...
25-09-25 17:51:32|ERROR |guidellm.backend.openai:chat_completions:385 - OpenAIHTTPBackend request with headers: {'Content-Type': 'application/json', 'Authorization': 'Bearer '} and params: {} and payload: {'temperature': 0, 'messages': [{'role': 'user', 'content': 'Test connection'}], 'model': 'openai/gpt-4.1-mini', 'stream': True, 'stream_options': {'include_usage': True}, 'max_tokens': 1, 'stop': None} failed: Client error '400 Bad Request' for url
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400
Traceback (most recent call last):
File "/usr/local/bin/guidellm", line 7, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1161, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1082, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1697, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1697, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1443, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 788, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/guidellm/__main__.py", line 314, in run
asyncio.run(
File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/local/lib/python3.10/site-packages/guidellm/benchmark/entrypoints.py", line 29, in benchmark_with_scenario
return await benchmark_generative_text(**vars(scenario), **kwargs)
File "/usr/local/lib/python3.10/site-packages/guidellm/benchmark/entrypoints.py", line 71, in benchmark_generative_text
await backend.validate()
File "/usr/local/lib/python3.10/site-packages/guidellm/backend/backend.py", line 138, in validate
async for _ in self.chat_completions( # type: ignore[attr-defined]
File "/usr/local/lib/python3.10/site-packages/guidellm/backend/openai.py", line 393, in chat_completions
raise ex
File "/usr/local/lib/python3.10/site-packages/guidellm/backend/openai.py", line 374, in chat_completions
async for resp in self._iterative_completions_request(
File "/usr/local/lib/python3.10/site-packages/guidellm/backend/openai.py", line 605, in _iterative_completions_request
stream.raise_for_status()
File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 829, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request'
Additional context