Skip to content

test for openrouter/openai/gpt-4.1-mini does not work #370

@psydok

Description

@psydok

Describe the bug
I'm trying to test the openai model in openrouter. but an error occurs when checking the availability of the model due to specifying max_tokens:1

Expected behavior
The check will be successful.

Environment
Include all relevant environment information:

  1. OS [e.g. Ubuntu 20.04]:
  2. Python version [e.g. 3.12.2]:

To Reproduce
Exact steps to reproduce the behavior:
try sending a request with max_tokens:1, which is used by guidellm to verify the correct input, to openai/gpt-4.1-mini

Errors
Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 1 instead.

traceback

Creating backend...
25-09-25 17:51:32|ERROR            |guidellm.backend.openai:chat_completions:385 - OpenAIHTTPBackend request with headers: {'Content-Type': 'application/json', 'Authorization': 'Bearer '} and params: {} and payload: {'temperature': 0, 'messages': [{'role': 'user', 'content': 'Test connection'}], 'model': 'openai/gpt-4.1-mini', 'stream': True, 'stream_options': {'include_usage': True}, 'max_tokens': 1, 'stop': None} failed: Client error '400 Bad Request' for url 
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

Traceback (most recent call last):
  File "/usr/local/bin/guidellm", line 7, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/guidellm/__main__.py", line 314, in run
    asyncio.run(
  File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.10/site-packages/guidellm/benchmark/entrypoints.py", line 29, in benchmark_with_scenario
    return await benchmark_generative_text(**vars(scenario), **kwargs)
  File "/usr/local/lib/python3.10/site-packages/guidellm/benchmark/entrypoints.py", line 71, in benchmark_generative_text
    await backend.validate()
  File "/usr/local/lib/python3.10/site-packages/guidellm/backend/backend.py", line 138, in validate
    async for _ in self.chat_completions(  # type: ignore[attr-defined]
  File "/usr/local/lib/python3.10/site-packages/guidellm/backend/openai.py", line 393, in chat_completions
    raise ex
  File "/usr/local/lib/python3.10/site-packages/guidellm/backend/openai.py", line 374, in chat_completions
    async for resp in self._iterative_completions_request(
  File "/usr/local/lib/python3.10/site-packages/guidellm/backend/openai.py", line 605, in _iterative_completions_request
    stream.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 829, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request' 

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions