Guidellm fails with SGLang 0.4.6 due to unlimited output token generation

[git-jxj]

Describe the bug
Guidellm benchmarking fails when testing against SGLang 0.4.6 servers. While the same configuration works with SGLang 0.5.5.post3, requests sent to SGLang 0.4.6 result in extremely long generation sequences (hundreds to thousands of tokens), causing requests to hang and constraints like --max-requests to not work properly.

Expected behavior
guidellm can normally access sglang's OpenAI API interface

Environment
Include all relevant environment information:

1. OS [e.g. Ubuntu 22.04]:
2. Python version [e.g. 3.13.7]:

To Reproduce
Exact steps to reproduce the behavior:

• Guidellm version: latest (from source)
• Working SGLang version: 0.5.5.post3
• Non-working SGLang version: 0.4.6
• Model: Qwen3-32B
• Test command:

1 guidellm benchmark
2 --target "http://ip:port"
3 --processor "/root/qwen-model/tokenizer/qwen3-32b"
4 --rate-type "concurrent"
5 --rate 1
6 --max-requests 1
7 --data "prompt_tokens=32,output_tokens=32,samples=1"
When using guidellm to benchmark against SGLang 0.4.6, the following issues occur:

1. Unlimited token generation: Requests with output_tokens=64 generate hundreds or thousands of tokens instead of respecting the specified limit
2. Request state management failure: The generated requests don't complete properly in guidellm, causing --max-requests constraints to fail

Errors

Parameter mapping issue: Guidellm's output_tokens=64 parameter does not properly map to SGLang 0.4.6's max_tokens parameter

Additional context
Add any other context about the problem here. Also include any relevant files.

[sjmonson]

To me that sounds like something was broken in SGLang 0.4.6 and has been fixed in-between 0.4.6 and 0.5.5.post3. You can try switching to the /v1/completions endpoint with --request-type text_completions or omitting output_tokens to see if the server behaves better.

[git-jxj]

Switching to the /v1/completions endpoint is working normally.Configuring output_tokens as empty resulted in an error. It seems to be a required parameter.

[git-jxj]

guidellm benchmark
--target "http://127.0.0.1:30000"
--processor "/root/data/Qwen3-32B"
--rate-type "concurrent"
--rate 1
--max-requests 1
--request-type text_completions
--data "prompt_tokens=32,output_tokens=32,samples=1"
✔ OpenAIHTTPBackend backend validated with model /root/data/Qwen3-32B
{'target': 'http://127.0.0.1:30000', 'model': None, 'timeout': 60.0, 'http2': True, 'follow_redirects': True, 'verify': False, 'openai_paths': {'health': 'health', 'models':
'v1/models', 'text_completions': 'v1/completions', 'chat_completions': 'v1/chat/completions', 'audio_transcriptions': 'v1/audio/transcriptions', 'audio_translations':
'v1/audio/translations'}, 'validate_backend': {'method': 'GET', 'url': 'http://127.0.0.1:30000/health'}}
✔ Processor resolved
Using processor '/root/data/Qwen3-32B'
✔ Request loader initialized with inf unique requests
{'data': "['prompt_tokens=32,output_tokens=32,samples=1']", 'data_args': '[]', 'data_samples': -1, 'preprocessors': ['GenerativeColumnMapper',
'GenerativeTextCompletionsRequestFormatter'], 'collator': 'GenerativeRequestCollator', 'sampler': 'None', 'num_workers': 1, 'random_seed': 42}
✔ Resolved transient phase configurations
Warmup: percent=None value=None mode='prefer_duration'
Cooldown: percent=None value=None mode='prefer_duration'
Rampup (Throughput/Concurrent): 0.0
✔ ConcurrentProfile profile resolved
{'str': "type_='concurrent' completed_strategies=[] constraints={'max_requests': 1} rampup_duration=0.0 streams=[1] strategy_types=['concurrent']", 'type': 'ConcurrentProfile', 'class':
'ConcurrentProfile', 'module': 'guidellm.benchmark.profiles', 'attributes': {'type_': 'concurrent', 'completed_strategies': [], 'constraints': {'max_requests': 1}, 'rampup_duration':
0.0, 'streams': [1]}}
✔ Output formats resolved
{'json': "output_path=PosixPath('/root/data/jxj/guidellm')", 'csv': "output_path=PosixPath('/root/data/jxj/guidellm')", 'html': "output_path=PosixPath('/root/data/jxj/guidellm')"}
✔ Setup complete, starting benchmarks...

╭─ Benchmarks ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [08:15:35] 100% concurrent@1 (complete) Req: 1.3 req/s, 0.79s Lat, 1.0 Conc, 1 Comp, 0 Inc, 0 Err │
│ Tok: 43.9 gen/s, 87.9 tot/s, 66.2ms TTFT, 23.5ms ITL, 32 Prompt, 32 Gen
------------------------------------model log
[2025-11-24 08:15:35] INFO: 127.0.0.1:55108 - "POST /v1/completions HTTP/1.1" 200 OK
[2025-11-24 08:15:35 TP0] Prefill batch. #new-seq: 1, #new-token: 32, #cached-token: 0, token usage: 0.00, #running-req: 0, #queue-req: 0
[2025-11-24 08:15:35 TP0] Decode batch. #running-req: 1, #token: 40, token usage: 0.00, gen throughput (token/s): 0.01, #queue-req: 0