Skip to content

Commit ddbb116

Browse files
LinPolymikeiovine
authored andcommitted
[https://nvbugs/5564465][fix] Overwrite only if default_max_tokens is legal (#8538)
Signed-off-by: Pengyun Lin <[email protected]> Signed-off-by: Mike Iovine <[email protected]>
1 parent 584ed86 commit ddbb116

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

tensorrt_llm/executor/base_worker.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -428,7 +428,7 @@ def _deduce_max_tokens(request: GenerationRequest,
428428
# default_max_tokens is the biggest available value
429429
if max_tokens is None:
430430
return default_max_tokens
431-
elif max_tokens > default_max_tokens:
431+
elif max_tokens > default_max_tokens and default_max_tokens > 0:
432432
logger.warning(
433433
f"User-specified `max_tokens` ({max_tokens}) is greater than deduced "
434434
f"`default_max_tokens` ({default_max_tokens}), using default_max_tokens instead."

0 commit comments

Comments
 (0)