Setting adaptive SamplingParams is not allowed when the model is loaded from the torch backend.

[ccys-a11y]

System Info

tensorrt-llm 0.20.0

Who can help?

@bobboli
When I run quickstart_advanced.py for qwen2&3, only modifing the sampling_params as following:
sampling_params = SamplingParams(
temperature=1.0,
top_p=0.7,
max_tokens=args.max_tokens,
n = 32,
)
Errors come like following figure. And setting "enable_trtllm_sampler=True " does not work.

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

1. change the sampling_params in quickstart_advanced.py to: sampling_params = SamplingParams(
   temperature=1.0,
   top_p=0.7,
   max_tokens=args.max_tokens,
   n = 32,
   )
2. python3 quickstart_advanced.py --model_dir data_path_to_Qwen3-14B

Expected behavior

Setting adaptive SamplingParams works when the model is loaded from the torch backend.

actual behavior

additional notes

none

[bobboli]

There is an ongoing PR #5415 by @jaedeok-nvidia

[karljang]

@ccys-a11y , apologies for the very delayed response.
Could you please try the latest version to see if the issue still remains?

[jaedeok-nvidia]

Hi @karljang @ccys-a11y, apologies for missing this issue. As I mentioned in #5415 to @ccys-a11y, #5997 should resolve it.

[karljang]

@ccys-a11y ,
It seems the issue has been resolved in #5997, could you try the latest version to see if the issue persists?
I'm closing this issue, please feel free to open another one if you find any issues~