[TRTLLM-9735][feat] Add processed logprobs functionality to TorchSampler#9675

Open

stnie wants to merge 18 commits intoNVIDIA:mainfrom

stnie:develop/sampler/refactor_logprobs

Commits on Jan 5, 2026

[None][feat] Add processed logprobs functionality to TorchSampler

stnie
authored and
tongyuantongyu
committed
sampled logprob

hchings
authored and
tongyuantongyu
committed
update

hchings
authored and
tongyuantongyu
committed
[TRTLLM-9686][feat] Fix issues with processed logprobs functionality.

stnie
authored and
tongyuantongyu
committed
[TRTLLM-9687][fix] Enable pinned memory for tensor allocations in TorchSampler

stnie
authored and
tongyuantongyu
committed
[TRTLLM-9688][feat] Refactor processed logprobs to skip logprob calculation when not needed

stnie
authored and
tongyuantongyu
committed
[TRTLLM-9689][feat] Introduce max_topk_logprobs parameter and enhance logprobs handling

stnie
authored and
tongyuantongyu
committed
cache sampling_strategy
tongyuantongyu
committed
fix apply_embedding_bias perf
tongyuantongyu
committed
fix sample_batched_by_strategy perf
tongyuantongyu
committed
[TRTLLM-9686][chore] Added skip to testcase and fixed _request_strategy when using beam search

stnie
authored and
tongyuantongyu
committed
[TRTLLM-9686][chore] Refactored speculation_needs_probs to use request indices instead of a boolean tensor

stnie
authored and
tongyuantongyu
committed
Refactor logprob assertions in test_processed_logprobs_e2e to use torch.testing.assert_close for improved readability and precision

stnie
authored and
tongyuantongyu
committed
Nits, Update to ruff for test file
tongyuantongyu
committed
Update logprobs_mode definition
tongyuantongyu
committed
Avoid unnecessary initialization for runtime buffers
tongyuantongyu
committed
Address reviews
tongyuantongyu
committed
Address reviews 2
tongyuantongyu
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[TRTLLM-9735][feat] Add processed logprobs functionality to TorchSampler#9675

[TRTLLM-9735][feat] Add processed logprobs functionality to TorchSampler#9675
stnie wants to merge 18 commits intoNVIDIA:mainfrom
stnie:develop/sampler/refactor_logprobs

Commits on Jan 5, 2026

[None][feat] Add processed logprobs functionality to TorchSampler

sampled logprob

update

[TRTLLM-9686][feat] Fix issues with processed logprobs functionality.

[TRTLLM-9687][fix] Enable pinned memory for tensor allocations in TorchSampler

[TRTLLM-9688][feat] Refactor processed logprobs to skip logprob calculation when not needed

[TRTLLM-9689][feat] Introduce max_topk_logprobs parameter and enhance logprobs handling

cache sampling_strategy

fix apply_embedding_bias perf

fix sample_batched_by_strategy perf

[TRTLLM-9686][chore] Added skip to testcase and fixed _request_strategy when using beam search

[TRTLLM-9686][chore] Refactored speculation_needs_probs to use request indices instead of a boolean tensor

Refactor logprob assertions in test_processed_logprobs_e2e to use torch.testing.assert_close for improved readability and precision

Nits, Update to ruff for test file

Update logprobs_mode definition

Avoid unnecessary initialization for runtime buffers

Address reviews

Address reviews 2

[TRTLLM-9735][feat] Add processed logprobs functionality to TorchSampler#9675stnie wants to merge 18 commits intoNVIDIA:mainNVIDIA/TensorRT-LLM:mainfrom stnie:develop/sampler/refactor_logprobsstnie/TensorRT-LLM:develop/sampler/refactor_logprobsCopy head branch name to clipboard

Commits

Commits on Jan 5, 2026

[TRTLLM-9735][feat] Add processed logprobs functionality to TorchSampler#9675
stnie wants to merge 18 commits intoNVIDIA:mainfrom
stnie:develop/sampler/refactor_logprobs