Before submitting
Environment
Bug description
#3779 introduces checks for inactive rows in trtllm ragged prefill but causes synchronization inside the kernel API. alternatively we can supply CPU tensors q_seq_lens_cpu and kv_seq_lens_cpu to avoid device sync but that introduces non-trivial host overhead.
A potentially fix is to handle the edge case outside the kernel and once per forward pass instead of once every kernel call.
Minimal reproduction
Error messages / logs
Additional context
No response
Before submitting
Environment
Bug description
#3779 introduces checks for inactive rows in trtllm ragged prefill but causes synchronization inside the kernel API. alternatively we can supply CPU tensors
q_seq_lens_cpuandkv_seq_lens_cputo avoid device sync but that introduces non-trivial host overhead.A potentially fix is to handle the edge case outside the kernel and once per forward pass instead of once every kernel call.
Minimal reproduction
Error messages / logs
Additional context
No response