remove spurious cpu->gpu and gpu->cpu transfers

[suyoggupta]

@coderabbitai summary

[Copilot]

Pull Request Overview

This PR removes spurious CPU-to-GPU and GPU-to-CPU tensor transfers to improve performance. The changes implement optimizations by allocating tensors on the GPU device from the start, using pinned memory for efficient transfers, and avoiding unnecessary device copies.

• Pre-allocates GPU tensors in SequenceInfo to avoid repeated tensor creation overhead
• Replaces CPU-GPU round trips with direct GPU operations and pinned memory copies
• Updates executor interfaces to pass tensors instead of converting to lists

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tensorrt_llm/_torch/pyexecutor/py_executor.py	Minor logging update for pipeline executor tracing
tensorrt_llm/_torch/pyexecutor/config.py	Changes default attention backend from TRTLLM to FLASHINFER
tensorrt_llm/_torch/auto_deploy/transform/library/export_to_gm.py	Comment clarification for example sequence setting
tensorrt_llm/_torch/auto_deploy/shim/ad_executor.py	Major refactoring to avoid GPU-CPU transfers and improve tensor handling
tensorrt_llm/_torch/auto_deploy/custom_ops/attention_interface.py	Extensive optimization of SequenceInfo class with GPU tensor pre-allocation and pinned memory
tensorrt_llm/_torch/auto_deploy/compile/backends/torch_cudagraph.py	Uses non-blocking copy for input buffer transfers

[Copilot]

Debug print statement should be removed or replaced with proper logging for production code.

Suggested change

	print(" in seq_info for device: ", torch.cuda.current_device())
	ad_logger.info(f"In seq_info for device: {torch.cuda.current_device()}")

[Copilot]

Debug print statement should be removed or replaced with proper logging for production code.

[Copilot]

This operation moves seq_len to device before adding to host tensor, which defeats the purpose of keeping calculations on host. Consider converting seq_len to CPU first: self.input_pos_host[:bs] += seq_len.cpu()

Suggested change

	self.input_pos_host[:bs] += seq_len.to(self.device)
	self.input_pos_host[:bs] += seq_len.cpu()

[suyoggupta]

maybe it's better to not call update_pos here at all and introduce a different method that does what update_pos (update_position_ids=False) does? As-is, it is bit confusing to call update_pos without updating positions.
@galagam

[galagam]

Updating the position ids requires both the input positions and the sequence lengths, so it makes sense to update it whenever either is updated, but it's a bit wasteful.
A possible alternative would be to require the user to call it explicitly.
That is

si.update_input_pos()  #  rename update_pos
si.nest_sequences()
si.update_position_ids()

In any case, due to my recent changes, run time of update_position_ids decreased by x30, so it's not as critical to add this specific optimization as I initially believed. I'll run a more exhaustive check and consider to keep this optimization out of this PR for code simplicity.
@suyoggupta