High Latency in TensorRT Node for Image Segmentation on Jetson Orin Nano 8GB

[eterry-devops]

Issue Description
I'm experiencing unexpectedly high latency when running image segmentation using Isaac ROS DNN Inference on a Jetson Orin Nano 8GB. The TensorRT node appears to be the primary bottleneck, with processing delays averaging ~240-260ms.
Environment

Hardware: Jetson Orin Nano 8GB
Model: PeopleSegNet (deployable_quantized_vanilla_unet_onnx_v2.0)
Isaac ROS Version: 3.2
JetPack Version: 6.2
CUDA Version: 12.6

Command Used
bashros2 launch isaac_ros_examples isaac_ros_examples.launch.py
launch_fragments:=zed_mono_rect,unet
engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/peoplesemsegnet/deployable_quantized_vanilla_unet_onnx_v2.0/1/model.plan
input_binding_names:=['input_1:0']
output_binding_names:=['argmax_1']
network_output_type:='argmax'
interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_unet/zed2_quickstart_interface_specs.json
Performance Measurements
TensorRT Node Input (/tensor_sub)
ros2 topic delay /tensor_sub --window 100

Average delay: ~240-260ms
Min: 108ms
Max: 348ms
Std dev: ~0.04s

TensorRT Node Output (/tensor_pub)
ros2 topic delay /tensor_pub --window 100

Average delay: ~200-214ms
Min: 123ms
Max: 287ms
Std dev: ~0.03s

Analysis
The latency measurements show that:

Total pipeline latency is averaging 240-260ms
The TensorRT node itself appears to be adding significant processing time
There's considerable variance in processing times (std dev ~40ms on input)

Expected Behavior
For a Jetson Orin Nano 8GB with a quantized UNet model, I would expect much lower latency, ideally in the range of 140-160ms for real-time performance from image-to-mask