Support for Jetson Orin (NX or Nano)

[WilliamTambellini]

Hi nvidia @tongyuantongyu
1 Could you please confirm that TRTLLM is not supported (anymore? yet?) on Jetson Nano ?
2 Will the '0.12 jetson' branch be maintained?
Best
Ref:
https://github.com/NVIDIA/TensorRT-LLM/tree/v0.12.0-jetson
https://nvidia.github.io/TensorRT-LLM/reference/support-matrix.html

[tongyuantongyu]

Jetson architectures are not supported in the main branch.

@karljang, could you answer, or route this question to someone who knows about Jetson support better?

[karljang]

Hi @WilliamTambellini,
The TRTLLM team is currently concentrating on data center use cases, and edge device support is not included in our roadmap. I’ve had discussions with the leads about this.
cc. @laikhtewari.

[karljang]

Additionally, there’s another effort focused on enabling LLMs on edge devices, and we hope to make it public soon.