build error

[geraldstanje]

hi,

i run this command:
git clone https://github.com/triton-inference-server/server -b r25.09
cd server && ./build.py --backend tensorrtllm:release/1.0 --backend python:r25.09 --enable-gpu --build-type Release --target-platform linux --endpoint grpc --endpoint http

and get error:
-- Found CUDA: /usr/local/cuda (found version "13.0")
-- Found Python3: /usr/bin/python3.12 (found version "3.12.3") found components: Interpreter Development Development.Module Development.Embed
CMake Error at CMakeLists.txt:151 (find_library):
Could not find tensorrt_llm using the following names: libtensorrt_llm.so

-- Configuring incomplete, errors occurred!
error: build failed

how to make it work? if i need to build tensorrtllm - what to run? the tensorrt backend here is for triton 25.06 - and docker build: https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/dockerfile/Dockerfile.triton.trt_llm_backend

cc @rmccorm4 @statiraju @mc-nv @yinggeh @krishung5 @the-david-oy

[geraldstanje1]

hi @whoisj @krishung5 any idea who to loop in for ^^ ?

[whoisj]

I am attempting to reproduce this issue. I'll let you know my results.

[geraldstanje1]

@whoisj thank you! if successful please let me know all commands your run to get it work - for manually building image nvcr.io/nvidia/tritonserver:25.09-trtllm-python-py3

[whoisj]

I ended up with the same error, I am investigating now.

-- Found CUDA: /usr/local/cuda (found version "13.0")
-- Found Python3: /usr/bin/python3.12 (found version "3.12.3") found components: Interpreter Development Development.Module Development.Embed
CMake Error at CMakeLists.txt:151 (find_library):
  Could not find tensorrt_llm using the following names: libtensorrt_llm.so

[whoisj]

Until our build expert is available, the best suggestion I can give you is to do something like the following (in the tensortrtllm_backend folder):

export BASE_IMAGE=nvcr.io/nvidia/tritonserver:25.09-py3-min
export PYTORCH_IMAGE=nvcr.io/nvidia/pytorch:25.06-py3
export TRT_VERSION=1.0.0
export TENSORRTLLM_BACKEND_REPO_TAG=r25.09
export PYTHON_BACKEND_REPO_TAG=r25.09

wget https://raw.githubusercontent.com/triton-inference-server/tensorrtllm_backend/refs/heads/r25.07/build.sh
chmod +x build.sh

./build.sh

[geraldstanje1]

@whoisj but that uses r25.07 - why not use https://raw.githubusercontent.com/triton-inference-server/tensorrtllm_backend/refs/heads/r25.09/build.sh ?

but running that script also needs updated TRT_URL? i see 10.7.0 in that url...
TRT_URL_x86=${TRT_URL_x86:-https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.7.0/tars/TensorRT-${TRT_VERSION}.Linux.x86_64-gnu.cuda-12.6.tar.gz}
TRT_URL_ARM=${TRT_URL_ARM:-https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.7.0/tars/TensorRT-${TRT_VERSION}.ubuntu-24.04.aarch64-gnu.cuda-12.6.tar.gz}

and what about TRITON_SERVER_REPO_TAG?

it looks like this file just got deleted by @achartier: triton-inference-server/tensorrtllm_backend#810 (comment) - how to do what @whoisj suggested...

@whoisj do you know who we can loop in?

[geraldstanje1]

hi @whoisj any update on this?

[whoisj]

@mc-nv is this something you can assist with?

[linear]

from mchornyi:

I will review it and get back later this week.