ITensor::getDimensions: Error Code 4: API Usage Error failure of TensorRT 10.3 when running model conversion on GPU AGX Orin

[aw632]

Description

I converted a model (example: https://huggingface.co/onnx-community/dinov3-vits16-pretrain-lvd1689m-ONNX/tree/main) to ONNX then built with TensorRT on my RTX 4090 with TRT 10.13. This worked fine and I was able to successfully build the engine.

When trying to convert the exact same ONNX file on AGX Orin, I got this error:

[6] Invalid Node - /model/vc/backbone/rope_embed/If
ITensor::getDimensions: Error Code 4: API Usage Error (/model/vc/backbone/rope_embed/If_OutputLayer: IIfConditionalOutputLayer inputs must have the same shape. Shapes are [2] and [1].)

Although the TRT versions are different, I understand that in 10.13, the same restriction (outputs must be the same shape) applies, and the model was built successfully on the RTX 4090. AGX Orins have their TensorRT versions fixed, so I cannot upgrade.

Environment

TensorRT Version: 10.3

NVIDIA GPU: AGX Orin

NVIDIA Driver Version: N/A

CUDA Version: 12.6

CUDNN Version: N/A

Operating System: Jetpack 6

Python Version (if applicable): N/A

Tensorflow Version (if applicable): N/A

PyTorch Version (if applicable): N/A

Baremetal or Container (if so, version): N/A

Relevant Files

Model link: https://huggingface.co/onnx-community/dinov3-vits16-pretrain-lvd1689m-ONNX/tree/main

Steps To Reproduce

Commands or scripts: /usr/src/tensorrt/bin/trtexec --onnx=dinov3.onnx --saveEngine=dinov3.engine

Have you tried the latest release?: No

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): Yes

[haritsahm]

I hit the same issue. The root cause is angles = angles.tile(2) in rope_position_encoding.py#L102. Since torch.tile checks input dims, it triggers extra if ops in onnx/tensorrt. Replacing it with angles = angles.tile((1, 2)) fixes the problem.

[aw632]

Amazing find! Thanks

[Syd-Q]

Hello, I’m also deploying DINOv3 (ViT-S/16) on the Jetson AGX Orin platform. Following @haritsahm’s suggestion, I modified the RoPE implementation by changing: angles = angles.tile((1, 2)), and now I can convert dinov3 to onnx and successfully use trtexec to build trt. However, after conversion, I noticed a significant numerical discrepancy between the TensorRT engine and the original ONNX Runtime output. Have you encountered this situation? Below is the code I used to convert ONNX and build TRT

with torch.no_grad():
        torch.onnx.export(base_model, input_tensor, save_path, 
                    input_names=["input"],
                    output_names=["output"],
                    # dynamic_axes={"input": {0: "batch"}, "output": {0: "batch"}},
                    dynamic_axes=None,
                    opset_version=17,
                    verbose=True)


/usr/src/tensorrt/bin/trtexec --onnx=dinov3_vits16/dinov3_vits16.onnx --saveEngine=dinov3_vits16/dinov3_vits16_fp32.trt

and i used Polygraphy to compare outputs:
polygraphy run dinov3_vits16/dinov3_vits16_fp32.onnx --trt --onnxrt --validate
The result shows a large absolute error:
Absolute Difference | Stats: mean=0.41986, std-dev=0.35048, var=0.12284, median=0.35332, min=5.0589e-06 at (0, 365, 34, 3), max=4.8505 at (0, 112, 1, 12), avg-magnitude=0.41986, p90=0.83121, p95=0.99642, p99=1.4052

Has anyone encountered similar issues with DINOv3 on Jetson?

Any advice on ensuring numerical consistency during ONNX → TensorRT conversion would be greatly appreciated!