Is it possible to run onnx runtime with cuda/tensorRT EP on GPU tensors?

[bpodrygajlo]

Hi, haven't found a good sample for this: my data is already in GPU memory and onnxrutime seems to assume it has to copy data from host to GPU for CUDA/TensorRT EPs. Is there a way to configure onnxruntime/provider for this not to happen?

Currently I am using MemoryInfo::CreateCpu to create the tensors.

[Nietaa]

Can't help with 1st hand knowledge, as I'm using DML EP, but...
https://fs-eire.github.io/onnxruntime/docs/performance/tune-performance.html#iobinding
https://onnxruntime.ai/docs/api/python/api_summary.html#data-on-device

Also did you check [the sample test (C# API)]?(https://github.com/microsoft/onnxruntime/tree/main/csharp/test/Microsoft.ML.OnnxRuntime.Tests.Common)