Upgrade to ort 2.0.0-rc.10?

[m0rt3nlund]

Hi!

I was wondering if there was any work going on to upgrade this library to use ort 2.0.0-rc.10?
I have a project where I am going to use the new RTX 5090 and this is on the Blackwell architecture and require CUDA Compute version 120.
To be able to use this it looks like I need to use a newer version of ONNXRuntime then the current 1.16 that is used in the ortex library at the moment..
Error while trying to run with CUDA as EP:
Non-zero status code returned while running QuickGelu node. Name:'/model.0/act/Mul/QuickGeluFusion/' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

I tried just using a newer version of ONNXRuntime library by replacing the *.so files but then I get the following error:
{:load_failed, ~c"Failed to load NIF library: 'libonnxruntime.so: version "VERS_1.19.0" not found (required by /opt/maskon/cameratest/_build/dev/lib/ortex/priv/native/libortex.so)'"}}
This indicated to me that the libraries are incompatible and I started to rewrite ortex to use the newest ort version as this uses a newer version of ONNXRuntime.

Only problem is that my rust programming capabilities are somewhat limited and I have now reached a point in the rewrite where I just dont understand where to go on anymore..

This is as far as I have gotten and it might not even be correct, but its my try!

https://github.com/m0rt3nlund/ortex/tree/upgrade

Anyone else who want to chime in and help get this working work be amazing!

Thank you!

[mortont]

We've been waiting on ort to get a full 2.0 release so that it's API stable before upgrading ortex, otherwise the work we put in to upgrade may be a waste (this has already happened once since it's been in 2.0-rc). There's a decent amount of work to upgrade last I checked.

We've also considered decoupling from ort and going directly to libonnxruntime, but also it's a bigger lift. It would alleviate this issue, and with ort being in RC for almost 2 years now it's left us many versions behind on onnxruntime (as you've noticed).

[m0rt3nlund]

Hi :)
Thanks for the reply!
I noticed ort write on their frontpage that the api should now be more or less stable and they promote usage.
Thats why i got motivated to try to rewrite it.
Made PR for you with my current work, which compiles and works now :)
I forgot to run the tests but i have it running in my own environment now in a stress-test on real data using CUDA as EP on a RTX 5090 card.