Is it possible to run inference on Adreno GPU using ONNX Runtime?

[cbswj]

Hi ONNX Runtime team,

I have devices with Snapdragon 8 Gen 2 and Gen 3 SoCs, which both feature Adreno GPUs.

I would like to ask whether ONNX Runtime supports Adreno GPU acceleration for model inference across different operating systems — specifically Android, Windows, and Linux.

My questions are:

1. Does ONNX Runtime support running inference on Adreno GPU directly (not through CPU or DSP)?
2. Is Adreno GPU acceleration supported across all major platforms (Android / Windows / Linux)?
3. If supported, do I need to use the QNN Execution Provider or any specific backend for Adreno GPU?
4. Are there any sample projects, build guides, or documentation for enabling GPU inference on Adreno?

Any insights or official stance would be greatly appreciated.

Thank you in advance!

Best regards,

[quic-ashigarg]

GPU support through ORT QNN-EP was enabled in preview mode at Build. This is a very initial enablement, and we are expecting to get more workloads to be enabled through GPU in the coming months. Once it is enabled, it should work across Android/Windows.

[johnpaultaken]

To add some more information to Ashish's response,

1. QNN EP backend now has a choice of using the GPU backend. As Ashish mentioned it is in its initial state of enablement. more functionality coming with the upcoming QNN SDK 2.38 release. This is the recommended approach. Another approach is to use the DML EP, which uses the GPU as well, with greater functionality currently, but is going to be phased out soon.
2. QNN EP with GPU backend is currently supported on Windows. Support for Android will be enabled next. Linux support is yet to be determined.
3. Yes QNN EP with backend specified as QnnGpu.
4. Enabling GPU requires only changing the backend name in the QNN EP config. Everything else can remain the same. There is some information here https://www.qualcomm.com/developer/blog/2025/05/unlocking-power-of-qualcomm-qnn-execution-provider-gpu-backend-onnx-runtime
   Feel free to contact me if you need more information.