Support for Execution Providers - QNN

[adhurwit]

In order to make use of Qualcomm's NPU, we need to use the ONNX EP for QNN. There does not appear to be a way to specify that right now. Please add support for Qualcomm NPUs

[natke]

Hi @adhurwit, we are working on this! Stay tuned.

[adhurwit]

how is this coming along? is there a target date?

[natke]

Moving into Discussion

[Davidqian123]

Any update for QNN Execution Providers? Appreciate a lot!

[axcpy08]

Hi there. Do you have ETA for QNN EP? Thanks!

[hmartinez82]

Is there anything I can do to help? I can't even find instructions on how to build genai with QNN support enabled. I already have onnxruntime built locally for QNN.

[scsonic]

there were a branch named phi3.5_qnn_test / android
it's possible to share model builder with qnn?

[jasonw247]

Is there any update on this? Are there plans to support context binaries/split models for HTP to get around HTP rpc memory size limitations?

[baijumeswani]

Please take a look at these resources:

• https://onnxruntime.ai/docs/genai/tutorials/snapdragon.html
• https://onnxruntime.ai/docs/genai/howto/build-models-for-snapdragon.html

[axcpy08]

Thanks for providing the resources. I followed the instruction and faced the RuntimeError as below picture. Any suggestions for solving it?

[jasonw247]

Thanks! Are LoRa adapters supported with the QNN execution provider?

I see this issue which notes that we only verified the example with CPU and CUDA EP. Is this the current state?

I also see: https://onnxruntime.ai/docs/execution-providers/QNN-ExecutionProvider.html#lorav2-support. Is there more documentation around this? Does this support adapter loading/unloading/switching?

Is that compatible with: https://onnxruntime.ai/blogs/multilora?

[baijumeswani]

I see microsoft/Olive#1544 (comment) issue which notes that we only verified the example with CPU and CUDA EP. Is this the current state?

This is still accurate. We have not done validation with DML or QNN execution providers yet. As such CPU and Cuda are the primary execution providers where lora support can be expected to work without hiccups.