Qualcomm AI Engine Direct - Test QNN2.37 uplevel

[winskuo-quic]

Summary

[PLEASE REMOVE] See CONTRIBUTING.md's Pull Requests for ExecuTorch PR guidelines.

[PLEASE REMOVE] If this PR closes an issue, please add a Fixes #<issue-id> line.

[PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: " label. For a list of available release notes labels, check out CONTRIBUTING.md's Pull Requests.

Test plan

[PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13352

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• ROCm MI2xx CI/CD workflows failing due to : download from https://api.github.com/repos/pytorch/pytorch timed out.

❌ 7 New Failures, 8 Unrelated Failures

As of commit ebb55e1 with merge base 1520f9f ():

NEW FAILURES - The following jobs have failed:

• android-perf / benchmark-on-device (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, samsung_galaxy_s22, ... / mobile-job (android) (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/test-infra/tools/device-farm-runner'. No such file or directory
• android-perf / benchmark-on-device (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, samsung_galaxy_s22, ... / mobile-job (android) (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/test-infra/tools/device-farm-runner'. No such file or directory
• android-perf / build-benchmark-app / linux-job (gh)
  RuntimeError: Command docker exec -t a87dcc61b36eda06efe44857311239f04d08a98a4153c6895f83c52dec7f75a3 /exec failed with exit code 2
• android-perf / export-models (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, samsung_galaxy_s22, public... / linux-job (gh)
  RuntimeError: Command docker exec -t 2ebaab8fd779bec9df521e59f3bbadfed7cd0f061cd3d07e11608a16f505e8bb /exec failed with exit code 2
• android-perf / export-models (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, samsung_galaxy_s22, public... / linux-job (gh)
  RuntimeError: Command docker exec -t 46bc1fb39741d749afdd0970073a0bb955eb44dd5afa750944f1ac3cc77a037f /exec failed with exit code 2
• android-perf / upload-benchmark-results (gh)
  Could not assume role with user credentials: User: arn:aws:sts::308535385114:assumed-role/gh-ci-github-action-runners-runner-role/i-02a42a5f0bb71f0cd is not authorized to perform: sts:TagSession on resource: arn:aws:iam::308535385114:role/gha_workflow_upload-benchmark-results
• Build documentation / build (buck2) / Build doc (gh)
  At least one of the pre-conditions you specified did not hold

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh) (trunk failure)
  AttributeError: 'bool' object has no attribute 'lower'
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh) (trunk failure)
  AttributeError: 'bool' object has no attribute 'lower'
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh) (trunk failure)
  AttributeError: 'bool' object has no attribute 'lower'
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-torchao-lowbit / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-mcu-models (cmake) / linux-job (gh) (trunk failure)
  AttributeError: 'bool' object has no attribute 'lower'

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[cccclai]

@winskuo-quic do you know when it will be ready to bump the version?

[winskuo-quic]

@winskuo-quic do you know when it will be ready to bump the version?

Hi @cccclai,
I think this PR is ready and we can do uplevel first. Please have a look.
There are only a few UT failing under specific use cases in 2.37.0, which should not affect current GA enablement.
We can do up level first, and we will have another PR addressing these issues.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D81207378.

[cccclai]

There is a bit more hardcode logic introduced here #13687 mind updating together?

[winskuo-quic]

There is a bit more hardcode logic introduced here #13687 mind updating together?

Thanks for reviewing the PR.
I have pushed a new commit that should change 2.28.0 to 2.37.0.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D81207378.

[cccclai]

Thanks! Can you rebase?

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D81207378.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D81207378.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D81207378.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D81207378.

[cccclai]

Sorry still having trouble to import, can you rebase again?

[winskuo-quic]

A lot of CIs are failing, which did not happen in other PRs I pushed.
I am unsure if it is this PR that is causing these to fail. A lot of them seems related to AWS server error or $SECRET_EXECUTORCH_HF_TOKEN not found.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D81207378.

[cccclai]

A lot of CIs are failing, which did not happen in other PRs I pushed. I am unsure if it is this PR that is causing these to fail. A lot of them seems related to AWS server error or $SECRET_EXECUTORCH_HF_TOKEN not found.

I think it just trigger the trunk job. Aside from, the import works and I'll wait for the CI signal. If it's all good, we will merge this PR

[cccclai]

Thanks!

[cccclai]

@winskuo-quic there are a few more androd perf test failing seems related to this PR, can you take a look?
https://hud.pytorch.org/pytorch/executorch/commit/c3f8d64d9d24ad8a69537408ef2f05e3ba4670ac

[shoumikhin]

@winskuo-quic please check is this change has lead to the nightly CI failures, like this one
https://hud.pytorch.org/pytorch/executorch/commit/43bd889fc22199df12a301c6cfa7b2142e6930e7

2025-09-04T02:45:03.2835693Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnImplementation.cpp:8:
2025-09-04T02:45:03.2837864Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnImplementation.h:11:
2025-09-04T02:45:03.2840433Z /pytorch/executorch/backends/qualcomm/runtime/backends/QnnFunctionInterface.h:10:10: fatal error: 'QnnInterface.h' file not found
2025-09-04T02:45:03.2842379Z   #include "QnnInterface.h"
2025-09-04T02:45:03.2842983Z            ^~~~~~~~~~~~~~~~
2025-09-04T02:45:03.2843544Z   1 error generated.

2025-09-04T02:45:03.2863171Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnSysImplementation.cpp:10:
2025-09-04T02:45:03.2865475Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnSysImplementation.h:11:
2025-09-04T02:45:03.2868370Z /pytorch/executorch/backends/qualcomm/runtime/backends/QnnSysFunctionInterface.h:10:10: fatal error: 'System/QnnSystemInterface.h' file not found
2025-09-04T02:45:03.2871804Z   #include "System/QnnSystemInterface.h"
2025-09-04T02:45:03.2872568Z            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2025-09-04T02:45:03.2873241Z   1 error generated.

[winskuo-quic]

@winskuo-quic please check is this change has lead to the nightly CI failures, like this one https://hud.pytorch.org/pytorch/executorch/commit/43bd889fc22199df12a301c6cfa7b2142e6930e7

2025-09-04T02:45:03.2835693Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnImplementation.cpp:8:
2025-09-04T02:45:03.2837864Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnImplementation.h:11:
2025-09-04T02:45:03.2840433Z /pytorch/executorch/backends/qualcomm/runtime/backends/QnnFunctionInterface.h:10:10: fatal error: 'QnnInterface.h' file not found
2025-09-04T02:45:03.2842379Z   #include "QnnInterface.h"
2025-09-04T02:45:03.2842983Z            ^~~~~~~~~~~~~~~~
2025-09-04T02:45:03.2843544Z   1 error generated.

2025-09-04T02:45:03.2863171Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnSysImplementation.cpp:10:
2025-09-04T02:45:03.2865475Z In file included from /pytorch/executorch/backends/qualcomm/runtime/backends/QnnSysImplementation.h:11:
2025-09-04T02:45:03.2868370Z /pytorch/executorch/backends/qualcomm/runtime/backends/QnnSysFunctionInterface.h:10:10: fatal error: 'System/QnnSystemInterface.h' file not found
2025-09-04T02:45:03.2871804Z   #include "System/QnnSystemInterface.h"
2025-09-04T02:45:03.2872568Z            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2025-09-04T02:45:03.2873241Z   1 error generated.

Hi @shoumikhin,
I think there's a conflict when merging this PR, where another PR is also modifying the QNN_SDK_ROOT.
I believe this PR should be able to fix it: #13927.
Thanks