Qualcomm AI Engine Direct - LE support #12164

haowhsu-quic · 2025-07-02T14:41:19Z

Summary

enable linux embedded support for toolchain: aarch64-oe-linux-gcc9.3 (SXR1230P)
update document a bit

Test plan

python backends/qualcomm/tests/test_qnn_delegate.py TestQNNQuantizedOperator.test_qnn_backend_conv2d -b build-oe-linux/ -s 2ff76932 -m SXR1230P -t aarch64-oe-linux-gcc9.3

pytorch-bot · 2025-07-02T14:41:24Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12164

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Cancelled Jobs

As of commit 229a3f6 with merge base f4d801a ():

NEW FAILURES - The following jobs have failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t 23f26c144832d63d7c3aaafd0dfb0ba4e193d09b154da7695c158ce460b47284 /exec failed with exit code 1
Test CUDA Builds / export-voxtral-cuda-artifact / linux-job (gh)
RuntimeError: Command docker exec -t bb71cc968358aff140c9407a6fd83cc16afea6ab4302a39a4423ee2965116d36 /exec failed with exit code 2

CANCELLED JOBS - The following jobs were cancelled. Please retry:

pull / test-static-llama-qnn-linux / linux-job (gh)
##[error]The operation was canceled.
trunk / test-models-macos-coreml (efficient_sam) / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

haowhsu-quic · 2025-07-02T14:42:39Z

@pytorchbot label "release notes: qualcomm"

cccclai · 2025-07-04T00:24:42Z

docs/source/backends-qualcomm.md

 - SSG2115P
 - SSG2125P
- - SXR1230P
+ - SXR1230P (Linux Embedded)


I thought SXR1230P is just a SoC similar to others. How is it special compared with the others?

Sorry that I completely missed this. I think it depends on the device meta: like the info dumped from my device cat /firmware/verinfo/ver_info.txt: Aurora.LE.2.3-00040-STD.PROD-1.
Some SoCs might have LE / LA simultaneously, but I think this one we only provide the LE meta build.

cccclai · 2025-07-04T00:28:09Z

backends/qualcomm/tests/test_qnn_delegate.py

+        "-t",
+        "--target",
+        help="Target platform for deployment",
+        default="aarch64-android",


Can we add choices so users know what they can choose?

Suggested change

default="aarch64-android",

choices=["aarch64-android", "linux"]

default="aarch64-android",

That's even better, thank you.

cccclai · 2025-07-11T20:56:13Z

Sorry I need to spend a bit more time on this, because we don't have CI to test the pllm model and I'm worried it will cause breakage

haowhsu-quic · 2025-07-18T05:43:54Z

Sorry I need to spend a bit more time on this, because we don't have CI to test the pllm model and I'm worried it will cause breakage

No worries, this PR is also a PoC for IoT (the toolchain will mostly be LE) device users.

facebook-github-bot · 2025-09-11T16:18:27Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D82227074.

cccclai · 2025-09-11T16:18:33Z

there is a lint error and some ci error

chenweng-quic · 2025-09-17T06:07:29Z

Hi @cccclai,
It seems like CI is not running, could you please check that?

cccclai · 2025-09-17T16:52:28Z

Hi @cccclai, It seems like CI is not running, could you please check that?

Just approve and have it running, not quite sure why it's not triggered automatically...

cccclai · 2025-09-23T01:11:59Z

Seems like lint error and qnn related tests are failing

cccclai · 2025-09-23T17:45:00Z

backends/qualcomm/scripts/build.sh

-CMAKE_AARCH64="build-android"
+BUILD_ANDROID="true"
+CMAKE_ANDROID="build-android"
+BUILD_OE_LINUX="false"


If it's default to false, why do we still need to explicitly say --skip-le?

This is used when --no_clean appears. Sometimes user makes small changes on x86 only (e.g. pybind layer), they could skip the target platform like --skip_la or --skip_le.

Can you remind me what la is? le seems to be linux_embedding, if so, let's use the whole name just so it's easier to follow. Another comment is that, can we make le optional? I'd like to avoid patching it everywhere, and it seems like linux embedding isn't super common (?)

la stands for linux android and le will only be functional if user specifies "build.sh --le". It's commonly used in IoT devices, actually customers like Amazon have requirements for us to enable this.

I see, glad that Amazon is requesting this! Question about the PR, if le will only be functional when users specifybuild.sh --le, why do we need to explicitly skip -le in setup.py. Also, a minor comment to use linux_embedding and linux_android so it's easier to understand the code

I think it is la in setup.py and we will rename the variable, thank you for the suggestion.

cccclai · 2025-10-02T17:32:57Z

I feel like some CI is failing, but I'm not sure if it's related...also static-llama-qnn seems cancelled somehow

cccclai · 2025-10-02T17:34:22Z

After double check the log, I feel like this PR base commit is still old, can you rebase?

meta-codesync · 2025-10-08T03:34:17Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D82227074.

cccclai · 2025-10-09T17:18:53Z

There seems to be an error

  File "executorch/examples/qualcomm/oss_scripts/whisper/whisper.py", line 430
SyntaxError: keyword argument repeated: shared_buffer

chenweng-quic · 2025-10-15T23:53:10Z

Hi @cccclai,
Could you take a look of the CI error?
"Lint / lintrunner-mypy" pipeline failed at a file which I did not touch.
"pull / test-qnn-wheel-packages-linux" pipelines failed due to torch not found.

cccclai · 2025-10-16T16:11:03Z

"Lint / lintrunner-mypy" pipeline failed at a file which I did not touch.

Ah yeah, it was failing in main last week. I think rebasing should fix the issue

Summary: - enable linux embedded support for toolchain: aarch64-oe-linux-gcc9.3, aarch64-oe-linux-gcc11.2 (SXR1230P, QCS9100) - update document a bit

haowhsu-quic requested review from cccclai and mergennachin as code owners July 2, 2025 14:41

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 2, 2025

pytorch-bot bot added the release notes: qualcomm Changes to the Qualcomm backend delegate label Jul 2, 2025

cccclai approved these changes Jul 4, 2025

View reviewed changes

haowhsu-quic force-pushed the dev_enable_le branch from 5b51d08 to f7771ea Compare July 25, 2025 14:09

haowhsu-quic requested review from kirklandsign and larryliu0820 as code owners July 25, 2025 14:09

chenweng-quic force-pushed the dev_enable_le branch from f7771ea to 76af36b Compare September 11, 2025 04:04

cccclai reviewed Sep 23, 2025

View reviewed changes

chenweng-quic force-pushed the dev_enable_le branch 6 times, most recently from 7afa780 to 902aea7 Compare October 1, 2025 06:23

chenweng-quic force-pushed the dev_enable_le branch 3 times, most recently from be0f334 to d779ac4 Compare October 8, 2025 02:27

chenweng-quic force-pushed the dev_enable_le branch 2 times, most recently from 2c80292 to 7e14ae0 Compare October 9, 2025 08:31

chenweng-quic force-pushed the dev_enable_le branch from b29b46d to 5f21d9a Compare October 15, 2025 08:56

Qualcomm AI Engine Direct - LE support

229a3f6

Summary: - enable linux embedded support for toolchain: aarch64-oe-linux-gcc9.3, aarch64-oe-linux-gcc11.2 (SXR1230P, QCS9100) - update document a bit

chenweng-quic force-pushed the dev_enable_le branch from 5f21d9a to 229a3f6 Compare October 17, 2025 02:27

	default="aarch64-android",
	choices=["aarch64-android", "linux"]
	default="aarch64-android",

Qualcomm AI Engine Direct - LE support #12164

Are you sure you want to change the base?

Qualcomm AI Engine Direct - LE support #12164

Uh oh!

Conversation

haowhsu-quic commented Jul 2, 2025

Summary

Test plan

Uh oh!

pytorch-bot bot commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12164

❌ 2 New Failures, 2 Cancelled Jobs

Uh oh!

haowhsu-quic commented Jul 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cccclai commented Jul 11, 2025

Uh oh!

haowhsu-quic commented Jul 18, 2025

Uh oh!

facebook-github-bot commented Sep 11, 2025

Uh oh!

cccclai commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chenweng-quic commented Sep 17, 2025

Uh oh!

cccclai commented Sep 17, 2025

Uh oh!

cccclai commented Sep 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cccclai Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cccclai commented Oct 2, 2025

Uh oh!

cccclai commented Oct 2, 2025

Uh oh!

meta-codesync bot commented Oct 8, 2025

Uh oh!

cccclai commented Oct 9, 2025

Uh oh!

chenweng-quic commented Oct 15, 2025

Uh oh!

cccclai commented Oct 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pytorch-bot bot commented Jul 2, 2025 •

edited

Loading

cccclai commented Sep 11, 2025 •

edited

Loading

cccclai Sep 26, 2025 •

edited

Loading