Skip to content

Commit 698ea79

Browse files
Qualcomm AI Engine Direct - docs fix (pytorch#14881)
### Summary update README on python dependencies and recommended QNN version cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @cbilgin
1 parent 6520e06 commit 698ea79

File tree

3 files changed

+9
-88
lines changed

3 files changed

+9
-88
lines changed

docs/source/backends-qualcomm.md

Lines changed: 4 additions & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -74,10 +74,9 @@ This example is verified with SM8550 and SM8450.
7474
- A compiler to compile AOT parts, e.g., the GCC compiler comes with Ubuntu LTS.
7575
- [Android NDK](https://developer.android.com/ndk). This example is verified with NDK 26c.
7676
- [Qualcomm AI Engine Direct SDK](https://developer.qualcomm.com/software/qualcomm-ai-engine-direct-sdk)
77-
- Click the "Get Software" button to download a version of QNN SDK.
78-
- However, at the moment of updating this tutorial, the above website doesn't provide QNN SDK newer than 2.22.6.
79-
- The below is public links to download various QNN versions. Hope they can be publicly discoverable soon.
80-
- [QNN 2.37.0](https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.37.0.250724/v2.37.0.250724.zip)
77+
- Click the "Get Software" button to download the latest version of the QNN SDK.
78+
- Although newer versions are available, we have verified and recommend using QNN 2.37.0 for stability.
79+
- You can download it directly from the following link: [QNN 2.37.0](https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.37.0.250724/v2.37.0.250724.zip)
8180

8281
The directory with installed Qualcomm AI Engine Direct SDK looks like:
8382
```
@@ -136,86 +135,6 @@ cd $EXECUTORCH_ROOT
136135
./backends/qualcomm/scripts/build.sh --release
137136
```
138137

139-
### AOT (Ahead-of-time) components:
140-
141-
Python APIs on x64 are required to compile models to Qualcomm AI Engine Direct binary.
142-
143-
```bash
144-
cd $EXECUTORCH_ROOT
145-
mkdir build-x86
146-
cd build-x86
147-
# Note that the below command might change.
148-
# Please refer to the above build.sh for latest workable commands.
149-
cmake .. \
150-
-DCMAKE_INSTALL_PREFIX=$PWD \
151-
-DEXECUTORCH_BUILD_QNN=ON \
152-
-DQNN_SDK_ROOT=${QNN_SDK_ROOT} \
153-
-DEXECUTORCH_BUILD_DEVTOOLS=ON \
154-
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
155-
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
156-
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
157-
-DPYTHON_EXECUTABLE=python3
158-
159-
# nproc is used to detect the number of available CPU.
160-
# If it is not applicable, please feel free to use the number you want.
161-
cmake --build $PWD --target "PyQnnManagerAdaptor" "PyQnnWrapperAdaptor" -j$(nproc)
162-
163-
# install Python APIs to correct import path
164-
# The filename might vary depending on your Python and host version.
165-
cp -f backends/qualcomm/PyQnnManagerAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python
166-
cp -f backends/qualcomm/PyQnnWrapperAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python
167-
168-
# Workaround for .fbs files in exir/_serialize
169-
cp $EXECUTORCH_ROOT/schema/program.fbs $EXECUTORCH_ROOT/exir/_serialize/program.fbs
170-
cp $EXECUTORCH_ROOT/schema/scalar_type.fbs $EXECUTORCH_ROOT/exir/_serialize/scalar_type.fbs
171-
```
172-
173-
### Runtime:
174-
175-
An example `qnn_executor_runner` executable would be used to run the compiled `pte` model.
176-
177-
Commands to build `qnn_executor_runner` for Android:
178-
179-
```bash
180-
cd $EXECUTORCH_ROOT
181-
mkdir build-android
182-
cd build-android
183-
# build executorch & qnn_executorch_backend
184-
cmake .. \
185-
-DCMAKE_INSTALL_PREFIX=$PWD \
186-
-DEXECUTORCH_BUILD_QNN=ON \
187-
-DQNN_SDK_ROOT=$QNN_SDK_ROOT \
188-
-DEXECUTORCH_BUILD_DEVTOOLS=ON \
189-
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
190-
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
191-
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
192-
-DPYTHON_EXECUTABLE=python3 \
193-
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_ROOT/build/cmake/android.toolchain.cmake \
194-
-DANDROID_ABI='arm64-v8a' \
195-
-DANDROID_PLATFORM=android-30
196-
197-
# nproc is used to detect the number of available CPU.
198-
# If it is not applicable, please feel free to use the number you want.
199-
cmake --build $PWD --target install -j$(nproc)
200-
201-
cmake ../examples/qualcomm \
202-
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_ROOT/build/cmake/android.toolchain.cmake \
203-
-DANDROID_ABI='arm64-v8a' \
204-
-DANDROID_PLATFORM=android-30 \
205-
-DCMAKE_PREFIX_PATH="$PWD/lib/cmake/ExecuTorch;$PWD/third-party/gflags;" \
206-
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH \
207-
-DPYTHON_EXECUTABLE=python3 \
208-
-Bexamples/qualcomm
209-
210-
cmake --build examples/qualcomm -j$(nproc)
211-
212-
# qnn_executor_runner can be found under examples/qualcomm
213-
# The full path is $EXECUTORCH_ROOT/build-android/examples/qualcomm/executor_runner/qnn_executor_runner
214-
ls examples/qualcomm
215-
```
216-
217-
**Note:** If you want to build for release, add `-DCMAKE_BUILD_TYPE=Release` to the `cmake` command options.
218-
219138

220139
## Deploying and running on device
221140

@@ -365,7 +284,7 @@ The model, inputs, and output location are passed to `qnn_executorch_runner` by
365284

366285
## Supported model list
367286

368-
Please refer to `$EXECUTORCH_ROOT/examples/qualcomm/scripts/` and `EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/` to the list of supported models.
287+
Please refer to `$EXECUTORCH_ROOT/examples/qualcomm/scripts/` and `$EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/` to the list of supported models.
369288

370289
## How to Support a Custom Model in HTP Backend
371290

examples/qualcomm/README.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -111,12 +111,13 @@ This section outlines the essential APIs and utilities provided to streamline th
111111
Creates a clean directory for storing model outputs or intermediate results. If the directory already exists, it will be deleted and recreated to ensure a consistent environment for each run.
112112

113113
## Additional Dependency
114+
This example requires the following Python packages:
115+
- pandas and scikit-learn: used in the mobilebert multi-class text classification example.
116+
- graphviz (optional): used for visualizing QNN graphs during debugging.
114117

115-
The mobilebert multi-class text classification example requires `pandas` and `sklearn`.
116118
Please install them by something like
117-
118119
```bash
119-
pip install scikit-learn pandas
120+
pip install scikit-learn pandas graphviz
120121
```
121122

122123
## Limitation

examples/qualcomm/oss_scripts/llama/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ We offer the following modes to execute the model:
3838
### Step 1: Setup
3939
1. Follow the [tutorial](https://pytorch.org/executorch/main/getting-started-setup) to set up ExecuTorch.
4040
2. Follow the [tutorial](https://pytorch.org/executorch/main/backends-qualcomm) to build Qualcomm AI Engine Direct Backend.
41+
3. Please install the llm eval dependency via [examples/models/llama/install_requirements.sh](https://github.com/pytorch/executorch/blob/main/examples/models/llama/install_requirements.sh)
4142

4243
### Step 2: Prepare Model
4344

0 commit comments

Comments
 (0)