Skip to content

Qualcomm AI Engine Direct - LE support #12164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Qualcomm AI Engine Direct - LE support #12164

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ci/scripts/build-qnn-sdk.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ build_qnn_backend() {
export EXECUTORCH_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)"

parallelism=$(( $(nproc) - 1 ))
bash backends/qualcomm/scripts/build.sh --skip_aarch64 --job_number ${parallelism} --release
bash backends/qualcomm/scripts/build.sh --skip_linux_android --skip_linux_embedding --job_number ${parallelism} --release
}

set_up_aot() {
Expand Down
1 change: 1 addition & 0 deletions backends/qualcomm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Please check `generate_qnn_executorch_compiler_spec()` in
- SXR1230P
- SXR2230P
- SXR2330P
- QCS9100

### Adding more supported Chipset
Currently, users cannot add additional chipset models because the chipset ID is not accessible to community users. If you have specific chipset models you wish to add, please contact one of the authors in the `Code Reviews` section at the bottom of this page.
Expand Down
1 change: 1 addition & 0 deletions backends/qualcomm/debugger/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ adb = SimpleADB(
device_id=args.device,
host_id=args.host,
soc_model=args.model,
target=args.target,
)
binaries_trace = generate_optrace(
args, adb, f"{args.artifact}/{pte_filename}.pte", example_input
Expand Down
108 changes: 101 additions & 7 deletions backends/qualcomm/scripts/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@ fi
usage() {
echo "Usage: Build the aarch64 version of executor runner or the python interface of Qnn Manager"
echo "First, you need to set the environment variable for QNN_SDK_ROOT"
echo ", and if you want to build the aarch64 version of executor runner"
echo ", and if you want to build the android version of executor runner"
echo ", you need to export ANDROID_NDK_ROOT=/path/to/android_ndkXX"
echo "(or export TOOLCHAIN_ROOT_HOST=/path/to/sysroots/xx_host, "
echo "TOOLCHAIN_ROOT_TARGET=/path/to/sysroots/xx_target for linux embedded with --enable_linux_embedding)"
echo "e.g.: executorch$ ./backends/qualcomm/scripts/build.sh --skip_x86_64"
exit 1
}
Expand All @@ -28,8 +30,10 @@ usage() {

BUILD_X86_64="true"
CMAKE_X86_64="build-x86"
BUILD_AARCH64="true"
CMAKE_AARCH64="build-android"
BUILD_ANDROID="true"
CMAKE_ANDROID="build-android"
BUILD_OE_LINUX="false"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's default to false, why do we still need to explicitly say --skip-le?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used when --no_clean appears. Sometimes user makes small changes on x86 only (e.g. pybind layer), they could skip the target platform like --skip_la or --skip_le.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remind me what la is? le seems to be linux_embedding, if so, let's use the whole name just so it's easier to follow. Another comment is that, can we make le optional? I'd like to avoid patching it everywhere, and it seems like linux embedding isn't super common (?)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

la stands for linux android and le will only be functional if user specifies "build.sh --le". It's commonly used in IoT devices, actually customers like Amazon have requirements for us to enable this.

Copy link
Contributor

@cccclai cccclai Sep 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, glad that Amazon is requesting this! Question about the PR, if le will only be functional when users specifybuild.sh --le, why do we need to explicitly skip -le in setup.py. Also, a minor comment to use linux_embedding and linux_android so it's easier to understand the code

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is la in setup.py and we will rename the variable, thank you for the suggestion.

CMAKE_OE_LINUX="build-oe-linux"
CLEAN="true"
BUILD_TYPE="RelWithDebInfo"
BUILD_JOB_NUMBER="16"
Expand All @@ -42,7 +46,7 @@ if [ -z BUCK2 ]; then
BUCK2="buck2"
fi

long_options=skip_x86_64,skip_aarch64,no_clean,release,job_number:
long_options=skip_x86_64,skip_linux_android,skip_linux_embedding,enable_linux_embedding,no_clean,release,job_number:

parsed_args=$(getopt -a --options '' --longoptions $long_options --name "$0" -- "$@")
eval set -- "$parsed_args"
Expand All @@ -51,7 +55,9 @@ eval set -- "$parsed_args"
while true ; do
case "$1" in
--skip_x86_64) BUILD_X86_64="false"; shift;;
--skip_aarch64) BUILD_AARCH64="false"; shift;;
--skip_linux_android) BUILD_ANDROID="false"; shift;;
--skip_linux_embedding) BUILD_OE_LINUX="false"; shift;;
--enable_linux_embedding) BUILD_ANDROID="false"; BUILD_OE_LINUX="true"; shift;;
--no_clean) CLEAN="false"; shift;;
--release) BUILD_TYPE="Release"; shift;;
--job_number) BUILD_JOB_NUMBER="$2"; shift 2;;
Expand All @@ -61,13 +67,13 @@ done

PRJ_ROOT="$( cd "$(dirname "$0")/../../.." ; pwd -P)"

if [ "$BUILD_AARCH64" = true ]; then
if [ "$BUILD_ANDROID" = true ]; then
if [[ -z ${ANDROID_NDK_ROOT} ]]; then
echo "Please export ANDROID_NDK_ROOT=/path/to/android_ndkXX"
exit -1
fi

BUILD_ROOT=$PRJ_ROOT/$CMAKE_AARCH64
BUILD_ROOT=$PRJ_ROOT/$CMAKE_ANDROID
if [ "$CLEAN" = true ]; then
rm -rf $BUILD_ROOT && mkdir $BUILD_ROOT
else
Expand Down Expand Up @@ -135,6 +141,94 @@ if [ "$BUILD_AARCH64" = true ]; then
cmake --build $LLAMA_EXAMPLE_ROOT -j$BUILD_JOB_NUMBER
fi

if [ "$BUILD_OE_LINUX" = true ]; then
if [[ -z ${TOOLCHAIN_ROOT_HOST} ]]; then
echo "Please export e.g. TOOLCHAIN_ROOT_HOST=/path/to/sysroots/x86_64-qtisdk-linux"
exit -1
fi
if [[ -z ${TOOLCHAIN_ROOT_TARGET} ]]; then
echo "Please export e.g. TOOLCHAIN_ROOT_TARGET=/path/to/sysroots/armv8a-oe-linux"
exit -1
fi

BUILD_ROOT=$PRJ_ROOT/$CMAKE_OE_LINUX
if [ "$CLEAN" = true ]; then
rm -rf $BUILD_ROOT && mkdir $BUILD_ROOT
else
# Force rebuild flatccrt for the correct platform
cd $BUILD_ROOT/third-party/flatcc && make clean
fi

TOOLCHAN_PREFIX=$TOOLCHAIN_ROOT_HOST/usr/bin/aarch64-oe-linux/aarch64-oe-linux-
cd $BUILD_ROOT
cmake .. \
-DCMAKE_INSTALL_PREFIX=$BUILD_ROOT \
-DCMAKE_BUILD_TYPE=$BUILD_TYPE \
-DCMAKE_C_COMPILER=${TOOLCHAN_PREFIX}gcc \
-DCMAKE_CXX_COMPILER=${TOOLCHAN_PREFIX}g++ \
-DCMAKE_SYSROOT=$TOOLCHAIN_ROOT_TARGET \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER \
-DEXECUTORCH_BUILD_QNN=ON \
-DEXECUTORCH_BUILD_DEVTOOLS=ON \
-DEXECUTORCH_BUILD_EXTENSION_LLM=ON \
-DEXECUTORCH_BUILD_EXTENSION_LLM_RUNNER=ON \
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
-DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \
-DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON \
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
-DEXECUTORCH_ENABLE_LOGGING=ON \
-DQNN_SDK_ROOT=$QNN_SDK_ROOT \
-DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
-DPYTHON_EXECUTABLE=$PYTHON_EXECUTABLE \
-B$BUILD_ROOT

cmake --build $BUILD_ROOT -j$BUILD_JOB_NUMBER --target install

EXAMPLE_ROOT=examples/qualcomm
CMAKE_PREFIX_PATH="${BUILD_ROOT};${BUILD_ROOT}/third-party/gflags;"

cmake $PRJ_ROOT/$EXAMPLE_ROOT \
-DCMAKE_BUILD_TYPE=$BUILD_TYPE \
-DCMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH \
-DSUPPORT_REGEX_LOOKAHEAD=ON \
-DBUILD_TESTING=OFF \
-DEXECUTORCH_ENABLE_LOGGING=ON \
-DCMAKE_C_COMPILER=${TOOLCHAN_PREFIX}gcc \
-DCMAKE_CXX_COMPILER=${TOOLCHAN_PREFIX}g++ \
-DCMAKE_SYSROOT=$TOOLCHAIN_ROOT_TARGET \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
-DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER \
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH \
-DPYTHON_EXECUTABLE=$PYTHON_EXECUTABLE \
-B$EXAMPLE_ROOT

cmake --build $EXAMPLE_ROOT -j$BUILD_JOB_NUMBER

LLAMA_EXAMPLE_ROOT=examples/models/llama
cmake $PRJ_ROOT/$LLAMA_EXAMPLE_ROOT \
-DBUILD_TESTING=OFF \
-DCMAKE_BUILD_TYPE=$BUILD_TYPE \
-DCMAKE_C_COMPILER=${TOOLCHAN_PREFIX}gcc \
-DCMAKE_CXX_COMPILER=${TOOLCHAN_PREFIX}g++ \
-DCMAKE_SYSROOT=$TOOLCHAIN_ROOT_TARGET \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER \
-DCMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH \
-DEXECUTORCH_ENABLE_LOGGING=ON \
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH \
-DPYTHON_EXECUTABLE=$PYTHON_EXECUTABLE \
-B$LLAMA_EXAMPLE_ROOT

cmake --build $LLAMA_EXAMPLE_ROOT -j$BUILD_JOB_NUMBER
fi

if [ "$BUILD_X86_64" = true ]; then
BUILD_ROOT=$PRJ_ROOT/$CMAKE_X86_64
if [ "$CLEAN" = true ]; then
Expand Down
1 change: 1 addition & 0 deletions backends/qualcomm/serialization/qc_compiler_spec.fbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ enum QcomChipset: int {
SXR1230P = 45,
SXR2230P = 53,
SXR2330P = 75,
QCS9100 = 77,
}

/// Indicate the information of the specified SoC.
Expand Down
2 changes: 2 additions & 0 deletions backends/qualcomm/serialization/qc_schema.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ class QcomChipset(IntEnum):
SXR1230P = 45 # v73
SXR2230P = 53 # v69
SXR2330P = 75 # v79
QCS9100 = 77 # v73


@dataclass
Expand All @@ -69,6 +70,7 @@ class SocInfo:
QcomChipset.SXR1230P: SocInfo(QcomChipset.SXR1230P, HtpInfo(HtpArch.V73, 2)),
QcomChipset.SXR2230P: SocInfo(QcomChipset.SXR2230P, HtpInfo(HtpArch.V69, 8)),
QcomChipset.SXR2330P: SocInfo(QcomChipset.SXR2330P, HtpInfo(HtpArch.V79, 8)),
QcomChipset.QCS9100: SocInfo(QcomChipset.QCS9100, HtpInfo(HtpArch.V73, 8)),
}


Expand Down
Loading
Loading