Skip to content

Commit 823dea1

Browse files
Jiseong-ohchong-chen01hoon98-choi
authored
Exynos Backend for Executorch to bring up on Exynos SoC (pytorch#13677)
### Summary This PR introduces support for the Exynos backend of Executorch, can be inferenced on Exynos acceletors. Exynos Backend is a crucial component designed to optimize the performance of AI models on Exynos chipsets. Leveraging Exynos AI LiteCore, it enables comprehensive support for NPU/DSP, CPU, and GPU, ensuring seamless execution of AI workloads on Exynos-based devices. Below is a description of the features: * Exynos Backend : The backends/samsung directory were made by ENN Partitioner, I/F for communication with Exynos LiteCore, runtime backend files and device-specific optimization. See backend/samsung/README.md for usage * Exynos examples: The examples/samsung directory privides scripts for AOT compilation, quantization and so on. please refer examples/samsung/README.md ### Test plan This PR is tested with Exynos Backend on NPU/DSP/CPU/GPU of Exynos 2500 SoC. To run the layer and models tests with README.md cc: @mergennachin @SS-JIA @digantdesai @kimishpatel --------- Signed-off-by: jiseong.oh <[email protected]> Signed-off-by: chong-chen <[email protected]> Co-authored-by: chong-chen <[email protected]> Co-authored-by: Hoon-Choi <[email protected]>
1 parent 879e7ba commit 823dea1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+4298
-0
lines changed

‎.ci/scripts/setup-samsung-linux-deps.sh‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
#!/bin/bash
2+
# Copyright (c) Meta Platforms, Inc. and affiliates.
3+
# Copyright (c) Samsung Electronics Co. LTD
4+
# All rights reserved.
5+
#
6+
# This source code is licensed under the BSD-style license found in the
7+
# LICENSE file in the root directory of this source tree.
8+
9+
set -ex
10+
11+
12+
download_ai_lite_core() {
13+
API_BASE="https://soc-developer.semiconductor.samsung.com/api/v1/resource/ai-litecore/download"
14+
API_KEY="kn10SoSY3hkC-9Qny5TqD2mnqVrlupv3krnjLeBt5cY"
15+
16+
VERSION="0.5"
17+
OS_NAME="Ubuntu 22.04"
18+
OUT_FILE="/tmp/exynos-ai-litecore-v${VERSION}.tar.gz"
19+
TARGET_PATH="/tmp/exynos_ai_lite_core"
20+
21+
mkdir -p ${TARGET_PATH}
22+
# Presigned issue URL
23+
JSON_RESP=$(curl -sS -G \
24+
--location --fail --retry 3 \
25+
-H "apikey: ${API_KEY}" \
26+
--data-urlencode "version=${VERSION}" \
27+
--data-urlencode "os=${OS_NAME}" \
28+
"${API_BASE}")
29+
30+
DOWNLOAD_URL=$(echo "$JSON_RESP" | sed -n 's/.*"data":[[:space:]]*"\([^"]*\)".*/\1/p')
31+
32+
if [[ -z "$DOWNLOAD_URL" ]]; then
33+
echo "Failed to extract download URL"
34+
echo "$JSON_RESP"
35+
exit 1
36+
fi
37+
38+
# Download LiteCore
39+
curl -sS -L --fail --retry 3 \
40+
--output "$OUT_FILE" \
41+
"$DOWNLOAD_URL"
42+
43+
echo "Download done: $OUT_FILE"
44+
45+
46+
tar -C "${TARGET_PATH}" --strip-components=1 -xzvf "${OUT_FILE}"
47+
48+
export EXYNOS_AI_LITECORE_ROOT=${TARGET_PATH}
49+
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-}:${EXYNOS_AI_LITECORE_ROOT}/lib/x86_64-linux
50+
}
51+
52+
install_enn_backend() {
53+
NDK_INSTALLATION_DIR=/opt/ndk
54+
rm -rf "${NDK_INSTALLATION_DIR}" && sudo mkdir -p "${NDK_INSTALLATION_DIR}"
55+
ANDROID_NDK_VERSION=r27b
56+
57+
pushd .
58+
cd /tmp
59+
curl -Os --retry 3 "https://ossci-android.s3.amazonaws.com/android-ndk-${ANDROID_NDK_VERSION}-linux.zip"
60+
unzip -qo "android-ndk-${ANDROID_NDK_VERSION}-linux.zip"
61+
62+
# Print the content for manual verification
63+
ls -lah "android-ndk-${ANDROID_NDK_VERSION}"
64+
sudo mv "android-ndk-${ANDROID_NDK_VERSION}"/* "${NDK_INSTALLATION_DIR}"
65+
popd
66+
# build Exynos backend
67+
export ANDROID_NDK_ROOT=${ANDROID_NDK_ROOT:-/opt/ndk}
68+
bash backends/samsung/build.sh --build all
69+
# set env variable
70+
export EXECUTORCH_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)"
71+
export PYTHONPATH=${PYTHONPATH:-}:${EXECUTORCH_ROOT}/..
72+
}
73+
74+
AI_LITE_CORE_VERSION=0.5.0
75+
76+
download_ai_lite_core ${AI_LITE_CORE_VERSION}
77+
install_enn_backend

‎.github/workflows/pull.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -866,6 +866,41 @@ jobs:
866866
PYTHON_EXECUTABLE=python bash examples/nxp/run_aot_example.sh mobilenetv2
867867
868868
869+
test-samsung-models-linux:
870+
name: test-samsung-models-linux
871+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
872+
permissions:
873+
id-token: write
874+
contents: read
875+
with:
876+
runner: linux.2xlarge
877+
docker-image: ci-image:executorch-ubuntu-22.04-gcc9
878+
submodules: 'recursive'
879+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
880+
timeout: 90
881+
script: |
882+
set -ex
883+
884+
# The generic Linux job chooses to use base env, not the one setup by the image
885+
CONDA_ENV=$(conda env list --json | jq -r ".envs | .[-1]")
886+
conda activate "${CONDA_ENV}"
887+
888+
# Setup python
889+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh --build-tool "cmake"
890+
891+
# Setup Samsung SDK (AI Lite Core) and install enn backend
892+
source .ci/scripts/setup-samsung-linux-deps.sh
893+
894+
# Test models serially
895+
models="mv2 ic3 resnet18 resnet50"
896+
for model in $models; do
897+
python -m executorch.examples.samsung.aot_compiler --model_name=$model -c E9955
898+
done
899+
900+
# Test ops
901+
python -m unittest discover -s backends/samsung/test/ops -p "test_*.py"
902+
903+
869904
test-vulkan-models-linux:
870905
name: test-vulkan-models-linux
871906
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main

‎CMakeLists.txt‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -571,6 +571,11 @@ if(EXECUTORCH_BUILD_QNN)
571571
list(APPEND _executorch_backends qnn_executorch_backend)
572572
endif()
573573

574+
if(EXECUTORCH_BUILD_ENN)
575+
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/backends/samsung)
576+
list(APPEND _executorch_backends enn_backend)
577+
endif()
578+
574579
if(EXECUTORCH_BUILD_XNNPACK)
575580
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/backends/xnnpack)
576581
list(APPEND _executorch_backends xnnpack_backend)
@@ -817,6 +822,10 @@ if(EXECUTORCH_BUILD_PYBIND)
817822
list(APPEND _dep_libs qnn_executorch_backend)
818823
endif()
819824

825+
if(EXECUTORCH_BUILD_ENN)
826+
list(APPEND _dep_libs enn_backend)
827+
endif()
828+
820829
if(EXECUTORCH_BUILD_XNNPACK)
821830
# need to explicitly specify XNNPACK and xnnpack-microkernels-prod here
822831
# otherwise uses XNNPACK and microkernel-prod symbols from libtorch_cpu

‎LICENSE‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ Copyright (c) Qualcomm Innovation Center, Inc.
88
Copyright (c) 2023 Apple Inc.
99
Copyright (c) 2024 MediaTek Inc.
1010
Copyright 2023 NXP
11+
Copyright (c) 2025 Samsung Electronics Co. LTD
1112

1213
Redistribution and use in source and binary forms, with or without modification,
1314
are permitted provided that the following conditions are met:

‎backends/samsung/CMakeLists.txt‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# Copyright (c) 2025 Samsung Electronics Co. LTD
2+
# All rights reserved
3+
#
4+
# This source code is licensed under the BSD-style license found in the
5+
# LICENSE file in the root directory of this source tree.
6+
7+
cmake_minimum_required(VERSION 3.15)
8+
set(CMAKE_CXX_STANDARD 17)
9+
set(CMAKE_CXX_STANDARD_REQUIRED ON)
10+
11+
get_filename_component(
12+
EXECUTORCH_SOURCE_DIR "${CMAKE_CURRENT_LIST_DIR}/../.." ABSOLUTE
13+
)
14+
15+
if(NOT DEFINED EXYNOS_AI_LITECORE_ROOT)
16+
message(
17+
FATAL_ERROR
18+
"Please define EXYNOS_AI_LIRECORE_PATH by adding cmake parameter -DEXYNOS_AI_LITECORE_ROOT=<...>"
19+
)
20+
endif()
21+
if(CMAKE_TOOLCHAIN_FILE MATCHES ".*(iOS|ios\.toolchain)\.cmake$")
22+
message(FATAL_ERROR "IOS is not supported on Exynos.")
23+
endif()
24+
25+
if(NOT FLATC_EXECUTABLE)
26+
set(FLATC_EXECUTABLE flatc)
27+
endif()
28+
29+
add_compile_options(-Wall -Werror -fPIC)
30+
if(CMAKE_BUILD_TYPE STREQUAL "Release")
31+
# strip symbols
32+
add_link_options("-s")
33+
# hide dynamic symbols
34+
set(CMAKE_C_VISIBILITY_PRESET hidden)
35+
set(CMAKE_CXX_VISIBILITY_PRESET hidden)
36+
add_definitions(-DNDEBUG)
37+
endif()
38+
39+
include_directories(
40+
${EXECUTORCH_SOURCE_DIR}/..
41+
${EXECUTORCH_SOURCE_DIR}/runtime/core/portable_type/c10
42+
${EXYNOS_AI_LITECORE_ROOT}
43+
)
44+
add_compile_definitions(C10_USING_CUSTOM_GENERATED_MACROS)
45+
46+
if(${ANDROID})
47+
find_library(android_log log)
48+
endif()
49+
50+
# add logging library
51+
add_library(enn_logging STATIC)
52+
53+
if(${CMAKE_SYSTEM_PROCESSOR} MATCHES "x86_64")
54+
add_subdirectory(
55+
${EXECUTORCH_SOURCE_DIR}/third-party/pybind11
56+
${CMAKE_CURRENT_BINARY_DIR}/pybind11
57+
)
58+
add_library(PyEnnWrapperAdaptor MODULE)
59+
60+
find_library(
61+
GG_API_LIB
62+
NAMES graphgen_api
63+
HINTS ${EXYNOS_AI_LITECORE_ROOT}/lib/x86_64-linux
64+
)
65+
add_library(graphgen_api SHARED IMPORTED GLOBAL)
66+
set_target_properties(
67+
graphgen_api
68+
PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
69+
"${EXYNOS_AI_LITECORE_ROOT}/include" IMPORTED_LOCATION
70+
"${GG_API_LIB}"
71+
)
72+
73+
set(_enn_compile_options_schema
74+
${CMAKE_CURRENT_SOURCE_DIR}/serialization/compile_options_def.fbs
75+
)
76+
77+
set(_enn_schema_generate_dir
78+
"${CMAKE_BINARY_DIR}/schema/include/executorch/backends/samsung"
79+
)
80+
# Paths to headers generated from the .fbs files.
81+
string(REGEX REPLACE "serialization/([^/]+)[.]fbs$" "\\1_generated.h"
82+
generated_header "${fbs_file}"
83+
)
84+
set(_enn_schema_output "${_enn_schema_generate_dir}/${generated_header}")
85+
86+
# Generate the headers from the .fbs files.
87+
add_custom_command(
88+
OUTPUT ${_enn_schema_output}
89+
COMMAND ${FLATC_EXECUTABLE} --cpp --cpp-std c++11 --scoped-enums -o
90+
"${_enn_schema_generate_dir}" ${_enn_compile_options_schema}
91+
DEPENDS ${_enn_compile_options_schema}
92+
WORKING_DIRECTORY ${EXECUTORCH_SOURCE_DIR}
93+
COMMENT "Generating enn compile options headers"
94+
VERBATIM
95+
)
96+
add_custom_target(
97+
enn_compile_options_output ALL DEPENDS ${_enn_schema_output}
98+
)
99+
100+
set_target_properties(
101+
PyEnnWrapperAdaptor PROPERTIES CXX_VISIBILITY_PRESET hidden
102+
)
103+
target_link_libraries(
104+
PyEnnWrapperAdaptor PRIVATE pybind11::module pybind11::lto graphgen_api
105+
enn_logging
106+
)
107+
target_include_directories(
108+
PyEnnWrapperAdaptor BEFORE
109+
PRIVATE ${CMAKE_BINARY_DIR}/schema/include
110+
${EXECUTORCH_SOURCE_DIR}/third-party/flatbuffers/include
111+
)
112+
add_dependencies(PyEnnWrapperAdaptor enn_compile_options_output)
113+
pybind11_extension(PyEnnWrapperAdaptor)
114+
115+
# PyGraphWrapperAdaptor
116+
add_library(PyGraphWrapperAdaptor MODULE)
117+
#
118+
find_library(
119+
GRAPH_WRAPPER_LIB
120+
NAMES graph_wrapper
121+
HINTS ${EXYNOS_AI_LITECORE_ROOT}/lib/x86_64-linux
122+
)
123+
add_library(graph_wrapper SHARED IMPORTED GLOBAL)
124+
set_target_properties(
125+
graph_wrapper
126+
PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
127+
"${EXYNOS_AI_LITECORE_ROOT}/include" IMPORTED_LOCATION
128+
"${GRAPH_WRAPPER_LIB}"
129+
)
130+
set_target_properties(
131+
PyGraphWrapperAdaptor PROPERTIES CXX_VISIBILITY_PRESET hidden
132+
)
133+
target_link_libraries(
134+
PyGraphWrapperAdaptor PRIVATE pybind11::module pybind11::lto graph_wrapper
135+
enn_logging
136+
)
137+
pybind11_extension(PyGraphWrapperAdaptor)
138+
139+
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/aot)
140+
endif()
141+
142+
if(${ANDROID})
143+
target_link_libraries(enn_logging PRIVATE ${android_log})
144+
add_library(enn_backend STATIC)
145+
target_link_libraries(enn_backend PRIVATE enn_logging)
146+
executorch_target_link_options_shared_lib(enn_backend)
147+
target_compile_options(enn_backend PRIVATE -Wno-deprecated-declarations)
148+
149+
set(__enn_executor_runner_srcs
150+
${EXECUTORCH_SOURCE_DIR}/examples/samsung/executor_runner/enn_executor_runner.cpp
151+
)
152+
add_executable(enn_executor_runner ${__enn_executor_runner_srcs})
153+
add_dependencies(enn_executor_runner enn_backend)
154+
target_link_libraries(
155+
enn_executor_runner PRIVATE enn_logging enn_backend gflags executorch
156+
extension_data_loader portable_ops_lib
157+
)
158+
set_target_properties(
159+
enn_executor_runner PROPERTIES CXX_VISIBILITY_PRESET hidden
160+
)
161+
install(
162+
TARGETS enn_backend enn_logging
163+
EXPORT ExecuTorchTargets
164+
DESTINATION lib
165+
)
166+
endif()
167+
168+
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/runtime)

‎backends/samsung/README.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# ExecuTorch Samsung Exynos Delegate
2+
3+
The subtree contains Exynos delegate implementation for ExecuTorch. The target of delegation
4+
is deploying torch model to run with exynos NPU/DSP.
5+
6+
This backend is implemented on the top of [EXYNOS_LITECORE](https://soc-developer.semiconductor.samsung.com/global/development/ai-litecore)
7+
Please prepare the SDK before you start, it is important to code compilation and runtime.
8+
9+
## Delegate Options
10+
11+
### Supported Chipset
12+
- Exynos 2500 (E9955)
13+
14+
### Supported Inference Type
15+
- Quantized (i8/u8/i16/u16)
16+
- FP16
17+
18+
## Directory Structure
19+
20+
```
21+
backends/samsung
22+
├── aot # Codes for generating binary buffer for ENN runtime.
23+
├── builders # Codes for lowering each operators.
24+
├── partition # ENN Partitioner.
25+
├── passes # Various passes helping lower models to ENN backend.
26+
├── python # Places to put pybind artifacts for accessing samsung libraries.
27+
├── runtime # ENN runtime for executing lowered models.
28+
├── scripts # Misc supporting scripts, not related to core functionality.
29+
└── serialization # Codes for building Graph IR for Exynos and serializing.
30+
31+
examples
32+
└── samsung # Examples to run ENN backends.
33+
```
34+
35+
## How to build
36+
Please download Exynos AI LiteCore, and set the root path of SDK directory to `EXYNOS_AI_LITECORE_ROOT`.</br>
37+
Please navigate to [Android NDK](https://developer.android.com/ndk) and download a version of NDK.
38+
`ANDROID_NDK` refers the root path of NDK directory.</br>
39+
40+
### Set up environment variables
41+
```bash
42+
export LD_LIBRARY_PATH=${EXYNOS_AI_LITECORE_ROOT}/lib/x86_64-linux/
43+
```
44+
45+
### Build AOT Targets
46+
Generates python artifacts that allow user call `Compile` interface to lower a model to Exynos backend in python script.
47+
```bash
48+
./backends/samsung/build.sh -b x86_64
49+
```
50+
51+
### Build ENN Executor Runner
52+
```bash
53+
./backends/samsung/build.sh -b android --ndk ${ANDROID_NDK}
54+
```
55+
ANDROID_ABI=arm64-v8a is default, necessary runtime executable generated in `build_exynos_android` directory.
56+
57+
### Build Anroid Extension
58+
This is later exposed Java app. Please turn on CMake option `EXECUTORCH_BUILD_ENN`, and ENN runtime will be added.
59+
```bash
60+
cmake extension/android \
61+
-DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \
62+
-DANDROID_ABI="${ANDROID_ABI}" \
63+
-DCMAKE_INSTALL_PREFIX=cmake-android-out \
64+
-Bcmake-android-out/extension/android
65+
66+
cmake --build cmake-android-out/extension/android -j8
67+
```
68+
69+
## Examples
70+
python -m executorch.examples.samsung.aot_compiler --chipset e9955 -m ic3
71+
72+
Please see this [README.md](../../examples/samsung/README.md).

0 commit comments

Comments
 (0)