diff --git a/backends/mediatek/scripts/README.md b/backends/mediatek/README.md similarity index 70% rename from backends/mediatek/scripts/README.md rename to backends/mediatek/README.md index 0550fb23d51..5d478da3f00 100644 --- a/backends/mediatek/scripts/README.md +++ b/backends/mediatek/README.md @@ -1,12 +1,19 @@ -# Build Instructions +# MediaTek Backend on ExecuTorch +MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices that equips with MediaTek Neuron Processing Unit (NPU). This document offers a step-by-step guide to set up the build environment for the MediaTek ExecuTorch libraries. -This document provides a step-by-step guide to set up the build environment for the MediaTek ExercuTorch libraries. +## Supported Chips -## Prerequisites +The examples provided in this repository are tested and supported on the following MediaTek chip: + +- MediaTek Dimensity 9300 (D9300) + +## Build Instructions + +### Prerequisites Before you begin, ensure you have the following prerequisites installed and configured: -### 1. Buck2 Build Tool +#### 1. Buck2 Build Tool - **Download Buck2**: Obtain Buck2 from the official [releases page](https://github.com/facebook/buck2/releases/tag/2024-02-01). - **Add to PATH**: Extract the downloaded file and add the directory to your system's `$PATH` environment variable. @@ -14,7 +21,7 @@ Before you begin, ensure you have the following prerequisites installed and conf export PATH=:$PATH ``` -### 2. Android NDK +#### 2. Android NDK - **Download Android NDK**: Acquire the Android NDK version 26.3.11579264 from the [Android developer site](https://developer.android.com/ndk/downloads). - **Set NDK Path**: Ensure that the `$ANDROID_NDK` environment variable is set to the path where the NDK is located. @@ -22,7 +29,7 @@ Before you begin, ensure you have the following prerequisites installed and conf export ANDROID_NDK= ``` -### 3. MediaTek ExercuTorch Libraries +#### 3. MediaTek ExecuTorch Libraries Download [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress) from MediaTek's NeuroPilot portal: @@ -31,11 +38,11 @@ Download [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/publ - `mtk_converter-8.8.0.dev20240723+public.d1467db9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`: This library preprocess the model into a MediaTek representation. - `mtk_neuron-8.2.2-py3-none-linux_x86_64.whl`: This library converts the model to binaries. -## Setup +### Setup Follow the steps below to setup your build environment: -1. **Setup ExercuTorch Environment**: Refer to the [Setting up ExercuTorch](https://pytorch.org/executorch/stable/getting-started-setup) guide for detailed instructions on setting up the ExercuTorch environment. +1. **Setup ExecuTorch Environment**: Refer to the [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) guide for detailed instructions on setting up the ExecuTorch environment. 2. **Setup MediaTek Backend Environment** - Install the dependent libs. Ensure that you are inside backends/mediatek/ directory @@ -52,18 +59,21 @@ Follow the steps below to setup your build environment: export NEURON_BUFFER_ALLOCATOR_LIB= ``` -## Build +### Build +1. Navigate to `scripts/` directory. -1. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process, MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so` +2. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process, MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so` ```bash ./mtk_build.sh ``` -## Run +### Run -1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExercuTorch with MediaTek backend. +1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExecuTorch with MediaTek backend. ```bash export LD_LIBRARY_PATH=::$LD_LIBRARY_PATH ``` + +Please refer to `executorch/examples/mediatek/` for export and execution examples of various of models. \ No newline at end of file diff --git a/docs/source/build-run-mediatek-backend.md b/docs/source/build-run-mediatek-backend.md new file mode 100644 index 00000000000..eeaa2b8dc88 --- /dev/null +++ b/docs/source/build-run-mediatek-backend.md @@ -0,0 +1,94 @@ +# Building and Running ExecuTorch with MediaTek Backend + +MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices that equips with MediaTek Neuron Processing Unit (NPU). This document offers a step-by-step guide to set up the build environment for the MediaTek ExecuTorch libraries. + +::::{grid} 2 +:::{grid-item-card} What you will learn in this tutorial: +:class-card: card-prerequisites +* How to export and lower a PyTorch model ahead of time with ExecuTorch for MediaTek devices. +* How to build MediaTek backend and examples. +* How to deploy the exported models on device with ExecuTorch runtime. +::: +:::{grid-item-card} Tutorials we recommend you complete before this: +:class-card: card-prerequisites +* [Introduction to ExecuTorch](intro-how-it-works.md) +* [Setting up ExecuTorch](getting-started-setup.md) +* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) +::: +:::: + + +## Prerequisites (Hardware and Software) + +### Host OS +- Linux operating system + +### Supported Chips: +- MediaTek Dimensity 9300 (D9300) + +### Software: + +- [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress) is a lightweight SDK for deploying AI applications on MediaTek SOC devices. + +## Setting up your developer environment + +Follow the steps below to setup your build environment: + +1. **Setup ExecuTorch Environment**: Refer to the [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) guide for detailed instructions on setting up the ExecuTorch environment. + +2. **Setup MediaTek Backend Environment** +- Install the dependent libs. Ensure that you are inside `backends/mediatek/` directory + ```bash + pip3 install -r requirements.txt + ``` +- Install the two .whl downloaded from NeuroPilot Portal + ```bash + pip3 install mtk_neuron-8.2.13-py3-none-linux_x86_64.whl + pip3 install mtk_converter-8.9.1+public-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl + ``` +- Set evironment variables for building backend + ```bash + export NEURON_BUFFER_ALLOCATOR_LIB= + ``` + +## Build + +### Ahead of time: + +**Exporting a PyTorch Model for MediaTek Backend**: +1. Lower and export the `.pte` file for on-device execution. The export script samples are povided under `example/mediatek/`. For example, the following commnad exports the `.pte` using the scripts provided. +```bash +cd executorch + +./examples/mediatek/shell_scripts/export_oss.sh mobilenetv3 +``` + +2. Find the `.pte` files under the directory named as same as the model. + +### Runtime: + +**Build MediaTek Backend for ExecuTorch Runtime** +1. Navigate to `backends/mediatek/scripts/` directory. + +2. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process: + ```bash + ./mtk_build.sh + ``` + +3. MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so`. + +**Build a runner to execute the model on the device**: +1. Build the runners and the backend by exedcuting the script: +```bash +./mtk_build_examples.sh +``` + +2. The runners will be built under `cmake-android-out/examples/` + +## Deploying and running on a device + +1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExecuTorch with MediaTek backend. + + ```bash + export LD_LIBRARY_PATH=::$LD_LIBRARY_PATH + ``` \ No newline at end of file diff --git a/docs/source/index.rst b/docs/source/index.rst index 0f8ebedd9f3..d79de11d5a8 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -111,6 +111,7 @@ Topics in this section will help you get started with ExecuTorch. customcarditem entries below. executorch-arm-delegate-tutorial build-run-coreml + build-run-mediatek-backend build-run-mps build-run-qualcomm-ai-engine-direct-backend build-run-xtensa @@ -331,6 +332,13 @@ ExecuTorch tutorials. :link: build-run-coreml.html :tags: Export,Backend,Delegation,CoreML +.. customcarditem:: + :header: Building and Running ExecuTorch with MediaTek Backend + :card_description: A tutorial that walks you through the process of building ExecuTorch with MediaTek Backend + :image: _static/img/generic-pytorch-logo.png + :link: build-run-mediatek-backend.html + :tags: Export,Backend,Delegation,MediaTek + .. customcarditem:: :header: Building and Running ExecuTorch with MPS Backend :card_description: A tutorial that walks you through the process of building ExecuTorch with MPSGraph Backend diff --git a/examples/mediatek/README.md b/examples/mediatek/README.md index 17b73ce9372..c63a522ffcc 100644 --- a/examples/mediatek/README.md +++ b/examples/mediatek/README.md @@ -72,12 +72,6 @@ bash shell_scripts/export_oss.sh - `model_name`: deeplabv3/edsr/inceptionv3/inceptionv4/mobilenetv2/mobilenetv3/resnet18/resnet50 # Runtime -## Supported Chips - -The examples provided in this repository are tested and supported on the following MediaTek chip: - -- MediaTek Dimensity 9300 (D9300) - ## Environment Setup To set up the build environment for the `mtk_executor_runner`: