Skip to content

Commit 112233e

Browse files
thchenqtithchenqti
authored
Qualcomm AI Engine Direct - Add&update README (#13504)
### Summary - add README for executorch/examples/qualcomm/oss_scripts - add README for executorch/examples/qualcomm/scripts - introduce our official API functions in executorch/examples/qualcomm/README.md - add argument -p in efficientSAM.py ### Test plan N/A
1 parent 4c4cbdf commit 112233e

File tree

4 files changed

+242
-6
lines changed

4 files changed

+242
-6
lines changed

‎examples/qualcomm/README.md‎

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,17 @@
22

33
This directory contains examples for some AI models.
44

5-
We have seperated the example scripts into the following subfolders, please refer to [README.md](../../backends/qualcomm/README.md) for the example scripts' directory structure:
5+
We have separated the example scripts into the following subfolders, please refer to [README.md](../../backends/qualcomm/README.md) for the example scripts' directory structure:
66

77
1. executor_runner: This folder contains a general executor runner capable of running most of the models. As a rule of thumb, if a model does not have its own customized runner, execute the model using [executor_runner](executor_runner/qnn_executor_runner.cpp). On the other hand, if a model has its own runner, such as [llama](oss_scripts/llama/qnn_llama_runner.cpp), use the customized runner to execute the model. Customized runner should be located under the same folder as the model's python script.
88

99
2. oss_scripts: OSS stands for Open Source Software. This folder contains python scripts for open source models. Some models under this folder might also have their own customized runner.
1010
For example, [llama](oss_scripts/llama/qnn_llama_runner.cpp) contains not only the python scripts to prepare the model but also a customized runner for executing the model.
1111

12-
3. qaihub_scripts: QAIHub stands for [Qualcomm AI Hub](https://aihub.qualcomm.com/). On QAIHub, users can find pre-compiled context binaries, a format used by QNN to save its models. This provides users with a new option for model deployment. Different from oss_scripts & scripts, which the example scripts are converting a model from nn.Module to ExecuTorch .pte files, qaihub_scripts provides example scripts for converting pre-compiled context binaries to ExecuTorch .pte files. Additionaly, users can find customized example runners specific to the QAIHub models for execution. For example [qaihub_llama2_7b](qaihub_scripts/llama/llama2/qaihub_llama2_7b.py) is a script converting context binaries to ExecuTorch .pte files, and [qaihub_llama2_7b_runner](qaihub_scripts/llama/llama2/qaihub_llama2_7b_runner.cpp) is a customized example runner to execute llama2 .pte files. Please be aware that context-binaries downloaded from QAIHub are tied to a specific QNN SDK version.
12+
3. qaihub_scripts: QAIHub stands for [Qualcomm AI Hub](https://aihub.qualcomm.com/). On QAIHub, users can find pre-compiled context binaries, a format used by QNN to save its models. This provides users with a new option for model deployment. Different from oss_scripts & scripts, which the example scripts are converting a model from nn.Module to ExecuTorch .pte files, qaihub_scripts provides example scripts for converting pre-compiled context binaries to ExecuTorch .pte files. Additionally, users can find customized example runners specific to the QAIHub models for execution. For example [qaihub_llama2_7b](qaihub_scripts/llama/llama2/qaihub_llama2_7b.py) is a script converting context binaries to ExecuTorch .pte files, and [qaihub_llama2_7b_runner](qaihub_scripts/llama/llama2/qaihub_llama2_7b_runner.cpp) is a customized example runner to execute llama2 .pte files. Please be aware that context-binaries downloaded from QAIHub are tied to a specific QNN SDK version.
1313
Before executing the scripts and runner, please ensure that you are using the QNN SDK version that is matching the context binary. Please refer to [Check context binary version](#check-context-binary-version) for tutorial on how to check the QNN Version for a context binary.
1414

15-
4. scripts: This folder contains scripts to build models provided by Executorch.
15+
4. scripts: This folder contains scripts to build models provided by ExecuTorch.
1616

1717

1818

@@ -22,7 +22,7 @@ Here are some general information and limitations.
2222

2323
## Prerequisite
2424

25-
Please finish tutorial [Setting up executorch](https://pytorch.org/executorch/main/getting-started-setup).
25+
Please finish tutorial [Setting up ExecuTorch](https://pytorch.org/executorch/main/getting-started-setup).
2626

2727
Please finish [setup QNN backend](../../docs/source/backends-qualcomm.md).
2828

@@ -44,7 +44,7 @@ commands in the `SimpleADB` class inside [utils.py](utils.py).
4444

4545
## Please use python xxx.py --help for information of each examples.
4646

47-
Some CLI examples here. Please adjust according to your environment. If you want to export the model without running it, please add `-compile_only` to the command.:
47+
Some CLI examples here. Please adjust according to your environment. If you want to export the model without running it, please add `--compile_only` to the command.:
4848

4949
#### First switch to following folder
5050
```bash
@@ -83,6 +83,32 @@ If you run into the following error, that means the ${QNN_SDK_ROOT} that you are
8383
```
8484
Error: Failed to get context binary info.
8585
```
86+
## Model Structure
87+
This section outlines the essential APIs and utilities provided to streamline the process of model conversion, deployment, and evaluation on Qualcomm hardware using ExecuTorch.
88+
89+
1. `build_executorch_binary()`:
90+
91+
build_executorch_binary is a high-level API used to convert a PyTorch model into a Qualcomm-compatible .pte binary format. This function streamlines the process of quantization, transformation, optimization, and export, enabling users to efficiently deploy models on Qualcomm hardware.
92+
93+
2. `SimpleADB`:
94+
95+
SimpleADB is a Python class that provides a simplified interface for interacting with Android devices. It allows users to execute ADB commands, retrieve device information, and manage files on the device.
96+
97+
3. `get_imagenet_dataset`:
98+
99+
If the model requires ImageNet, this function can be used to load the dataset and apply the necessary preprocessing steps to prepare it for inference or quantization calibration.
100+
101+
4. `topk_accuracy`:
102+
103+
Calculates the Top-K accuracy for classification models, used to evaluate model performance.
104+
105+
5. `parse_skip_delegation_node`:
106+
107+
Parses command-line arguments to identify node IDs or operation types that should be skipped during model conversion.
108+
109+
6. `make_output_dir`:
110+
111+
Creates a clean directory for storing model outputs or intermediate results. If the directory already exists, it will be deleted and recreated to ensure a consistent environment for each run.
86112

87113
## Additional Dependency
88114

@@ -95,7 +121,7 @@ pip install scikit-learn pandas
95121

96122
## Limitation
97123

98-
1. QNN 2.24 is used for all examples. Newer or older QNN might work,
124+
1. QNN 2.28 is used for all examples. Newer or older QNN might work,
99125
but the performance and accuracy number can differ.
100126

101127
2. The mobilebert example is on QNN HTP fp16, which is only supported by a limited

‎examples/qualcomm/oss_scripts/README.md‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# Usage Guide for Models Provided by ExecuTorch
2+
3+
This guide provides examples and instructions for open source models. Some models under this folder might also have their own customized runner.
4+
5+
## Model categories
6+
The following models can be categorized based on their primary use cases.
7+
8+
1. Language Model:
9+
- albert
10+
- bert
11+
- distilbert
12+
- eurobert
13+
- llama
14+
- roberta
15+
16+
2. Vision Model:
17+
- conv_former
18+
- cvt
19+
- deit
20+
- dino_v2
21+
- dit
22+
- efficientnet
23+
- efficientSAM
24+
- esrgan
25+
- fastvit
26+
- fbnet
27+
- focalnet
28+
- gMLP_image_classification
29+
- mobilevit1
30+
- mobilevit_v2
31+
- pvt
32+
- regnet
33+
- retinanet
34+
- squeezenet
35+
- ssd300_vgg16
36+
- swin_transformer
37+
38+
## Prerequisite
39+
Please follow another [README](../README.md) first to set up environment.
40+
41+
## Model running
42+
Some models require specific datasets. Please download them in advance and place them in the appropriate folders.
43+
44+
Detailed instructions for each model are provided below.
45+
If you want to export the model without running it, please add `--compile_only` to the command.
46+
47+
1. `albert`,`bert`,`distilbert`, `eurobert`, `roberta`:
48+
- Required Dataset : wikisent2
49+
50+
download [dataset](https://www.kaggle.com/datasets/mikeortman/wikipedia-sentences) first, and place it in a valid folder.
51+
```bash
52+
python albert.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} -d path/to/wikisent2
53+
54+
2. `conv_former`,`cvt`,`deit`,`dino_v2`,`efficientnet`,`fbnet`, `focalnet`, `gMLP_image_classification`, `mobilevit1`,`mobilevit_v2`, `pvt`, `squeezenet`, `swin_transformer` :
55+
- Required Dataset : ImageNet
56+
57+
Download [dataset](https://www.kaggle.com/datasets/ifigotin/imagenetmini-1000) first, and place it in a valid folder.
58+
```bash
59+
python SCRIPT_NAME.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} -d path/to/ImageNet
60+
61+
3. `dit`:
62+
63+
```bash
64+
python dit.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL}
65+
4. `esrgan`:
66+
- Required Dataset: B100
67+
68+
Will be downloaded automatically if -d is specified. Alternatively, you can provide your own dataset using `--hr_ref_dir` and `--lr_dir`.
69+
70+
- Required OSS Repo: Real-ESRGAN
71+
72+
Clone [OSS Repo](https://github.com/ai-forever/Real-ESRGAN) first, and place it in a valid folder.
73+
74+
```bash
75+
python esrgan.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} --oss_repo path/to/Real-ESRGAN
76+
77+
5. `fastvit`:
78+
- Required Dataset: ImageNet
79+
80+
Download [dataset](https://www.kaggle.com/datasets/ifigotin/imagenetmini-1000) first, and place it in a valid folder.
81+
82+
- Required OSS Repo: ml-fastvit
83+
84+
Clone [OSS Repo](https://github.com/apple/ml-fastvit) first, and place it in a valid folder.
85+
86+
- Pretrained weight:
87+
88+
Download [pretrained weight](https://docs-assets.developer.apple.com/ml-research/models/fastvit/image_classification_distilled_models/fastvit_s12_reparam.pth.tar) first, and place it in a valid folder(should be fastvit_s12_reparam.pth.tar).
89+
```bash
90+
python fastvit.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} --oss_repo path/to/ml-fastvit -p path/to/pretrained_weight -d path/to/ImageNet
91+
92+
6. `regnet`:
93+
- Required Dataset: ImageNet
94+
95+
Download [dataset](https://www.kaggle.com/datasets/ifigotin/imagenetmini-1000) first, and place it in a valid folder.
96+
- Weights: regnet_y_400mf, regnet_x_400mf
97+
98+
use `--weights` to specify which regent weights/model to execute.
99+
```bash
100+
python regnet.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} -d path/to/ImageNet --weights <WEIGHTS>
101+
102+
7. `retinanet`:
103+
- Required Dataset: COCO
104+
105+
Download [val2017](http://images.cocodataset.org/zips/val2017.zip) and [annotations](http://images.cocodataset.org/annotations/annotations_trainval2017.zip) first, and place it in a valid folder.
106+
107+
```bash
108+
python retinanet.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} -d path/to/PATH/TO/COCO #(which contains 'val_2017' & 'annotations')
109+
110+
8. `ssd300_vgg16`:
111+
- Required OSS Repo:
112+
113+
Clone [OSS Repo](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Object-Detection) first, and place it in a valid folder.
114+
115+
- Pretrained weight:
116+
117+
Download [pretrained weight](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Object-Detection) first, and place it in a valid folder.(checkpoint_ssd300.pth.tar)
118+
119+
- Required Dataset: VOCSegmentation
120+
download [VOC 2007](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Object-Detection?tab=readme-ov-file#download) first, and place it in a valid folder.
121+
```bash
122+
python ssd300_vgg16.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} --oss_repo path/to/a-PyTorch-Tutorial-to-Object-Detection -p path/to/pretrained_weight
123+
124+
9. `llama`:
125+
For llama, please check [README](llama/README.md) under llama folder for more details.
126+
127+
10. `efficientSAM`:
128+
For efficientSAM, please get access to efficientSAM folder.
129+
- Pretrained weight:
130+
131+
Download [EfficientSAM-S](https://github.com/yformer/EfficientSAM/blob/main/weights/efficient_sam_vits.pt.zip) or [EfficientSAM-Ti](https://github.com/yformer/EfficientSAM/blob/main/weights/efficient_sam_vitt.pt) first, and place it in a valid folder.
132+
- Required Dataset: ImageNet
133+
134+
Download [dataset](https://www.kaggle.com/datasets/ifigotin/imagenetmini-1000) first, and place it in a valid folder.
135+
136+
- Required OSS Repo:
137+
138+
Clone [OSS Repo](https://github.com/yformer/EfficientSAM) first, and place it in a valid folder.
139+
```bash
140+
python efficientSAM.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} --oss_repo path/to/EfficientSAM -p path/to/pretrained_weight -d path/to/ImageNet
141+

‎examples/qualcomm/oss_scripts/efficientSAM/efficientSAM.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,7 @@ def post_process():
311311
)
312312

313313
parser.add_argument(
314+
"-p",
314315
"--pretrained_weight",
315316
help="Path to ESAM checkpoint, such as ./efficient_sam_vitt.pt or ./efficient_sam_vits.pt.zip",
316317
type=str,

‎examples/qualcomm/scripts/README.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Usage Guide for Models Provided by ExecuTorch
2+
3+
This guide provides examples and instructions for deploying and executing models using the ExecuTorch runtime on Qualcomm platforms.
4+
5+
## Model categories
6+
The following models in two folders can be categorized based on their primary use cases.
7+
8+
1. Language Model:
9+
- mobilebert_fine_tune
10+
11+
2. Speech Model:
12+
- wav2letter
13+
14+
3. Vision Model:
15+
- edsr
16+
- inception_v3
17+
- inception_v4
18+
- mobilenet_v2
19+
- mobilenet_v3
20+
- torchvision_vit
21+
22+
## Prerequisite
23+
Please follow another [README](../README.md) first to set up environment.
24+
25+
## Model running
26+
Some models require specific datasets. Please download them in advance and place them in the appropriate folders.
27+
28+
Detailed instructions for each model are provided below.
29+
If you want to export the model without running it, please add `--compile_only` to the command.
30+
31+
1. `deeplab_v3`:
32+
- Required Dataset : VOCSegmentation
33+
Will be downloaded automatically if `-d` is specified
34+
35+
```bash
36+
python deeplab_v3.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL}
37+
38+
2. `edsr`:
39+
- Required Dataset : DIV2K
40+
41+
Will be downloaded automatically if -d is specified. Alternatively, you can provide your own dataset using `--hr_ref_dir` and `--lr_dir`.
42+
43+
```bash
44+
pip install piq
45+
python edsr.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} -d
46+
47+
3. `inception_v3`, `inception_v4`, `mobilenet_v2`,`mobilenet_v3`, `torchvision_vit`:
48+
- Required Dataset : ImageNet
49+
50+
Download [dataset](https://www.kaggle.com/datasets/ifigotin/imagenetmini-1000) first, and place it in a valid folder.
51+
52+
```bash
53+
python SCRIPT_NAME.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} -d path/to/ImageNet
54+
55+
4. `mobilebert_fine_tune`:
56+
- You can specify the pretrained weight using `-p <path/to/pretrained_weight>`, if no pretrained weights are provided, using `--num_epochs` to set number of epochs to train the model.
57+
- `-F --use_fp16`: If specified, the model will run in FP16 mode and the PTQ will be ignored.
58+
59+
```bash
60+
python mobilebert_fine_tune.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} --num_epochs <number_of_epochs>
61+
62+
5. `wav2letter`:
63+
- Pretrained weight :
64+
65+
for torchaudio.models.Wav2Letter version, please download at [here](https://github.com/nipponjo/wav2letter-ctc-pytorch/tree/main?tab=readme-ov-file#wav2letter-ctc-pytorch), and place it in a valid folder.
66+
```bash
67+
python wav2letter.py -m ${SOC_MODEL} -b path/to/build-android/ -s ${DEVICE_SERIAL} -p path/to/pretrained_weight
68+

0 commit comments

Comments
 (0)