Skip to content

Commit bdee603

Browse files
committed
minor changes and reordering
1 parent 44f3c40 commit bdee603

File tree

1 file changed

+20
-19
lines changed

1 file changed

+20
-19
lines changed

examples/demo-apps/android/LlamaDemo/docs/delegates/mediatek_README.md

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -95,43 +95,44 @@ Note: Exporting model flow can take 2.5 hours (114GB RAM for num_chunks=4) to co
9595

9696
Before continuing forward, make sure to modify the tokenizer, token embedding, and model paths in the examples/mediatek/executor_runner/run_llama3_sample.sh.
9797

98+
### Deploy
99+
First, make sure your Android phone’s chipset version is compatible with this demo (MediaTek Dimensity 9300 (D9300)) chip. Once you have the model, tokenizer, and runner generated ready, you can push them and the .so files to the device before we start running using the runner via shell.
100+
101+
```
102+
adb shell mkdir -p /data/local/tmp/et-mtk/ (or any other directory name)
103+
adb push embedding_<model_name>_fp32.bin /data/local/tmp/et-mtk
104+
adb push tokenizer.model /data/local/tmp/et-mtk
105+
adb push <exported_prompt_model_0>.pte /data/local/tmp/et-mtk
106+
adb push <exported_prompt_model_1>.pte /data/local/tmp/et-mtk
107+
...
108+
adb push <exported_prompt_model_n>.pte /data/local/tmp/et-mtk
109+
adb push <exported_gen_model_0>.pte /data/local/tmp/et-mtk
110+
adb push <exported_gen_model_1>.pte /data/local/tmp/et-mtk
111+
...
112+
adb push <exported_gen_model_n>.pte /data/local/tmp/et-mtk
113+
```
114+
98115
## Populate Model Paths in Runner
99116

100-
### Populate Model Paths in Runner
101-
The Mediatek runner (`examples/mediatek/executor_runner/mtk_llama_runner.cpp`)) contains the logic for implementing the function calls that come from the Android app.
117+
The Mediatek runner (`examples/mediatek/executor_runner/mtk_llama_runner.cpp`) contains the logic for implementing the function calls that come from the Android app.
102118

103119
**Important**: Currently the model paths are set in the runner-level. Modify the values in `examples/mediatek/executor_runner/llama_runner/llm_helper/include/llama_runner_values.h` to set the model paths, tokenizer path, embedding file path, and other metadata.
104120

105121

106122
## Build AAR Library
107123

108-
Next we need to build and compile the MediaTek backend and MediaTek Llama runner.
124+
Next we need to build and compile the MediaTek backend and MediaTek Llama runner. By setting `NEURON_BUFFER_ALLOCATOR_LIB`, the script will build the MediaTek backend.
109125
```
110126
sh build/build_android_llm_demo.sh
111127
```
112128

113-
**Output**: This will generate a .aar file is already imported into the expected directory for the Android app. It lives in `examples/demo-apps/android/Llamademo/app/libs`.
129+
**Output**: This will generate an .aar file that is already imported into the expected directory for the Android app. It will live in `examples/demo-apps/android/Llamademo/app/libs`.
114130

115131
If you were to unzip the .aar file or open it in Android Studio, you can see that it contains the following related to MediaTek backend:
116132
* libneuron_buffer_allocator.so
117133
* libneuronusdk_adapter.mtk.so
118134
* libneuron_backend.so (generated during build)
119135

120-
### Deploy
121-
First, make sure your Android phone’s chipset version is compatible with this demo (MediaTek Dimensity 9300 (D9300)) chip. Once you have the model, tokenizer, and runner generated ready, you can push them and the .so files to the device before we start running using the runner via shell.
122-
123-
```
124-
adb shell mkdir -p /data/local/tmp/llama
125-
adb push examples/mediatek/executor_runner/run_llama3_sample.sh /data/local/tmp/llama
126-
adb push sample_prompt.txt /data/local/tmp/llama
127-
adb push cmake-android-out/examples/mediatek/mtk_llama_executor_runner /data/local/tmp/llama
128-
adb push cmake-android-out/backends/mediatek/libneuron_backend.so /data/local/tmp/llama
129-
adb push libneuron_buffer_allocator.so /data/local/tmp/llama
130-
adb push libneuronusdk_adapter.mtk.so /data/local/tmp/llama
131-
adb push embedding_<model_name>_fp32.bin /data/local/tmp/llama
132-
adb push tokenizer.model /data/local/tmp/llama
133-
```
134-
135136
## Run Demo
136137

137138
### Alternative 1: Android Studio (Recommended)

0 commit comments

Comments
 (0)