You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/demo-apps/android/LlamaDemo/docs/delegates/mediatek_README.md
+20-19Lines changed: 20 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -95,43 +95,44 @@ Note: Exporting model flow can take 2.5 hours (114GB RAM for num_chunks=4) to co
95
95
96
96
Before continuing forward, make sure to modify the tokenizer, token embedding, and model paths in the examples/mediatek/executor_runner/run_llama3_sample.sh.
97
97
98
+
### Deploy
99
+
First, make sure your Android phone’s chipset version is compatible with this demo (MediaTek Dimensity 9300 (D9300)) chip. Once you have the model, tokenizer, and runner generated ready, you can push them and the .so files to the device before we start running using the runner via shell.
100
+
101
+
```
102
+
adb shell mkdir -p /data/local/tmp/et-mtk/ (or any other directory name)
The Mediatek runner (`examples/mediatek/executor_runner/mtk_llama_runner.cpp`)) contains the logic for implementing the function calls that come from the Android app.
117
+
The Mediatek runner (`examples/mediatek/executor_runner/mtk_llama_runner.cpp`) contains the logic for implementing the function calls that come from the Android app.
102
118
103
119
**Important**: Currently the model paths are set in the runner-level. Modify the values in `examples/mediatek/executor_runner/llama_runner/llm_helper/include/llama_runner_values.h` to set the model paths, tokenizer path, embedding file path, and other metadata.
104
120
105
121
106
122
## Build AAR Library
107
123
108
-
Next we need to build and compile the MediaTek backend and MediaTek Llama runner.
124
+
Next we need to build and compile the MediaTek backend and MediaTek Llama runner. By setting `NEURON_BUFFER_ALLOCATOR_LIB`, the script will build the MediaTek backend.
109
125
```
110
126
sh build/build_android_llm_demo.sh
111
127
```
112
128
113
-
**Output**: This will generate a .aar file is already imported into the expected directory for the Android app. It lives in `examples/demo-apps/android/Llamademo/app/libs`.
129
+
**Output**: This will generate an .aar file that is already imported into the expected directory for the Android app. It will live in `examples/demo-apps/android/Llamademo/app/libs`.
114
130
115
131
If you were to unzip the .aar file or open it in Android Studio, you can see that it contains the following related to MediaTek backend:
116
132
* libneuron_buffer_allocator.so
117
133
* libneuronusdk_adapter.mtk.so
118
134
* libneuron_backend.so (generated during build)
119
135
120
-
### Deploy
121
-
First, make sure your Android phone’s chipset version is compatible with this demo (MediaTek Dimensity 9300 (D9300)) chip. Once you have the model, tokenizer, and runner generated ready, you can push them and the .so files to the device before we start running using the runner via shell.
0 commit comments