You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/openvino/README.md
+4-49Lines changed: 4 additions & 49 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,10 @@ Below is the layout of the `examples/openvino` directory, which includes the nec
9
9
```
10
10
examples/openvino
11
11
├── README.md # Documentation for examples (this file)
12
-
└── aot_optimize_and_infer.py # Example script to export and execute models
12
+
├── aot_optimize_and_infer.py # Example script to export and execute models
13
+
└── llama
14
+
├── README.md # Documentation for Llama example
15
+
└── llama3_2_ov_4wo.yaml # Configuration file for exporting Llama3.2 with OpenVINO backend
13
16
```
14
17
15
18
# Build Instructions for Examples
@@ -183,51 +186,3 @@ Run inference with a given model for 10 iterations:
183
186
--model_path=model.pte \
184
187
--num_executions=10
185
188
```
186
-
187
-
# Export Llama with OpenVINO Backend
188
-
189
-
## Download the Model
190
-
Follow the [instructions](../../examples/models/llama#step-2-prepare-model) to download the required model files. Export Llama with OpenVINO backend is only verified with Llama-3.2-1B variants at this time.
191
-
192
-
## Environment Setup
193
-
Follow the [instructions](../../backends/openvino/README.md) of **Prerequisites** and **Setup** in `backends/openvino/README.md` to set up the OpenVINO backend.
194
-
195
-
## Export the model:
196
-
Execute the commands below to export the model. Update the model file paths to match the location where your model is downloaded.
First, build the backend libraries by executing the script below in `<executorch_root>/backends/openvino/scripts` folder:
220
-
```bash
221
-
./openvino_build.sh
222
-
```
223
-
Then, build the llama runner by executing the script below (with `--llama_runner` argument) also in `<executorch_root>/backends/openvino/scripts` folder:
224
-
```bash
225
-
./openvino_build.sh --llama_runner
226
-
```
227
-
The executable is saved in `<executorch_root>/cmake-out/examples/models/llama/llama_main`
228
-
229
-
## Execute Inference Using Llama Runner
230
-
Update the model tokenizer file path to match the location where your model is downloaded and replace the prompt.
0 commit comments