Skip to content

Commit 86553e8

Browse files
authored
Update README.md and add model optimization sample (#426)
1 parent 3788b18 commit 86553e8

File tree

1 file changed

+5
-4
lines changed

1 file changed

+5
-4
lines changed

mobile/examples/phi-3/android/README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
1-
# ONNX Runtime Mobile Phi-3 Android sample application with ORT GenAI library support
1+
# Local Chatbot on Android with Phi-3, ONNX Runtime Mobile and ONNX Runtime Generate() API
22

33
## Overview
44

5-
This is a basic [Phi-3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx) example application for [ONNX Runtime](https://github.com/microsoft/onnxruntime) on Android with [Ort-GenAI](https://github.com/microsoft/onnxruntime-genai) support for running generative AI models. This app demonstrates the usage of phi-3 model in a simple question answering chatbot mode.
5+
This is a basic [Phi-3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) Android example application with [ONNX Runtime mobile](https://onnxruntime.ai/docs/tutorials/mobile/) and [ONNX Runtime Generate() API](https://github.com/microsoft/onnxruntime-genai) with support for efficiently running generative AI models. This app demonstrates the usage of phi-3 model in a simple question answering chatbot mode.
66

77
### Model
8-
The model used here is from the source: [HuggingFace](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx#phi-3-mini-4k-instruct-onnx-models)
9-
and optimized for int4 CPU and Mobile usage.
8+
The model used here is [ONNX Phi-3 model on HuggingFace](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/tree/main/cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4) with INT4 quantization and optimizations for mobile usage.
9+
10+
You can also optimize your fine-tuned PyTorch Phi-3 model for mobile usage following this example [Phi3 optimization with Olive](https://github.com/microsoft/Olive/tree/main/examples/phi3).
1011

1112
### Requirements
1213
- Android Studio Giraffe | 2022.3.1 or later (installed on Mac/Windows/Linux)

0 commit comments

Comments
 (0)