Skip to content

Commit 53c49fb

Browse files
helunwencserfacebook-github-bot
authored andcommitted
update doc for phi-3-mini (#5320)
Summary: Update doc for phi-3-mini to reflect the latest state. Pull Request resolved: #5320 Reviewed By: tarun292 Differential Revision: D62601679 Pulled By: helunwencser fbshipit-source-id: 3d75151d62494327a0ed6771aa7a468fe4ea7c04
1 parent 523b41e commit 53c49fb

File tree

2 files changed

+15
-5
lines changed

2 files changed

+15
-5
lines changed

examples/models/phi-3-mini/README.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@ This example demonstrates how to run a [Phi-3-mini](https://huggingface.co/micro
44
# Instructions
55
## Step 1: Setup
66
1. Follow the [tutorial](https://pytorch.org/executorch/main/getting-started-setup) to set up ExecuTorch. For installation run `./install_requirements.sh --pybind xnnpack`
7-
2. To export Phi-3-mini, we need this [PR](https://github.com/huggingface/transformers/pull/32339). Install transformers from master with the following command:
7+
2. Currently, we support transformers v4.44.2. Install transformers with the following command:
88
```
9-
pip uninstall -y transformers ; pip install git+https://github.com/huggingface/transformers
9+
pip uninstall -y transformers ; pip install transformers==4.44.2
1010
```
1111
## Step 2: Prepare and run the model
1212
1. Download the `tokenizer.model` from HuggingFace and create `tokenizer.bin`.
@@ -53,5 +53,14 @@ cmake --build cmake-out/examples/models/phi-3-mini -j16 --config Release
5353
```
5454
- Run model. Options available [here](https://github.com/pytorch/executorch/blob/main/examples/models/phi-3-mini/main.cpp#L13-L30)
5555
```
56-
cmake-out/examples/models/phi-3-mini/phi_3_mini_runner --model_path=<model pte file> --tokenizer_path=<tokenizer.bin> --seq_len=128 --prompt=<prompt>
56+
cmake-out/examples/models/phi-3-mini/phi_3_mini_runner \
57+
--model_path=phi-3-mini.pte \
58+
--tokenizer_path=tokenizer.bin \
59+
--seq_len=128 \
60+
--temperature=0 \
61+
--prompt="<|system|>
62+
You are a helpful assistant.<|end|>
63+
<|user|>
64+
What is the capital of France?<|end|>
65+
<|assistant|>"
5766
```

examples/models/phi-3-mini/runner.cpp

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ uint64_t Runner::logits_to_token(const exec_aten::Tensor& logits_tensor) {
8181
}
8282

8383
uint64_t Runner::prefill(std::vector<uint64_t>& tokens) {
84-
auto result = module_->forward(from_blob(
84+
auto result = module_->forward(executorch::extension::from_blob(
8585
tokens.data(),
8686
{1, static_cast<exec_aten::SizesType>(tokens.size())},
8787
ScalarType::Long));
@@ -91,7 +91,8 @@ uint64_t Runner::prefill(std::vector<uint64_t>& tokens) {
9191
}
9292

9393
uint64_t Runner::run_model_step(uint64_t token) {
94-
auto result = module_->forward(from_blob(&token, {1, 1}, ScalarType::Long));
94+
auto result = module_->forward(
95+
executorch::extension::from_blob(&token, {1, 1}, ScalarType::Long));
9596
ET_CHECK_MSG(
9697
result.error() == Error::Ok,
9798
"Failed to run forward() for token %" PRIu64,

0 commit comments

Comments
 (0)