Skip to content

Commit 6259a29

Browse files
mergennachinfacebook-github-bot
authored andcommitted
IMprove README page
Summary: - Add Llama3.2 news to top-level README page - Quick summary of what we did with Llama 3.2 Reviewed By: helunwencser Differential Revision: D63429498 fbshipit-source-id: 18757500d0ab72c5c70b8dfaa9d6a629ce083316
1 parent 7c647cd commit 6259a29

File tree

2 files changed

+7
-0
lines changed

2 files changed

+7
-0
lines changed

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ Check out the [Getting Started](https://pytorch.org/executorch/stable/getting-st
2424

2525
Check out the examples of [Llama](./examples/models/llama2/README.md), [Llava](./examples/models/llava/README.md) and [other models](./examples/README.md) running on edge devices using ExecuTorch.
2626

27+
28+
**[UPDATE - 09/25]** We have added support for running [Llama 3.2 1B/3B](./examples/models/llama2/README.md) models via ExecuTorch.
29+
2730
## Feedback
2831

2932
We welcome any feedback, suggestions, and bug reports from the community to help

examples/models/llama2/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,10 @@ Please note that the models are subject to the [Llama 2 Acceptable Use Policy](h
2424

2525
Since Llama 2 7B or Llama 3 8B model needs at least 4-bit quantization to fit even within some of the highend phones, results presented here correspond to 4-bit groupwise post-training quantized model.
2626

27+
For Llama 3.2 1B/3B, we validated the models by running them in their original bf16 datatype and unquantized on both Android and iOS phones. The 3B version required high-end phones with larger RAMs to fit the model.
28+
29+
Additionally, these models are sensitive to accuracy when regular PTQ quantization is applied, so we employed [SpinQuant](https://github.com/facebookresearch/SpinQuant/tree/main) to achieve a good balance between accuracy and performance.
30+
2731
<table>
2832
<tr>
2933
<td>

0 commit comments

Comments
 (0)