Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/source/llm/export-llm.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

Instead of needing to manually write code to call torch.export(), use ExecuTorch's assortment of lowering APIs, or even interact with TorchAO quantize_ APIs for quantization, we have provided an out of box experience which performantly exports a selection of supported models to ExecuTorch.

## Prerequisites

The LLM export functionality requires the `pytorch_tokenizers` package. If you encounter a `ModuleNotFoundError: No module named 'pytorch_tokenizers'` error, install it from the ExecutorTorch source code:

```bash
pip install -e ./extension/llm/tokenizers/
```

## Supported Models

As of this doc, the list of supported LLMs include the following:
- Llama 2/3/3.1/3.2
- Qwen 2.5/3
Expand Down
7 changes: 7 additions & 0 deletions docs/source/using-executorch-faqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ sudo apt install python<version>-dev
```
if you are using Ubuntu, or use an equivalent install command.

### ModuleNotFoundError: No module named 'pytorch_tokenizers'

The `pytorch_tokenizers` package is required for LLM export functionality. Install it from the ExecutorTorch source code:
```
pip install -e ./extension/llm/tokenizers/
```

## Export

### Missing out variants: { _ }
Expand Down
Loading