Skip to content

Commit d2c291b

Browse files
mergennachinmergennachin
committed
Update top-level README.md file
1 parent d00279d commit d2c291b

File tree

1 file changed

+221
-50
lines changed

1 file changed

+221
-50
lines changed

‎README.md‎

Lines changed: 221 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,72 +1,243 @@
11
<div align="center">
2-
<img src="docs/source/_static/img/et-logo.png" alt="Logo" width="200">
3-
<h1 align="center">ExecuTorch: A powerful on-device AI Framework</h1>
2+
<img src="docs/source/_static/img/et-logo.png" alt="ExecuTorch logo mark" width="200">
3+
<h1>ExecuTorch</h1>
4+
<p><strong>On-device AI inference powered by PyTorch</strong></p>
45
</div>
56

6-
77
<div align="center">
8-
<a href="https://github.com/pytorch/executorch/graphs/contributors"><img src="https://img.shields.io/github/contributors/pytorch/executorch?style=for-the-badge&color=blue" alt="Contributors"></a>
9-
<a href="https://github.com/pytorch/executorch/stargazers"><img src="https://img.shields.io/github/stars/pytorch/executorch?style=for-the-badge&color=blue" alt="Stargazers"></a>
10-
<a href="https://discord.gg/Dh43CKSAdc"><img src="https://img.shields.io/badge/Discord-Join%20Us-purple?logo=discord&logoColor=white&style=for-the-badge" alt="Join our Discord community"></a>
11-
<a href="https://pytorch.org/executorch/main/index"><img src="https://img.shields.io/badge/Documentation-000?logo=googledocs&logoColor=FFE165&style=for-the-badge" alt="Check out the documentation"></a>
12-
<hr>
8+
<a href="https://pypi.org/project/executorch/"><img src="https://img.shields.io/pypi/v/executorch?style=for-the-badge&color=blue" alt="PyPI - Version"></a>
9+
<a href="https://github.com/pytorch/executorch/graphs/contributors"><img src="https://img.shields.io/github/contributors/pytorch/executorch?style=for-the-badge&color=blue" alt="GitHub - Contributors"></a>
10+
<a href="https://github.com/pytorch/executorch/stargazers"><img src="https://img.shields.io/github/stars/pytorch/executorch?style=for-the-badge&color=blue" alt="GitHub - Stars"></a>
11+
<a href="https://discord.gg/Dh43CKSAdc"><img src="https://img.shields.io/badge/Discord-Join%20Us-purple?logo=discord&logoColor=white&style=for-the-badge" alt="Discord - Chat with Us"></a>
12+
<a href="https://docs.pytorch.org/executorch/main/index.html"><img src="https://img.shields.io/badge/Documentation-000?logo=googledocs&logoColor=FFE165&style=for-the-badge" alt="Documentation"></a>
13+
<br>
1314
</div>
1415

15-
**ExecuTorch** is an end-to-end solution for on-device inference and training. It powers much of Meta's on-device AI experiences across Facebook, Instagram, Meta Quest, Ray-Ban Meta Smart Glasses, WhatsApp, and more.
16+
**ExecuTorch** is PyTorch's end-to-end solution for deploying AI models on edge devices—from smartphones to microcontrollers. Built for privacy, performance, and portability, it powers Meta's on-device AI across **Instagram, WhatsApp, Quest 3, Ray-Ban Meta Smart Glasses**, and [more](https://docs.pytorch.org/executorch/main/success-stories.html).
17+
18+
Deploy **LLMs, vision, speech, and multimodal models** with the same PyTorch APIs you already know—accelerating research to production with seamless model export, optimization, and deployment. No manual C++ rewrites. No format conversions. No vendor lock-in.
19+
20+
<details>
21+
<summary><strong>📘 Table of Contents</strong></summary>
22+
23+
- [Why ExecuTorch?](#why-executorch)
24+
- [How It Works](#how-it-works)
25+
- [Quick Start](#quick-start)
26+
- [Installation](#installation)
27+
- [Export and Deploy in 3 Steps](#export-and-deploy-in-3-steps)
28+
- [Run on Device](#run-on-device)
29+
- [LLM Example: Llama](#llm-example-llama)
30+
- [Platform & Hardware Support](#platform--hardware-support)
31+
- [Production Deployments](#production-deployments)
32+
- [Examples & Models](#examples--models)
33+
- [Key Features](#key-features)
34+
- [Documentation](#documentation)
35+
- [Community & Contributing](#community--contributing)
36+
- [License](#license)
37+
38+
</details>
39+
40+
## Why ExecuTorch?
41+
42+
- **🔒 Native PyTorch Export** — Direct export from PyTorch. No .onnx, .tflite, or intermediate format conversions. Preserve model semantics.
43+
- **⚡ Production-Proven** — Powers billions of users at [Meta with real-time on-device inference](https://engineering.fb.com/2025/07/28/android/executorch-on-device-ml-meta-family-of-apps/).
44+
- **💾 Tiny Runtime** — 50KB base footprint. Runs on microcontrollers to high-end smartphones.
45+
- **🚀 [12+ Hardware Backends](https://docs.pytorch.org/executorch/main/backends-overview.html)** — Open-source acceleration for Apple, Qualcomm, ARM, MediaTek, Vulkan, and more.
46+
- **🎯 One Export, Multiple Backends** — Switch hardware targets with a single line change. Deploy the same model everywhere.
47+
48+
## How It Works
49+
50+
ExecuTorch uses **ahead-of-time (AOT) compilation** to prepare PyTorch models for edge deployment:
51+
52+
1. **Export** — Capture PyTorch model as a graph using `torch.export()`
53+
2. **Compile** — Apply quantization, optimizations, and partition to hardware backends → `.pte` file
54+
3. **Execute** — Load `.pte` file on-device with lightweight C++ runtime
55+
56+
Models use a standardized [Core ATen operator set](https://docs.pytorch.org/executorch/main/concepts.html#core-aten-operators). [Partitioners](https://docs.pytorch.org/executorch/main/compiler-delegate-and-partitioner.html) delegate subgraphs to specialized hardware (NPU/GPU) with CPU fallback.
57+
58+
Learn more: [How ExecuTorch Works](https://docs.pytorch.org/executorch/main/intro-how-it-works.html) • [Architecture Guide](https://docs.pytorch.org/executorch/main/getting-started-architecture.html)
59+
60+
## Quick Start
61+
62+
### Installation
63+
64+
```bash
65+
pip install executorch
66+
```
67+
68+
For platform-specific setup (Android, iOS, embedded systems), see the [Quick Start](https://docs.pytorch.org/executorch/main/quick-start-section.html) documentation for additional info.
69+
70+
### Export and Deploy in 3 Steps
71+
72+
```python
73+
import torch
74+
from executorch.exir import to_edge_transform_and_lower
75+
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
76+
77+
# 1. Export your PyTorch model
78+
model = MyModel().eval()
79+
example_inputs = (torch.randn(1, 3, 224, 224),)
80+
exported_program = torch.export.export(model, example_inputs)
81+
82+
# 2. Optimize for target hardware (switch backends with one line)
83+
program = to_edge_transform_and_lower(
84+
exported_program,
85+
partitioner=[XnnpackPartitioner()] # CPU | CoreMLPartitioner() for iOS | QnnPartitioner() for Qualcomm
86+
).to_executorch()
87+
88+
# 3. Save for deployment
89+
with open("model.pte", "wb") as f:
90+
f.write(program.buffer)
91+
```
92+
93+
### Run on Device
94+
95+
**[C++](https://docs.pytorch.org/executorch/main/using-executorch-cpp.html)**
96+
```cpp
97+
#include <executorch/extension/module/module.h>
98+
#include <executorch/extension/tensor/tensor.h>
99+
100+
Module module("model.pte");
101+
auto tensor = make_tensor_ptr({2, 2}, {1.0f, 2.0f, 3.0f, 4.0f});
102+
auto outputs = module.forward({tensor});
103+
```
104+
105+
**[Swift (iOS)](https://docs.pytorch.org/executorch/main/ios-section.html)**
106+
```swift
107+
let module = Module(filePath: "model.pte")
108+
let input = Tensor<Float>([1.0, 2.0, 3.0, 4.0])
109+
let outputs: [Value] = try module.forward([input])
110+
```
111+
112+
**[Kotlin (Android)](https://docs.pytorch.org/executorch/main/android-section.html)**
113+
```kotlin
114+
val module = Module.load("model.pte")
115+
val inputTensor = Tensor.fromBlob(floatArrayOf(1.0f, 2.0f, 3.0f, 4.0f), longArrayOf(2, 2))
116+
val outputs = module.forward(EValue.from(inputTensor))
117+
```
118+
119+
### LLM Example: Llama
120+
121+
Export Llama models using the [`export_llm`](https://docs.pytorch.org/executorch/main/llm/export-llm.html) script or [Optimum-ExecuTorch](https://github.com/huggingface/optimum-executorch):
122+
123+
```bash
124+
# Using export_llm
125+
python -m executorch.extension.llm.export.export_llm --model llama3_2 --output llama.pte
16126

17-
It supports a wide range of models including LLMs (Large Language Models), CV (Computer Vision), ASR (Automatic Speech Recognition), and TTS (Text to Speech).
127+
# Using Optimum-ExecuTorch
128+
optimum-cli export executorch \
129+
--model meta-llama/Llama-3.2-1B \
130+
--task text-generation \
131+
--recipe xnnpack \
132+
--output_dir llama_model
133+
```
18134

19-
Platform Support:
20-
- Operating Systems:
21-
- iOS
22-
- MacOS (ARM64)
23-
- Android
24-
- Linux
25-
- Microcontrollers
135+
Run on-device with the LLM runner API:
26136

27-
- Hardware Acceleration:
28-
- Apple
29-
- Arm
30-
- Cadence
31-
- MediaTek
32-
- NXP
33-
- OpenVINO
34-
- Qualcomm
35-
- Vulkan
36-
- XNNPACK
137+
**[C++](https://docs.pytorch.org/executorch/main/llm/run-with-c-plus-plus.html)**
138+
```cpp
139+
#include <executorch/extension/llm/runner/text_llm_runner.h>
37140

38-
Key value propositions of ExecuTorch are:
141+
auto runner = create_llama_runner("llama.pte", "tiktoken.bin");
142+
executorch::extension::llm::GenerationConfig config{
143+
.seq_len = 128, .temperature = 0.8f};
144+
runner->generate("Hello, how are you?", config);
145+
```
39146
40-
- **Portability:** Compatibility with a wide variety of computing platforms,
41-
from high-end mobile phones to highly constrained embedded systems and
42-
microcontrollers.
43-
- **Productivity:** Enabling developers to use the same toolchains and Developer
44-
Tools from PyTorch model authoring and conversion, to debugging and deployment
45-
to a wide variety of platforms.
46-
- **Performance:** Providing end users with a seamless and high-performance
47-
experience due to a lightweight runtime and utilizing full hardware
48-
capabilities such as CPUs, NPUs, and DSPs.
147+
**[Swift (iOS)](https://docs.pytorch.org/executorch/main/llm/run-on-ios.html)**
148+
```swift
149+
let runner = TextRunner(modelPath: "llama.pte", tokenizerPath: "tiktoken.bin")
150+
try runner.generate("Hello, how are you?", Config {
151+
$0.sequenceLength = 128
152+
}) { token in
153+
print(token, terminator: "")
154+
}
155+
```
49156

50-
## Getting Started
51-
To get started you can:
157+
**Kotlin (Android)** — [API Docs](https://docs.pytorch.org/executorch/main/javadoc/org/pytorch/executorch/extension/llm/package-summary.html) • [Demo App](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android/LlamaDemo)
158+
```kotlin
159+
val llmModule = LlmModule("llama.pte", "tiktoken.bin", 0.8f)
160+
llmModule.load()
161+
llmModule.generate("Hello, how are you?", 128, object : LlmCallback {
162+
override fun onResult(result: String) { print(result) }
163+
override fun onStats(stats: String) { }
164+
})
165+
```
52166

53-
- Visit the [Step by Step Tutorial](https://pytorch.org/executorch/stable/getting-started.html) to get things running locally and deploy a model to a device
54-
- Use this [Colab Notebook](https://colab.research.google.com/drive/1qpxrXC3YdJQzly3mRg-4ayYiOjC6rue3?usp=sharing) to start playing around right away
55-
- Jump straight into LLM use cases by following specific instructions for popular open-source models such as [Llama](examples/models/llama/README.md), [Qwen 3](examples/models/qwen3/README.md), [Phi-4-mini](examples/models/phi_4_mini/README.md), [Llava](examples/models/llava/README.md), [Voxtral](examples/models/voxtral/README.md), and [LFM2](examples/models/lfm2/README.md).
167+
For multimodal models (vision, audio), use the [MultiModal runner API](extension/llm/runner) which extends the LLM runner to handle image and audio inputs alongside text. See [Llava](examples/models/llava/README.md) and [Voxtral](examples/models/voxtral/README.md) examples.
56168

57-
## Feedback and Engagement
169+
See [examples/models/llama](examples/models/llama/README.md) for complete workflow including quantization, mobile deployment, and advanced options.
58170

59-
We welcome any feedback, suggestions, and bug reports from the community to help
60-
us improve our technology. Check out the [Discussion Board](https://github.com/pytorch/executorch/discussions) or chat real time with us on [Discord](https://discord.gg/Dh43CKSAdc)
171+
**Next Steps:**
172+
- 📖 [Step-by-step tutorial](https://docs.pytorch.org/executorch/main/getting-started.html) — Complete walkthrough for your first model
173+
- ⚡ [Colab notebook](https://colab.research.google.com/drive/1qpxrXC3YdJQzly3mRg-4ayYiOjC6rue3?usp=sharing) — Try ExecuTorch instantly in your browser
174+
- 🤖 [Deploy Llama models](examples/models/llama/README.md) — LLM workflow with quantization and mobile demos
61175

62-
## Contributing
176+
## Platform & Hardware Support
63177

64-
We welcome contributions. To get started review the [guidelines](CONTRIBUTING.md) and chat with us on [Discord](https://discord.gg/Dh43CKSAdc)
178+
| Platform | Backend |
179+
|----------|---------|
180+
| **Android** | XNNPACK, Vulkan, Qualcomm, MediaTek, Samsung Exynos |
181+
| **iOS** | XNNPACK, MPS, CoreML (Neural Engine) |
182+
| **Linux/Windows** | XNNPACK, OpenVINO, CUDA (via AOTI, experimental) |
183+
| **MacOS** | XNNPACK, MPS, Metal (via AOTI, experimental) |
184+
| **Embedded** | XNNPACK, ARM Ethos-U, NXP, Cadence DSP |
65185

186+
See [Backend Documentation](https://docs.pytorch.org/executorch/main/backends-overview.html) for detailed hardware requirements and optimization guides.
66187

67-
## Directory Structure
188+
## Production Deployments
68189

69-
Please refer to the [Codebase structure](CONTRIBUTING.md#codebase-structure) section of the [Contributing Guidelines](CONTRIBUTING.md) for more details.
190+
ExecuTorch powers on-device AI at scale across Meta's family of apps, VR/AR devices, and partner deployments. [View success stories →](https://docs.pytorch.org/executorch/main/success-stories.html)
191+
192+
## Examples & Models
193+
194+
**LLMs:** [Llama 3.2/3.1/3](examples/models/llama/README.md), [Qwen 3](examples/models/qwen3/README.md), [Phi-4-mini](examples/models/phi_4_mini/README.md), [LiquidAI LFM2](examples/models/lfm2/README.md)
195+
196+
**Multimodal:** [Llava](examples/models/llava/README.md) (vision-language), [Voxtral](examples/models/voxtral/README.md) (audio-language)
197+
198+
**Vision/Speech:** [MobileNetV2](https://github.com/meta-pytorch/executorch-examples/tree/main/mv2), [DeepLabV3](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3)
199+
200+
**Resources:** [`examples/`](examples/) directory • [executorch-examples](https://github.com/meta-pytorch/executorch-examples) mobile demos • [Optimum-ExecuTorch](https://github.com/huggingface/optimum-executorch) for HuggingFace models
201+
202+
## Key Features
203+
204+
ExecuTorch provides advanced capabilities for production deployment:
205+
206+
- **Quantization** — Built-in support via [torchao](https://docs.pytorch.org/ao) for 8-bit, 4-bit, and dynamic quantization
207+
- **Memory Planning** — Optimize memory usage with ahead-of-time allocation strategies
208+
- **Developer Tools** — ETDump profiler, ETRecord inspector, and model debugger
209+
- **Selective Build** — Strip unused operators to minimize binary size
210+
- **Custom Operators** — Extend with domain-specific kernels
211+
- **Dynamic Shapes** — Support variable input sizes with bounded ranges
212+
213+
See [Advanced Topics](https://docs.pytorch.org/executorch/main/advanced-topics-section.html) for quantization techniques, custom backends, and compiler passes.
214+
215+
## Documentation
216+
217+
- [**Documentation Home**](https://docs.pytorch.org/executorch/main/index.html) — Complete guides and tutorials
218+
- [**API Reference**](https://docs.pytorch.org/executorch/main/api-section.html) — Python, C++, Java/Kotlin APIs
219+
- [**Backend Integration**](https://docs.pytorch.org/executorch/main/backend-delegates-integration.html) — Build custom hardware backends
220+
- [**Troubleshooting**](https://docs.pytorch.org/executorch/main/using-executorch-troubleshooting.html) — Common issues and solutions
221+
222+
## Community & Contributing
223+
224+
We welcome contributions from the community!
225+
226+
- 💬 [**GitHub Discussions**](https://github.com/pytorch/executorch/discussions) — Ask questions and share ideas
227+
- 🎮 [**Discord**](https://discord.gg/Dh43CKSAdc) — Chat with the team and community
228+
- 🐛 [**Issues**](https://github.com/pytorch/executorch/issues) — Report bugs or request features
229+
- 🤝 [**Contributing Guide**](CONTRIBUTING.md) — Guidelines and codebase structure
70230

71231
## License
72-
ExecuTorch is BSD licensed, as found in the LICENSE file.
232+
233+
ExecuTorch is BSD licensed, as found in the [LICENSE](LICENSE) file.
234+
235+
---
236+
237+
<div align="center">
238+
<p><strong>Part of the PyTorch ecosystem</strong></p>
239+
<p>
240+
<a href="https://github.com/pytorch/executorch">GitHub</a> •
241+
<a href="https://docs.pytorch.org/executorch">Documentation</a> •
242+
</p>
243+
</div>

0 commit comments

Comments
 (0)