Skip to content

Commit adb594f

Browse files
mergennachinmergennachin
committed
Update top-level README.md file
1 parent d00279d commit adb594f

File tree

1 file changed

+222
-50
lines changed

1 file changed

+222
-50
lines changed

‎README.md‎

Lines changed: 222 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,72 +1,244 @@
11
<div align="center">
2-
<img src="docs/source/_static/img/et-logo.png" alt="Logo" width="200">
3-
<h1 align="center">ExecuTorch: A powerful on-device AI Framework</h1>
2+
<img src="docs/source/_static/img/et-logo.png" alt="ExecuTorch logo mark" width="200">
3+
<h1>ExecuTorch</h1>
4+
<p><strong>On-device AI inference powered by PyTorch</strong></p>
45
</div>
56

6-
77
<div align="center">
8-
<a href="https://github.com/pytorch/executorch/graphs/contributors"><img src="https://img.shields.io/github/contributors/pytorch/executorch?style=for-the-badge&color=blue" alt="Contributors"></a>
9-
<a href="https://github.com/pytorch/executorch/stargazers"><img src="https://img.shields.io/github/stars/pytorch/executorch?style=for-the-badge&color=blue" alt="Stargazers"></a>
10-
<a href="https://discord.gg/Dh43CKSAdc"><img src="https://img.shields.io/badge/Discord-Join%20Us-purple?logo=discord&logoColor=white&style=for-the-badge" alt="Join our Discord community"></a>
11-
<a href="https://pytorch.org/executorch/main/index"><img src="https://img.shields.io/badge/Documentation-000?logo=googledocs&logoColor=FFE165&style=for-the-badge" alt="Check out the documentation"></a>
12-
<hr>
8+
<a href="https://pypi.org/project/executorch/"><img src="https://img.shields.io/pypi/v/executorch?style=for-the-badge&color=blue" alt="PyPI - Version"></a>
9+
<a href="https://github.com/pytorch/executorch/graphs/contributors"><img src="https://img.shields.io/github/contributors/pytorch/executorch?style=for-the-badge&color=blue" alt="GitHub - Contributors"></a>
10+
<a href="https://github.com/pytorch/executorch/stargazers"><img src="https://img.shields.io/github/stars/pytorch/executorch?style=for-the-badge&color=blue" alt="GitHub - Stars"></a>
11+
<a href="https://discord.gg/Dh43CKSAdc"><img src="https://img.shields.io/badge/Discord-Join%20Us-blue?logo=discord&logoColor=white&style=for-the-badge" alt="Discord - Chat with Us"></a>
12+
<a href="https://docs.pytorch.org/executorch/main/index.html"><img src="https://img.shields.io/badge/Documentation-blue?logo=googledocs&logoColor=white&style=for-the-badge" alt="Documentation"></a>
1313
</div>
1414

15-
**ExecuTorch** is an end-to-end solution for on-device inference and training. It powers much of Meta's on-device AI experiences across Facebook, Instagram, Meta Quest, Ray-Ban Meta Smart Glasses, WhatsApp, and more.
15+
**ExecuTorch** is PyTorch's unified solution for deploying AI models on-device—from smartphones to microcontrollers—built for privacy, performance, and portability. It powers Meta's on-device AI across **Instagram, WhatsApp, Quest 3, Ray-Ban Meta Smart Glasses**, and [more](https://docs.pytorch.org/executorch/main/success-stories.html).
16+
17+
Deploy **LLMs, vision, speech, and multimodal models** with the same PyTorch APIs you already know—accelerating research to production with seamless model export, optimization, and deployment. No manual C++ rewrites. No format conversions. No vendor lock-in.
18+
19+
<details>
20+
<summary><strong>📘 Table of Contents</strong></summary>
21+
22+
- [Why ExecuTorch?](#why-executorch)
23+
- [How It Works](#how-it-works)
24+
- [Quick Start](#quick-start)
25+
- [Installation](#installation)
26+
- [Export and Deploy in 3 Steps](#export-and-deploy-in-3-steps)
27+
- [Run on Device](#run-on-device)
28+
- [LLM Example: Llama](#llm-example-llama)
29+
- [Platform & Hardware Support](#platform--hardware-support)
30+
- [Production Deployments](#production-deployments)
31+
- [Examples & Models](#examples--models)
32+
- [Key Features](#key-features)
33+
- [Documentation](#documentation)
34+
- [Community & Contributing](#community--contributing)
35+
- [License](#license)
36+
37+
</details>
38+
39+
## Why ExecuTorch?
40+
41+
- **🔒 Native PyTorch Export** — Direct export from PyTorch. No .onnx, .tflite, or intermediate format conversions. Preserve model semantics.
42+
- **⚡ Production-Proven** — Powers billions of users at [Meta with real-time on-device inference](https://engineering.fb.com/2025/07/28/android/executorch-on-device-ml-meta-family-of-apps/).
43+
- **💾 Tiny Runtime** — 50KB base footprint. Runs on microcontrollers to high-end smartphones.
44+
- **🚀 [12+ Hardware Backends](https://docs.pytorch.org/executorch/main/backends-overview.html)** — Open-source acceleration for Apple, Qualcomm, ARM, MediaTek, Vulkan, and more.
45+
- **🎯 One Export, Multiple Backends** — Switch hardware targets with a single line change. Deploy the same model everywhere.
46+
47+
## How It Works
48+
49+
ExecuTorch uses **ahead-of-time (AOT) compilation** to prepare PyTorch models for edge deployment:
50+
51+
1. **🧩 Export** — Capture your PyTorch model graph with `torch.export()`
52+
2. **⚙️ Compile** — Quantize, optimize, and partition to hardware backends → `.pte`
53+
3. **🚀 Execute** — Load `.pte` on-device via lightweight C++ runtime
54+
55+
Models use a standardized [Core ATen operator set](https://docs.pytorch.org/executorch/main/concepts.html#core-aten-operators). [Partitioners](https://docs.pytorch.org/executorch/main/compiler-delegate-and-partitioner.html) delegate subgraphs to specialized hardware (NPU/GPU) with CPU fallback.
56+
57+
Learn more: [How ExecuTorch Works](https://docs.pytorch.org/executorch/main/intro-how-it-works.html) • [Architecture Guide](https://docs.pytorch.org/executorch/main/getting-started-architecture.html)
58+
59+
## Quick Start
60+
61+
### Installation
62+
63+
```bash
64+
pip install executorch
65+
```
66+
67+
For platform-specific setup (Android, iOS, embedded systems), see the [Quick Start](https://docs.pytorch.org/executorch/main/quick-start-section.html) documentation for additional info.
68+
69+
### Export and Deploy in 3 Steps
70+
71+
```python
72+
import torch
73+
from executorch.exir import to_edge_transform_and_lower
74+
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
75+
76+
# 1. Export your PyTorch model
77+
model = MyModel().eval()
78+
example_inputs = (torch.randn(1, 3, 224, 224),)
79+
exported_program = torch.export.export(model, example_inputs)
80+
81+
# 2. Optimize for target hardware (switch backends with one line)
82+
program = to_edge_transform_and_lower(
83+
exported_program,
84+
partitioner=[XnnpackPartitioner()] # CPU | CoreMLPartitioner() for iOS | QnnPartitioner() for Qualcomm
85+
).to_executorch()
86+
87+
# 3. Save for deployment
88+
with open("model.pte", "wb") as f:
89+
f.write(program.buffer)
90+
```
91+
92+
### Run on Device
93+
94+
**[C++](https://docs.pytorch.org/executorch/main/using-executorch-cpp.html)**
95+
```cpp
96+
#include <executorch/extension/module/module.h>
97+
#include <executorch/extension/tensor/tensor.h>
98+
99+
Module module("model.pte");
100+
auto tensor = make_tensor_ptr({2, 2}, {1.0f, 2.0f, 3.0f, 4.0f});
101+
auto outputs = module.forward({tensor});
102+
```
103+
104+
**[Swift (iOS)](https://docs.pytorch.org/executorch/main/ios-section.html)**
105+
```swift
106+
let module = Module(filePath: "model.pte")
107+
let input = Tensor<Float>([1.0, 2.0, 3.0, 4.0])
108+
let outputs: [Value] = try module.forward([input])
109+
```
110+
111+
**[Kotlin (Android)](https://docs.pytorch.org/executorch/main/android-section.html)**
112+
```kotlin
113+
val module = Module.load("model.pte")
114+
val inputTensor = Tensor.fromBlob(floatArrayOf(1.0f, 2.0f, 3.0f, 4.0f), longArrayOf(2, 2))
115+
val outputs = module.forward(EValue.from(inputTensor))
116+
```
117+
118+
### LLM Example: Llama
119+
120+
Export Llama models using the [`export_llm`](https://docs.pytorch.org/executorch/main/llm/export-llm.html) script or [Optimum-ExecuTorch](https://github.com/huggingface/optimum-executorch):
121+
122+
```bash
123+
# Using export_llm
124+
python -m executorch.extension.llm.export.export_llm --model llama3_2 --output llama.pte
125+
126+
# Using Optimum-ExecuTorch
127+
optimum-cli export executorch \
128+
--model meta-llama/Llama-3.2-1B \
129+
--task text-generation \
130+
--recipe xnnpack \
131+
--output_dir llama_model
132+
```
16133

17-
It supports a wide range of models including LLMs (Large Language Models), CV (Computer Vision), ASR (Automatic Speech Recognition), and TTS (Text to Speech).
134+
Run on-device with the LLM runner API:
18135

19-
Platform Support:
20-
- Operating Systems:
21-
- iOS
22-
- MacOS (ARM64)
23-
- Android
24-
- Linux
25-
- Microcontrollers
136+
**[C++](https://docs.pytorch.org/executorch/main/llm/run-with-c-plus-plus.html)**
137+
```cpp
138+
#include <executorch/extension/llm/runner/text_llm_runner.h>
26139

27-
- Hardware Acceleration:
28-
- Apple
29-
- Arm
30-
- Cadence
31-
- MediaTek
32-
- NXP
33-
- OpenVINO
34-
- Qualcomm
35-
- Vulkan
36-
- XNNPACK
140+
auto runner = create_llama_runner("llama.pte", "tiktoken.bin");
141+
executorch::extension::llm::GenerationConfig config{
142+
.seq_len = 128, .temperature = 0.8f};
143+
runner->generate("Hello, how are you?", config);
144+
```
37145
38-
Key value propositions of ExecuTorch are:
146+
**[Swift (iOS)](https://docs.pytorch.org/executorch/main/llm/run-on-ios.html)**
147+
```swift
148+
let runner = TextRunner(modelPath: "llama.pte", tokenizerPath: "tiktoken.bin")
149+
try runner.generate("Hello, how are you?", Config {
150+
$0.sequenceLength = 128
151+
}) { token in
152+
print(token, terminator: "")
153+
}
154+
```
39155

40-
- **Portability:** Compatibility with a wide variety of computing platforms,
41-
from high-end mobile phones to highly constrained embedded systems and
42-
microcontrollers.
43-
- **Productivity:** Enabling developers to use the same toolchains and Developer
44-
Tools from PyTorch model authoring and conversion, to debugging and deployment
45-
to a wide variety of platforms.
46-
- **Performance:** Providing end users with a seamless and high-performance
47-
experience due to a lightweight runtime and utilizing full hardware
48-
capabilities such as CPUs, NPUs, and DSPs.
156+
**Kotlin (Android)** — [API Docs](https://docs.pytorch.org/executorch/main/javadoc/org/pytorch/executorch/extension/llm/package-summary.html) • [Demo App](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android/LlamaDemo)
157+
```kotlin
158+
val llmModule = LlmModule("llama.pte", "tiktoken.bin", 0.8f)
159+
llmModule.load()
160+
llmModule.generate("Hello, how are you?", 128, object : LlmCallback {
161+
override fun onResult(result: String) { print(result) }
162+
override fun onStats(stats: String) { }
163+
})
164+
```
49165

50-
## Getting Started
51-
To get started you can:
166+
For multimodal models (vision, audio), use the [MultiModal runner API](extension/llm/runner) which extends the LLM runner to handle image and audio inputs alongside text. See [Llava](examples/models/llava/README.md) and [Voxtral](examples/models/voxtral/README.md) examples.
52167

53-
- Visit the [Step by Step Tutorial](https://pytorch.org/executorch/stable/getting-started.html) to get things running locally and deploy a model to a device
54-
- Use this [Colab Notebook](https://colab.research.google.com/drive/1qpxrXC3YdJQzly3mRg-4ayYiOjC6rue3?usp=sharing) to start playing around right away
55-
- Jump straight into LLM use cases by following specific instructions for popular open-source models such as [Llama](examples/models/llama/README.md), [Qwen 3](examples/models/qwen3/README.md), [Phi-4-mini](examples/models/phi_4_mini/README.md), [Llava](examples/models/llava/README.md), [Voxtral](examples/models/voxtral/README.md), and [LFM2](examples/models/lfm2/README.md).
168+
See [examples/models/llama](examples/models/llama/README.md) for complete workflow including quantization, mobile deployment, and advanced options.
56169

57-
## Feedback and Engagement
170+
**Next Steps:**
171+
- 📖 [Step-by-step tutorial](https://docs.pytorch.org/executorch/main/getting-started.html) — Complete walkthrough for your first model
172+
- ⚡ [Colab notebook](https://colab.research.google.com/drive/1qpxrXC3YdJQzly3mRg-4ayYiOjC6rue3?usp=sharing) — Try ExecuTorch instantly in your browser
173+
- 🤖 [Deploy Llama models](examples/models/llama/README.md) — LLM workflow with quantization and mobile demos
58174

59-
We welcome any feedback, suggestions, and bug reports from the community to help
60-
us improve our technology. Check out the [Discussion Board](https://github.com/pytorch/executorch/discussions) or chat real time with us on [Discord](https://discord.gg/Dh43CKSAdc)
175+
## Platform & Hardware Support
61176

62-
## Contributing
177+
| **Platform** | **Supported Backends** |
178+
|------------------|----------------------------------------------------------|
179+
| Android | XNNPACK, Vulkan, Qualcomm, MediaTek, Samsung Exynos |
180+
| iOS | XNNPACK, MPS, CoreML (Neural Engine) |
181+
| Linux / Windows | XNNPACK, OpenVINO, CUDA *(experimental)* |
182+
| macOS | XNNPACK, MPS, Metal *(experimental)* |
183+
| Embedded / MCU | XNNPACK, ARM Ethos-U, NXP, Cadence DSP |
63184

64-
We welcome contributions. To get started review the [guidelines](CONTRIBUTING.md) and chat with us on [Discord](https://discord.gg/Dh43CKSAdc)
185+
See [Backend Documentation](https://docs.pytorch.org/executorch/main/backends-overview.html) for detailed hardware requirements and optimization guides.
65186

187+
## Production Deployments
66188

67-
## Directory Structure
189+
ExecuTorch powers on-device AI at scale across Meta's family of apps, VR/AR devices, and partner deployments. [View success stories →](https://docs.pytorch.org/executorch/main/success-stories.html)
68190

69-
Please refer to the [Codebase structure](CONTRIBUTING.md#codebase-structure) section of the [Contributing Guidelines](CONTRIBUTING.md) for more details.
191+
## Examples & Models
192+
193+
**LLMs:** [Llama 3.2/3.1/3](examples/models/llama/README.md), [Qwen 3](examples/models/qwen3/README.md), [Phi-4-mini](examples/models/phi_4_mini/README.md), [LiquidAI LFM2](examples/models/lfm2/README.md)
194+
195+
**Multimodal:** [Llava](examples/models/llava/README.md) (vision-language), [Voxtral](examples/models/voxtral/README.md) (audio-language)
196+
197+
**Vision/Speech:** [MobileNetV2](https://github.com/meta-pytorch/executorch-examples/tree/main/mv2), [DeepLabV3](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3)
198+
199+
**Resources:** [`examples/`](examples/) directory • [executorch-examples](https://github.com/meta-pytorch/executorch-examples) mobile demos • [Optimum-ExecuTorch](https://github.com/huggingface/optimum-executorch) for HuggingFace models
200+
201+
## Key Features
202+
203+
ExecuTorch provides advanced capabilities for production deployment:
204+
205+
- **Quantization** — Built-in support via [torchao](https://docs.pytorch.org/ao) for 8-bit, 4-bit, and dynamic quantization
206+
- **Memory Planning** — Optimize memory usage with ahead-of-time allocation strategies
207+
- **Developer Tools** — ETDump profiler, ETRecord inspector, and model debugger
208+
- **Selective Build** — Strip unused operators to minimize binary size
209+
- **Custom Operators** — Extend with domain-specific kernels
210+
- **Dynamic Shapes** — Support variable input sizes with bounded ranges
211+
212+
See [Advanced Topics](https://docs.pytorch.org/executorch/main/advanced-topics-section.html) for quantization techniques, custom backends, and compiler passes.
213+
214+
## Documentation
215+
216+
- [**Documentation Home**](https://docs.pytorch.org/executorch/main/index.html) — Complete guides and tutorials
217+
- [**API Reference**](https://docs.pytorch.org/executorch/main/api-section.html) — Python, C++, Java/Kotlin APIs
218+
- [**Backend Integration**](https://docs.pytorch.org/executorch/main/backend-delegates-integration.html) — Build custom hardware backends
219+
- [**Troubleshooting**](https://docs.pytorch.org/executorch/main/using-executorch-troubleshooting.html) — Common issues and solutions
220+
221+
## Community & Contributing
222+
223+
We welcome contributions from the community!
224+
225+
- 💬 [**GitHub Discussions**](https://github.com/pytorch/executorch/discussions) — Ask questions and share ideas
226+
- 🎮 [**Discord**](https://discord.gg/Dh43CKSAdc) — Chat with the team and community
227+
- 🐛 [**Issues**](https://github.com/pytorch/executorch/issues) — Report bugs or request features
228+
- 🤝 [**Contributing Guide**](CONTRIBUTING.md) — Guidelines and codebase structure
70229

71230
## License
72-
ExecuTorch is BSD licensed, as found in the LICENSE file.
231+
232+
ExecuTorch is BSD licensed, as found in the [LICENSE](LICENSE) file.
233+
234+
<br><br>
235+
236+
---
237+
238+
<div align="center">
239+
<p><strong>Part of the PyTorch ecosystem</strong></p>
240+
<p>
241+
<a href="https://github.com/pytorch/executorch">GitHub</a> •
242+
<a href="https://docs.pytorch.org/executorch">Documentation</a>
243+
</p>
244+
</div>

0 commit comments

Comments
 (0)