Skip to content

Commit ad1fdcf

Browse files
authored
Update README with AOTI documentation link
Added a link to the AOTI documentation in the README.
1 parent a81dabd commit ad1fdcf

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

desktop/README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ ExecuTorch is a lightweight, flexible runtime designed for efficient AI inferenc
1010
- **No CUDA Support**: For a long time, ExecuTorch did not offer a CUDA backend, limiting GPU acceleration on NVIDIA hardware.
1111

1212
## Recent Developments
13-
With increased demand for local inference on consumer desktops and laptops, exemplified by popular runtimes like llama.cpp and MLX, ExecuTorch is now experimenting with CUDA and Metal support. This is achieved by leveraging Inductor compiler technology from PyTorch, specifically using Ahead-of-Time Inductor (AOTI) to avoid reinventing the wheel.
13+
With increased demand for local inference on consumer desktops and laptops, exemplified by popular runtimes like llama.cpp and MLX, ExecuTorch is now experimenting with CUDA and Metal support. This is achieved by leveraging Inductor compiler technology from PyTorch, specifically using Ahead-of-Time Inductor [AOTI](https://docs.pytorch.org/docs/stable/torch.compiler_aot_inductor.html) to avoid reinventing the wheel.
1414

1515
## Key Benefits
1616
- **Model Agnostic**: Validated on models such as [Voxtral](../examples/models/voxtral), [Gemma3-4b](../examples/models/gemma3), ResNet, and Whisper (WIP). Theoretically, any model exportable via torch.export is supported.
@@ -29,3 +29,4 @@ Backends leveraging AoTi
2929
- **Experimental Status**: CUDA and Metal backends via AoTi are currently experimental. Contributions and feedback are welcome!
3030
- **Model Compatibility**: While most models exportable via torch.export should work, validation is ongoing for broader model support.
3131
- **Portability**: Figuring out the balance and trade-off between performance, portability and model filesize.
32+
- **Windows-native WIP**: On windows we only supports WSL right now. Native Windows support is WIP.

0 commit comments

Comments
 (0)