Skip to content

Commit 5c812c1

Browse files
grorge123hydai
authored andcommitted
docs(MLX): add README
Signed-off-by: grorge <[email protected]>
1 parent ee26cb6 commit 5c812c1

File tree

2 files changed

+52
-1
lines changed

2 files changed

+52
-1
lines changed

wasmedge-mlx/whisper/README.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Whisper example with WasmEdge WASI-NN MLX plugin
2+
3+
This example demonstrates using WasmEdge WASI-NN MLX plugin to perform an inference task with whisper model.
4+
5+
## Install WasmEdge with WASI-NN MLX plugin
6+
7+
The MLX backend relies on [MLX](https://github.com/ml-explore/mlx), but we will auto-download MLX when you build WasmEdge. You do not need to install it yourself. If you want to custom MLX, install it yourself or set the `CMAKE_PREFIX_PATH` variable when configuring cmake.
8+
9+
Build and install WasmEdge from source:
10+
11+
``` bash
12+
cd <path/to/your/wasmedge/source/folder>
13+
14+
cmake -GNinja -Bbuild -DCMAKE_BUILD_TYPE=Release -DWASMEDGE_PLUGIN_WASI_NN_BACKEND="mlx"
15+
cmake --build build
16+
17+
# For the WASI-NN plugin, you should install this project.
18+
cmake --install build
19+
```
20+
21+
Then you will have an executable `wasmedge` runtime under `/usr/local/bin` and the WASI-NN with MLX backend plug-in under `/usr/local/lib/wasmedge/libwasmedgePluginWasiNN.so` after installation.
22+
23+
## Download the model and tokenizer
24+
25+
In this example, we will use `whisper-tiny`.
26+
27+
``` bash
28+
git clone https://huggingface.co/grorge123/whisper-tiny
29+
cp -r whisper-tiny/assets .
30+
wget https://raw.githubusercontent.com/ml-explore/mlx-examples/refs/heads/main/whisper/mlx_whisper/assets/multilingual.tiktoken -P assets
31+
```
32+
33+
## Build wasm
34+
35+
Run the following command to build wasm, the output WASM file will be at `target/wasm32-wasip1/release/`.
36+
Then we use AOT-compiled WASM to improve the performance.
37+
38+
```bash
39+
cargo build --target wasm32-wasip1 --release
40+
```
41+
## Execute
42+
43+
Download the audio and save as `audio.mp3`
44+
45+
Execute the WASM with the `wasmedge` using nn-preload to load model.
46+
47+
``` bash
48+
wasmedge --dir .:. \
49+
--nn-preload default:mlx:AUTO:whisper-tiny/weights.safetensors \
50+
target/wasm32-wasip1/release/wasmedge-whisper.wasm default whisper-tiny audio.mp3
51+
```

wasmedge-mlx/whisper/src/main.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,9 +32,9 @@ fn read_json(path: &str) -> io::Result<Value> {
3232
}
3333
fn main() {
3434
let args: Vec<String> = env::args().collect();
35-
let audio = "audio.mp3";
3635
let model_name: &str = &args[1];
3736
let model_dir = &args[2];
37+
let audio = &args[3];
3838
let config = read_json(&format!("{}/config.json", model_dir)).unwrap();
3939
let graph = GraphBuilder::new(GraphEncoding::Mlx, ExecutionTarget::AUTO)
4040
.config(config.to_string())

0 commit comments

Comments
 (0)