Skip to content

Commit 610f4c2

Browse files
committed
[Docs] ggml: add notes about using wasi_nn config
Signed-off-by: dm4 <[email protected]>
1 parent 5c064aa commit 610f4c2

File tree

1 file changed

+9
-2
lines changed

1 file changed

+9
-2
lines changed

wasmedge-ggml/README.md

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -207,9 +207,16 @@ Currently, the WASI-NN ggml plugin supports several ways to set the metadata for
207207
When constructing the graph, you can set the metadata by using the `config` method.
208208

209209
```rust
210-
... wasi_nn::GraphBuilder::new(...).config(serde_json::to_string(&options).unwrap()) ...
210+
wasmedge_wasi_nn::GraphBuilder::new(...)
211+
.config(serde_json::to_string(&options).unwrap())
212+
.build_from_cache(...)
213+
.unwrap();
211214
```
212215

216+
> [!NOTE]
217+
> The config will only be set when constructing the graph using `build_from_cache`.
218+
> Due to the file size limitation, you **SHOULD** use `build_from_cache` with `--nn-preload` to load the large model file.
219+
213220
2. From the input tensor
214221

215222
When setting input to the context, specify the index with 1 for the metadata.
@@ -220,7 +227,7 @@ If you modify the `n-gpu-layers` parameter, the model will be reloaded.
220227
context
221228
.set_input(
222229
1,
223-
wasi_nn::TensorType::U8,
230+
wasmedge_wasi_nn::TensorType::U8,
224231
&[1],
225232
serde_json::to_string(&options).expect("Failed to serialize options").as_bytes().to_vec(),
226233
)

0 commit comments

Comments
 (0)