Skip to content

Commit d75c738

Browse files
committed
[Docs] ggml: add TriLM and BitNet documents in basic/README.md
Signed-off-by: dm4 <[email protected]>
1 parent 610f4c2 commit d75c738

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed

wasmedge-ggml/basic/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,3 +82,26 @@ $ wasmedge --dir .:. \
8282
--nn-preload default:GGML:AUTO:grok-1-Q2_K-split-00001-of-00009.gguf \
8383
wasmedge-ggml-basic.wasm default 'hello'
8484
```
85+
86+
## TriLM & BitNet Models
87+
88+
After the following pull requests are merged, the `TriLM` and `BitNet` models will be supported by the `ggml` plugin with model type `TQ1_0` and `TQ2_0`:
89+
- https://github.com/ggerganov/llama.cpp/pull/7931
90+
- https://github.com/ggerganov/llama.cpp/pull/8151
91+
92+
### Get the Model
93+
94+
Download the `TriLM` model:
95+
96+
```bash
97+
curl -LO https://huggingface.co/Green-Sky/TriLM_3.9B-GGUF/resolve/main/TriLM_3.9B_Unpacked-4.0B-TQ2_0.gguf
98+
```
99+
100+
### Execute
101+
102+
```console
103+
$ wasmedge --dir .:. \
104+
--env n_predict=100 \
105+
--nn-preload default:GGML:AUTO:TriLM_3.9B_Unpacked-4.0B-TQ2_0.gguf \
106+
wasmedge-ggml-basic.wasm default
107+
```

0 commit comments

Comments
 (0)