Skip to content

Commit eee15c3

Browse files
committed
Add a small README
Signed-off-by: Dennis Keck <[email protected]>
1 parent c45a061 commit eee15c3

File tree

1 file changed

+27
-0
lines changed

1 file changed

+27
-0
lines changed

examples/mtmd/README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Rust mtmd-cli implementation
2+
3+
Partial port of the mtmd-cli.cpp example in the llama-cpp repository.
4+
5+
## Usage
6+
7+
### Command Line Interface
8+
9+
To run the mtmd example, you first need to download the model gguf file and the multimodal projection file, e.g. for Gemma3 you may use:
10+
11+
```sh
12+
wget https://huggingface.co/unsloth/gemma-3-4b-it-GGUF/resolve/main/gemma-3-4b-it-Q4_K_M.gguf \
13+
https://huggingface.co/unsloth/gemma-3-4b-it-GGUF/resolve/main/mmproj-F16.gguf
14+
```
15+
16+
To then run the example on CPU, provide an image file `my_image.jpg` and run:
17+
18+
```sh
19+
cargo run --release --example mtmd -- \
20+
--model ./gemma-3-4b-it-Q4_K_M.gguf \
21+
--mmproj ./mmproj-F16.gguf \
22+
--image my_image.jpg \
23+
--prompt "What is in the picture?" \
24+
--no-gpu \
25+
--no-mmproj-offload \
26+
--marker "<start_of_image>"
27+
```

0 commit comments

Comments
 (0)