File tree Expand file tree Collapse file tree 1 file changed +27
-0
lines changed Expand file tree Collapse file tree 1 file changed +27
-0
lines changed Original file line number Diff line number Diff line change
1
+ # Rust mtmd-cli implementation
2
+
3
+ Partial port of the mtmd-cli.cpp example in the llama-cpp repository.
4
+
5
+ ## Usage
6
+
7
+ ### Command Line Interface
8
+
9
+ To run the mtmd example, you first need to download the model gguf file and the multimodal projection file, e.g. for Gemma3 you may use:
10
+
11
+ ``` sh
12
+ wget https://huggingface.co/unsloth/gemma-3-4b-it-GGUF/resolve/main/gemma-3-4b-it-Q4_K_M.gguf \
13
+ https://huggingface.co/unsloth/gemma-3-4b-it-GGUF/resolve/main/mmproj-F16.gguf
14
+ ```
15
+
16
+ To then run the example on CPU, provide an image file ` my_image.jpg ` and run:
17
+
18
+ ``` sh
19
+ cargo run --release --example mtmd -- \
20
+ --model ./gemma-3-4b-it-Q4_K_M.gguf \
21
+ --mmproj ./mmproj-F16.gguf \
22
+ --image my_image.jpg \
23
+ --prompt " What is in the picture?" \
24
+ --no-gpu \
25
+ --no-mmproj-offload \
26
+ --marker " <start_of_image>"
27
+ ```
You can’t perform that action at this time.
0 commit comments