Skip to content

Commit 0bec4f6

Browse files
tc-mbtc-mb
committed
update readme of minicpm-v
1 parent ef4222e commit 0bec4f6

File tree

3 files changed

+54
-164
lines changed

3 files changed

+54
-164
lines changed

‎examples/llava/README-minicpmo2.6.md‎

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,25 @@ Currently, this readme only supports minicpm-omni's image capabilities, and we w
55

66
Download [MiniCPM-o-2_6](https://huggingface.co/openbmb/MiniCPM-o-2_6) PyTorch model from huggingface to "MiniCPM-o-2_6" folder.
77

8+
9+
### Build llama.cpp
10+
Readme modification time: 20250206
11+
12+
If there are differences in usage, please refer to the official build [documentation](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md)
13+
814
Clone llama.cpp:
915
```bash
10-
git clone git@github.com:OpenBMB/llama.cpp.git
16+
git clone https://github.com/ggerganov/llama.cpp
1117
cd llama.cpp
12-
git checkout minicpm-omni
1318
```
1419

20+
Build llama.cpp using `CMake`:
21+
```bash
22+
cmake -B build
23+
cmake --build build --config Release
24+
```
25+
26+
1527
### Usage of MiniCPM-o 2.6
1628

1729
Convert PyTorch model to gguf files (You can also download the converted [gguf](https://huggingface.co/openbmb/MiniCPM-o-2_6-gguf) by us)
@@ -22,25 +34,15 @@ python ./examples/llava/minicpmv-convert-image-encoder-to-gguf.py -m ../MiniCPM-
2234
python ./convert_hf_to_gguf.py ../MiniCPM-o-2_6/model
2335

2436
# quantize int4 version
25-
./llama-quantize ../MiniCPM-o-2_6/model/ggml-model-f16.gguf ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf Q4_K_M
37+
./build/bin/llama-quantize ../MiniCPM-o-2_6/model/ggml-model-f16.gguf ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf Q4_K_M
2638
```
2739

28-
Build llama.cpp using `CMake`:
29-
https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md
30-
31-
```bash
32-
cmake -B build
33-
cmake --build build --config Release
34-
```
3540

3641
Inference on Linux or Mac
37-
```
42+
```bash
3843
# run f16 version
39-
./llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-f16.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
44+
./build/bin/llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-f16.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
4045

4146
# run quantized int4 version
42-
./llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
43-
44-
# or run in interactive mode
45-
./llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -i
47+
./build/bin/llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
4648
```

‎examples/llava/README-minicpmv2.5.md‎

Lines changed: 18 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,26 @@
44

55
Download [MiniCPM-Llama3-V-2_5](https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5) PyTorch model from huggingface to "MiniCPM-Llama3-V-2_5" folder.
66

7+
8+
### Build llama.cpp
9+
Readme modification time: 20250206
10+
11+
If there are differences in usage, please refer to the official build [documentation](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md)
12+
713
Clone llama.cpp:
814
```bash
915
git clone https://github.com/ggerganov/llama.cpp
1016
cd llama.cpp
1117
```
1218

13-
### Usage
19+
Build llama.cpp using `CMake`:
20+
```bash
21+
cmake -B build
22+
cmake --build build --config Release
23+
```
24+
25+
26+
### Usage of MiniCPM-Llama3-V 2.5
1427

1528
Convert PyTorch model to gguf files (You can also download the converted [gguf](https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf) by us)
1629

@@ -20,80 +33,15 @@ python ./examples/llava/minicpmv-convert-image-encoder-to-gguf.py -m ../MiniCPM-
2033
python ./convert_hf_to_gguf.py ../MiniCPM-Llama3-V-2_5/model
2134

2235
# quantize int4 version
23-
./llama-quantize ../MiniCPM-Llama3-V-2_5/model/model-8B-F16.gguf ../MiniCPM-Llama3-V-2_5/model/ggml-model-Q4_K_M.gguf Q4_K_M
36+
./build/bin/llama-quantize ../MiniCPM-Llama3-V-2_5/model/model-8B-F16.gguf ../MiniCPM-Llama3-V-2_5/model/ggml-model-Q4_K_M.gguf Q4_K_M
2437
```
2538

26-
Build for Linux or Mac
27-
28-
```bash
29-
make
30-
make llama-minicpmv-cli
31-
```
3239

3340
Inference on Linux or Mac
34-
```
41+
```bash
3542
# run f16 version
36-
./llama-minicpmv-cli -m ../MiniCPM-Llama3-V-2_5/model/model-8B-F16.gguf --mmproj ../MiniCPM-Llama3-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
43+
./build/bin/llama-minicpmv-cli -m ../MiniCPM-Llama3-V-2_5/model/model-8B-F16.gguf --mmproj ../MiniCPM-Llama3-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
3744

3845
# run quantized int4 version
39-
./llama-minicpmv-cli -m ../MiniCPM-Llama3-V-2_5/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-Llama3-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
40-
41-
# or run in interactive mode
42-
./llama-minicpmv-cli -m ../MiniCPM-Llama3-V-2_5/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-Llama3-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -i
43-
```
44-
45-
### Android
46-
47-
#### Build on Android device using Termux
48-
We found that build on Android device would bring better runtime performance, so we recommend to build on device.
49-
50-
[Termux](https://github.com/termux/termux-app#installation) is a terminal app on Android device (no root required).
51-
52-
Install tools in Termux:
53-
```
54-
apt update && apt upgrade -y
55-
apt install git make cmake
56-
```
57-
58-
It's recommended to move your model inside the `~/` directory for best performance:
59-
```
60-
cd storage/downloads
61-
mv model.gguf ~/
62-
```
63-
64-
#### Building the Project using Android NDK
65-
Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake.
66-
67-
Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux:
68-
69-
```bash
70-
mkdir build-android
71-
cd build-android
72-
export NDK=/your_ndk_path
73-
cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
74-
make
75-
```
76-
77-
Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice).
78-
79-
Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission:
80-
81-
(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`)
82-
```
83-
$cp -r /sdcard/llama.cpp/bin /data/data/com.termux/files/home/
84-
$cd /data/data/com.termux/files/home/bin
85-
$chmod +x ./*
86-
```
87-
88-
Download models and push them to `/sdcard/llama.cpp/`, then move it to `/data/data/com.termux/files/home/model/`
89-
90-
```
91-
$mv /sdcard/llama.cpp/ggml-model-Q4_K_M.gguf /data/data/com.termux/files/home/model/
92-
$mv /sdcard/llama.cpp/mmproj-model-f16.gguf /data/data/com.termux/files/home/model/
93-
```
94-
95-
Now, you can start chatting:
96-
```
97-
$cd /data/data/com.termux/files/home/bin
98-
$./llama-minicpmv-cli -m ../model/ggml-model-Q4_K_M.gguf --mmproj ../model/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
46+
./build/bin/llama-minicpmv-cli -m ../MiniCPM-Llama3-V-2_5/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-Llama3-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
9947
```

‎examples/llava/README-minicpmv2.6.md‎

Lines changed: 18 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,25 @@
44

55
Download [MiniCPM-V-2_6](https://huggingface.co/openbmb/MiniCPM-V-2_6) PyTorch model from huggingface to "MiniCPM-V-2_6" folder.
66

7+
8+
### Build llama.cpp
9+
Readme modification time: 20250206
10+
11+
If there are differences in usage, please refer to the official build [documentation](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md)
12+
713
Clone llama.cpp:
814
```bash
9-
git clone git@github.com:OpenBMB/llama.cpp.git
15+
git clone https://github.com/ggerganov/llama.cpp
1016
cd llama.cpp
11-
git checkout minicpmv-main
1217
```
1318

19+
Build llama.cpp using `CMake`:
20+
```bash
21+
cmake -B build
22+
cmake --build build --config Release
23+
```
24+
25+
1426
### Usage of MiniCPM-V 2.6
1527

1628
Convert PyTorch model to gguf files (You can also download the converted [gguf](https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf) by us)
@@ -21,87 +33,15 @@ python ./examples/llava/minicpmv-convert-image-encoder-to-gguf.py -m ../MiniCPM-
2133
python ./convert_hf_to_gguf.py ../MiniCPM-V-2_6/model
2234

2335
# quantize int4 version
24-
./llama-quantize ../MiniCPM-V-2_6/model/ggml-model-f16.gguf ../MiniCPM-V-2_6/model/ggml-model-Q4_K_M.gguf Q4_K_M
36+
./build/bin/llama-quantize ../MiniCPM-V-2_6/model/ggml-model-f16.gguf ../MiniCPM-V-2_6/model/ggml-model-Q4_K_M.gguf Q4_K_M
2537
```
2638

27-
Build for Linux or Mac
28-
29-
```bash
30-
make
31-
make llama-minicpmv-cli
32-
```
3339

3440
Inference on Linux or Mac
35-
```
41+
```bash
3642
# run f16 version
37-
./llama-minicpmv-cli -m ../MiniCPM-V-2_6/model/ggml-model-f16.gguf --mmproj ../MiniCPM-V-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
43+
./build/bin/llama-minicpmv-cli -m ../MiniCPM-V-2_6/model/ggml-model-f16.gguf --mmproj ../MiniCPM-V-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
3844

3945
# run quantized int4 version
40-
./llama-minicpmv-cli -m ../MiniCPM-V-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-V-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
41-
42-
# or run in interactive mode
43-
./llama-minicpmv-cli -m ../MiniCPM-V-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-V-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -i
44-
```
45-
46-
### Video
47-
Install FFmpeg
48-
```
49-
brew install ffmpeg
50-
brew install pkg-config
51-
```
52-
53-
### Android
54-
55-
#### Build on Android device using Termux
56-
We found that build on Android device would bring better runtime performance, so we recommend to build on device.
57-
58-
[Termux](https://github.com/termux/termux-app#installation) is a terminal app on Android device (no root required).
59-
60-
Install tools in Termux:
61-
```
62-
apt update && apt upgrade -y
63-
apt install git make cmake
64-
```
65-
66-
It's recommended to move your model inside the `~/` directory for best performance:
67-
```
68-
cd storage/downloads
69-
mv model.gguf ~/
70-
```
71-
72-
#### Building the Project using Android NDK
73-
Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake.
74-
75-
Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux:
76-
77-
```bash
78-
mkdir build-android
79-
cd build-android
80-
export NDK=/your_ndk_path
81-
cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
82-
make
83-
```
84-
85-
Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice).
86-
87-
Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission:
88-
89-
(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`)
90-
```
91-
$cp -r /sdcard/llama.cpp/bin /data/data/com.termux/files/home/
92-
$cd /data/data/com.termux/files/home/bin
93-
$chmod +x ./*
94-
```
95-
96-
Download models and push them to `/sdcard/llama.cpp/`, then move it to `/data/data/com.termux/files/home/model/`
97-
98-
```
99-
$mv /sdcard/llama.cpp/ggml-model-Q4_K_M.gguf /data/data/com.termux/files/home/model/
100-
$mv /sdcard/llama.cpp/mmproj-model-f16.gguf /data/data/com.termux/files/home/model/
101-
```
102-
103-
Now, you can start chatting:
104-
```
105-
$cd /data/data/com.termux/files/home/bin
106-
$./llama-minicpmv-cli -m ../model/ggml-model-Q4_K_M.gguf --mmproj ../model/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
46+
./build/bin/llama-minicpmv-cli -m ../MiniCPM-V-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-V-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"
10747
```

0 commit comments

Comments
 (0)