Skip to content

Commit f1377af

Browse files
authored
Updated quantizing tool guide to compile with Visual Studio 2022 (#288)
1 parent b1350f6 commit f1377af

File tree

1 file changed

+27
-0
lines changed

1 file changed

+27
-0
lines changed

tools/README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,40 @@ git apply ..\lcpp.patch
4040

4141
Compile the llama-quantize binary. This example uses cmake, on linux you can just use make.
4242

43+
### Visual Studio 2019, Linux, etc...
44+
4345
```
4446
mkdir build
4547
cmake -B build
4648
cmake --build build --config Debug -j10 --target llama-quantize
4749
cd ..
4850
```
4951

52+
### Visual Studio 2022
53+
54+
```
55+
mkdir build
56+
cmake -B build -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_FLAGS="-std=c++17"
57+
```
58+
59+
Edit the `llama.cpp\common\log.cpp` file, inserts two lines after the existing first line:
60+
61+
```
62+
#include "log.h"
63+
64+
#define _SILENCE_CXX23_CHRONO_DEPRECATION_WARNING
65+
#include <chrono>
66+
```
67+
68+
Then you can build the project:
69+
```
70+
cmake --build build --config Debug -j10 --target llama-quantize
71+
cd ..
72+
```
73+
74+
### Quantize your model
75+
76+
5077
Now you can use the newly build binary to quantize your model to the desired format:
5178
```
5279
llama.cpp\build\bin\Debug\llama-quantize.exe E:\models\unet\flux1-dev-BF16.gguf E:\models\unet\flux1-dev-Q4_K_S.gguf Q4_K_S

0 commit comments

Comments
 (0)