|
3 | 3 |
|
4 | 4 |  |
5 | 5 |
|
| 6 | +A highly configurable chat application for running LLMs, based on [llama.cpp](https://github.com/ggerganov/llama.cpp). |
| 7 | + |
6 | 8 | This project started from the main example of llama.cpp - the idea was to read parameters from .json files. Now it's a class, separate threads for running llama.cpp, structs for managing and settings - and only then wrapped into UI. |
7 | 9 |
|
8 | 10 | Additionally, chatTest console app is also added for testing purposes, since certain information (like memory used by the model) is printed outside of the original example (in llama.cpp, common.cpp, etc.). |
@@ -36,17 +38,21 @@ Tested on Windows only for now. AVX2 releases only for now, older releases were |
36 | 38 |
|
37 | 39 | ### Requirements |
38 | 40 |
|
| 41 | +Code: |
39 | 42 | * tinyfiledialogs https://sourceforge.net/projects/tinyfiledialogs/ |
40 | | -* Vulkan SDK (installer is preferred) https://vulkan.lunarg.com/#new_tab |
41 | 43 | * imgui https://github.com/ocornut/imgui |
| 44 | + |
| 45 | +Libraries: |
| 46 | +* Vulkan SDK (installer is preferred) https://vulkan.lunarg.com/#new_tab |
42 | 47 | * SDL2 https://github.com/libsdl-org/SDL |
43 | | -* OpenCL and CLBLAST if needed (see https://github.com/ggerganov/llama.cpp#clblast for installation guide) |
| 48 | +* If CLBLAST is needed: use releases of [OpenCL SDK](https://github.com/KhronosGroup/OpenCL-SDK) and [Clblast](https://github.com/CNugteren/CLBlast). |
44 | 49 |
|
45 | 50 | ### Building on Windows |
46 | 51 |
|
47 | 52 | * Download this repo or `git clone` it |
48 | 53 | * Use w64devkit https://github.com/skeeto/w64devkit/releases |
49 | | -* Install all prerequisites according to w64devkit installation |
| 54 | +* Install all prerequisite libraries according to w64devkit installation (which is, in most cases, copying `lib` and `include` folders into `w64devkit/x86_64-w64-mingw32`) |
| 55 | +* Download and copy all prerequisite code according to Makefile |
50 | 56 | * Launch w64devkit.exe and navigate to the project folder |
51 | 57 | * `make chat` for CPU-only |
52 | 58 | * `make chat_cl` for Clblast build |
|
0 commit comments