Skip to content

Commit 3340407

Browse files
committed
Update README
1 parent 2676638 commit 3340407

File tree

1 file changed

+27
-1
lines changed

1 file changed

+27
-1
lines changed

README.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,28 @@
11
# llama.cpp-static
2-
Static builds of llama.cpp (Currently only amd64 server builds are available)
2+
Your daily, minimal build of llama.cpp. [Also available on Docker Hub.](https://hub.docker.com/r/ezforever/llama.cpp-static)
3+
4+
Source code: <https://github.com/ggerganov/llama.cpp>
5+
Built from: <https://github.com/EZForever/llama.cpp-static>
6+
7+
## Usage
8+
9+
Please refer to [llama.cpp docker guide](https://github.com/ggerganov/llama.cpp/blob/master/README.md#docker) and [server README](https://github.com/ggerganov/llama.cpp/tree/master/examples/server).
10+
11+
## Tag format
12+
13+
**tl;dr:** Use `server-ssl-avx2` if you don't know what you're doing.
14+
15+
Server images are tagged in the format of `server-<ssl>-<avx>`.
16+
17+
`<ssl>` is one of the following:
18+
19+
- `nossl`: Minimal build with no SSL/TLS capability.
20+
- `ssl`: Built with OpenSSL (`LLAMA_SERVER_SSL=ON`), thus supports `--ssl-key-file` and `--ssl-cert-file`.
21+
22+
`<avx>` is one of the following:
23+
24+
- `noavx`: All AVX-related optimizations are disabled. Do not use this build unless you are working around some known bug, or running LLMs on a 10-year-old potato.
25+
- `avx`: (Only) AVX instruction set is enabled. Might be useful if you are using some old CPUs that don't support AVX2.
26+
- `avx2`: AVX2 instruction set is enabled. This build should support most modern/recent CPUs with reasonable performance.
27+
- `avx512`: AVX512 and AVX512-VNNI instruction sets are enabled. Currently only some high-end or server-grade CPUs support these instruction sets, so check your hardware specs before using this build.
28+

0 commit comments

Comments
 (0)