We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 3a64932 commit c643de8Copy full SHA for c643de8
examples/server/tests/features/embeddings.feature
@@ -10,7 +10,7 @@ Feature: llama.cpp server
10
And 42 as server seed
11
And 2 slots
12
# the bert-bge-small model has context size of 512
13
- # since the generated prompts are as big as the batch size, we need to set the batch size to 512
+ # since the generated prompts are as big as the batch size, we need to set the batch size to <= 512
14
# ref: https://huggingface.co/BAAI/bge-small-en-v1.5/blob/5c38ec7c405ec4b44b94cc5a9bb96e735b38267a/config.json#L20
15
And 128 as batch size
16
And 128 as ubatch size
0 commit comments