Skip to content

Commit 6ab8eac

Browse files
authored
examples : add -kvu to batched usage example [no ci] (ggml-org#17469)
This commit adds the --kv-unified flag to the usage example in the README.md file for the batched example. The motivation for this is that without this flag the example will fail with the following error: ```console Hello my name is split_equal: sequential split is not supported when there are coupled sequences in the input batch (you may need to use the -kvu flag) decode: failed to find a memory slot for batch of size 4 main: llama_decode() failed ```
1 parent 2d50b9d commit 6ab8eac

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/batched/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
The example demonstrates batched generation from a given prompt
44

55
```bash
6-
./llama-batched -m ./models/llama-7b-v2/ggml-model-f16.gguf -p "Hello my name is" -np 4
6+
./llama-batched -m ./models/llama-7b-v2/ggml-model-f16.gguf -p "Hello my name is" -np 4 --kv-unified
77

88
...
99

0 commit comments

Comments
 (0)