Skip to content

Commit afef6a4

Browse files
committed
added n_predict tests
1 parent 7867c95 commit afef6a4

File tree

3 files changed

+33
-28
lines changed

3 files changed

+33
-28
lines changed
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
@llama.cpp
2+
@n_predict
3+
Feature: llama.cpp server
4+
5+
Background: Server startup
6+
Given a server listening on localhost:8080
7+
And a model file test-model.gguf
8+
And a model alias tinyllama-2
9+
And 42 as server seed
10+
And 64 KV cache size
11+
12+
Scenario: Generate N tokens
13+
And 12 max tokens to predict
14+
Then the server is starting
15+
Then the server is healthy
16+
Given a prompt:
17+
"""
18+
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
19+
"""
20+
And a completion request with no api error
21+
Then 12 tokens are predicted
22+
23+
Scenario: Generate tokens until context is full
24+
And -2 server max tokens to predict
25+
Then the server is starting
26+
Then the server is healthy
27+
Given a prompt:
28+
"""
29+
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
30+
"""
31+
And a completion request with no api error
32+
Then 11 tokens are predicted

examples/server/tests/features/num_tokens.feature

Lines changed: 0 additions & 27 deletions
This file was deleted.

examples/server/tests/features/steps/steps.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ def step_n_slots(context, n_slots: int):
159159

160160
@step('{n_predict:d} server max tokens to predict')
161161
def step_server_n_predict(context, n_predict: int):
162-
context.n_server_predict = n_predict if n_predict > 0 else None
162+
context.n_server_predict = n_predict if n_predict > 0 or n_predict in (-1, -2) else None
163163

164164

165165
@step('{slot_save_path} as slot save path')

0 commit comments

Comments
 (0)