Skip to content

Commit 9ba8a48

Browse files
authored
Create common performance configs (#22)
1 parent 592c289 commit 9ba8a48

File tree

2 files changed

+17
-0
lines changed

2 files changed

+17
-0
lines changed

common/performance/client.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
data:
2+
prompt_tokens: 512
3+
prompt_tokens_stdev: 128
4+
prompt_tokens_min: 1
5+
prompt_tokens_max: 1024
6+
output_tokens: 256
7+
output_tokens_stdev: 64
8+
output_tokens_min: 1
9+
output_tokens_max: 1024
10+
rate-type: sweep
11+
max-seconds: 400
12+
warmup-percent: 0.2

common/performance/server.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
uvicorn-log-level: "debug"
2+
trust-remote-code: true
3+
enable-chunked-prefill: true
4+
tensor-parallel-size: 1
5+
max-model-len: 4096

0 commit comments

Comments
 (0)