We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 592c289 commit 9ba8a48Copy full SHA for 9ba8a48
common/performance/client.yml
@@ -0,0 +1,12 @@
1
+data:
2
+ prompt_tokens: 512
3
+ prompt_tokens_stdev: 128
4
+ prompt_tokens_min: 1
5
+ prompt_tokens_max: 1024
6
+ output_tokens: 256
7
+ output_tokens_stdev: 64
8
+ output_tokens_min: 1
9
+ output_tokens_max: 1024
10
+rate-type: sweep
11
+max-seconds: 400
12
+warmup-percent: 0.2
common/performance/server.yml
@@ -0,0 +1,5 @@
+uvicorn-log-level: "debug"
+trust-remote-code: true
+enable-chunked-prefill: true
+tensor-parallel-size: 1
+max-model-len: 4096
0 commit comments