Skip to content

Commit 1bb8483

Browse files
authored
Update performance config files (#5)
* Drop `model` key from server config * Drop perf client fields * Add data-type to client config * Fix data sub-arg
1 parent aff567f commit 1bb8483

File tree

2 files changed

+2
-5
lines changed

2 files changed

+2
-5
lines changed
Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
1-
target: "http://localhost:8000/v1"
2-
model: "meta-llama/Llama-3.1-8B-Instruct"
31
data:
42
prompt_tokens: 64
5-
output_tokens: 16
3+
generated_tokens: 16
4+
data-type: emulated
65
rate-type: throughput
76
max-seconds: 400
8-
output_path: ""

meta-llama/Llama-3.1-8B-Instruct/performance/server.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
# server configs for https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct
22
# config.yaml
3-
model: meta-llama/Llama-3.1-8B-Instruct
43
uvicorn-log-level: "debug"
54
trust-remote-code: true
65
enable-chunked-prefill: true

0 commit comments

Comments
 (0)