File tree Expand file tree Collapse file tree 2 files changed +18
-4
lines changed
cloud-infrastructure/ai-infra-gpu/ai-infrastructure/llm-benchmark-docker Expand file tree Collapse file tree 2 files changed +18
-4
lines changed Original file line number Diff line number Diff line change @@ -125,6 +125,24 @@ are gated and require an access token.
125125 results in the directory ` ./results ` , containing information about the vLLM
126126 parameters and the shape used.
127127
128+ To run only certain scenarios and concurrent request settings, modify
129+ [ ` compose.yaml ` ] ( files/compose.yaml ) and have the ` command ` for the ` perf `
130+ container read, i.e:
131+ ``` yaml
132+ command :
133+ - " wait-for-it.sh"
134+ - " --timeout=300"
135+ - " llm:8000"
136+ - " --"
137+ - " /appli/scripts/benchmark.py"
138+ - " --scenario"
139+ - " chatbot"
140+ - " --concurrency"
141+ - " 1"
142+ - " 4"
143+ - " 16"
144+ ` ` `
145+
1281465. Run the plotting:
129147 ` ` ` sh
130148 docker-compose run plot
Original file line number Diff line number Diff line change @@ -32,10 +32,6 @@ services:
3232 - " llm:8000"
3333 - " --"
3434 - " /appli/scripts/benchmark.py"
35- - " --concurrency"
36- - " 1"
37- - " 4"
38- - " 16"
3935 plot :
4036 build : plot
4137 container_name : plot
You can’t perform that action at this time.
0 commit comments