File tree Expand file tree Collapse file tree 1 file changed +6
-5
lines changed Expand file tree Collapse file tree 1 file changed +6
-5
lines changed Original file line number Diff line number Diff line change @@ -104,11 +104,12 @@ type Configuration struct {
104104 // KVCacheTransferOverheadStdDev similar to TimeToFirstTokenStdDev
105105 KVCacheTransferTimeStdDev int `yaml:"kv-cache-transfer-time-std-dev" json:"kv-cache-transfer-time-std-dev"`
106106
107- // TimeFactorUnderLoad is a multiplicative factor that affects the overall time taken for requests
108- // when parallel requests are being processed. The value of this factor must be >= 1.0, with a default of 1.0.
109- // If this factor is 1.0, no extra time is added. When the factor is x (where x > 1.0) and there are MaxNumSeqs
110- // requests, the total time will be multiplied by x.
111- // The extra time then decreases multiplicatively to 1.0 when the number of requests is less than MaxNumSeqs.
107+ // TimeFactorUnderLoad is a multiplicative factor that affects the overall time taken for requests when parallel
108+ // requests are being processed.
109+ // The value of this factor must be >= 1.0, with a default of 1.0.
110+ // - If this factor is 1.0, no extra time is added.
111+ // - When the factor is x (where x > 1.0) and there are MaxNumSeqs requests, the total time will be multiplied by x.
112+ // - The extra time then decreases multiplicatively to 1.0 when the number of requests is less than MaxNumSeqs.
112113 TimeFactorUnderLoad float64 `yaml:"time-factor-under-load" json:"time-factor-under-load"`
113114
114115 // Mode defines the simulator response generation mode, valid values: echo, random
You can’t perform that action at this time.
0 commit comments