|
| 1 | + |
| 2 | + |
| 3 | +Benchmarks Metadata: |
| 4 | + Run id:93e36b31-b454-471d-ba62-6b2671585485 |
| 5 | + Duration:30.2 seconds |
| 6 | + Profile:type=sweep, strategies=['synchronous', 'throughput', 'constant', |
| 7 | + 'constant', 'constant', 'constant', 'constant', 'constant', 'constant', |
| 8 | + 'constant'], max_concurrency=None |
| 9 | + Args:max_number=None, max_duration=30.0, warmup_number=None, |
| 10 | + warmup_duration=None, cooldown_number=None, cooldown_duration=None |
| 11 | + Worker:type_='generative_requests_worker' backend_type='openai_http' |
| 12 | + backend_target='example_target' backend_model='example_model' |
| 13 | + backend_info={'max_output_tokens': 16384, 'timeout': 300, 'http2': True, |
| 14 | + 'authorization': False, 'organization': None, 'project': None, |
| 15 | + 'text_completions_path': '/v1/completions', 'chat_completions_path': |
| 16 | + '/v1/chat/completions'} |
| 17 | + Request Loader:type_='generative_request_loader' |
| 18 | + data='prompt_tokens=256,output_tokens=128' data_args=None |
| 19 | + processor='example_processor' processor_args=None |
| 20 | + Extras:None |
| 21 | + |
| 22 | + |
| 23 | +Benchmarks Info: |
| 24 | +================================================================================ |
| 25 | +=================================================================== |
| 26 | +Metadata |||| Requests Made ||| Prompt |
| 27 | +Tok/Req ||| Output Tok/Req ||| Prompt Tok Total||| Output Tok Total || |
| 28 | + Benchmark| Start Time| End Time| Duration (s)| Comp| Inc| Err| Comp| |
| 29 | +Inc| Err| Comp| Inc| Err| Comp| Inc| Err| Comp| Inc| Err |
| 30 | +-----------|-----------|---------|-------------|------|-----|-----|------|------ |
| 31 | +|----|-------|-----|-----|-------|-----|-----|-------|------|------ |
| 32 | +synchronous| 16:59:28| 16:59:58| 30.0| 46| 1| 0| 257.1| |
| 33 | +256.0| 0.0| 128.0| 0.0| 0.0| 11827| 256| 0| 5888| 0| 0 |
| 34 | +================================================================================ |
| 35 | +=================================================================== |
| 36 | + |
| 37 | + |
| 38 | +Benchmarks Stats: |
| 39 | +================================================================================ |
| 40 | +=============================================================== |
| 41 | +Metadata | Request Stats || Out Tok/sec| Tot Tok/sec| Req Latency |
| 42 | +(sec) ||| TTFT (ms) ||| ITL (ms) ||| TPOT (ms) || |
| 43 | + Benchmark| Per Second| Concurrency| mean| mean| mean| median| |
| 44 | +p99| mean| median| p99| mean| median| p99| mean| median| p99 |
| 45 | +-----------|-----------|------------|------------|------------|------|--------|- |
| 46 | +-----|-----|-------|-----|-----|-------|----|-----|-------|---- |
| 47 | +synchronous| 1.55| 1.00| 198.1| 992.7| 0.64| 0.64| |
| 48 | +0.69| 16.8| 16.4| 21.3| 4.9| 4.9| 5.3| 4.9| 4.9| 5.2 |
| 49 | +================================================================================ |
| 50 | +=============================================================== |
0 commit comments