You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The tool supports three different test modes to measure different aspects of API performance:
88
+
89
+
#### Streaming Mode (Default)
90
+
Tests regular chat completion with streaming responses. This is the default mode.
91
+
92
+
```bash
93
+
# Explicit streaming mode (default behavior)
94
+
./llm-api-speed --provider nim
95
+
```
96
+
97
+
#### Tool-Calling Mode
98
+
Tests the API's tool/function calling capabilities with streaming. Measures performance when the model needs to generate tool calls.
99
+
100
+
```bash
101
+
# Test tool-calling performance
102
+
./llm-api-speed --provider nim --tool-calling
103
+
```
104
+
105
+
#### Mixed Mode
106
+
Runs 3 iterations of both streaming and tool-calling modes (6 total runs). Provides comprehensive performance metrics for both use cases.
107
+
108
+
```bash
109
+
# Test both streaming and tool-calling
110
+
./llm-api-speed --provider nim --mixed
111
+
112
+
# Test all providers with both modes
113
+
./llm-api-speed --all --mixed
114
+
```
115
+
116
+
### Diagnostic Mode
117
+
118
+
Diagnostic mode runs intensive stress testing with 10 concurrent workers for 1 minute, making requests every 15 seconds with a 30-second timeout per request. Perfect for:
119
+
- Load testing your API endpoints
120
+
- Identifying rate limits and throttling behavior
121
+
- Measuring performance under sustained concurrent load
122
+
- Debugging intermittent issues
123
+
124
+
```bash
125
+
# Run diagnostic mode with streaming
126
+
./llm-api-speed --provider nim --diagnostic
127
+
128
+
# Run diagnostic mode with tool-calling
129
+
./llm-api-speed --provider nim --diagnostic --tool-calling
130
+
131
+
# Run diagnostic mode with mixed workload (alternates between streaming and tool-calling)
132
+
./llm-api-speed --provider nim --diagnostic --mixed
133
+
```
134
+
135
+
Diagnostic mode produces:
136
+
- Detailed per-request logs for each worker
137
+
- Aggregated success/failure statistics
138
+
- Average metrics across all successful requests
139
+
- Error frequency analysis
140
+
- JSON summary file with all metrics
141
+
81
142
## Output
82
143
83
144
Each test run creates a session folder: `results/session-YYYYMMDD-HHMMSS/`
0 commit comments