|
1 | 1 | <div align="center"> |
2 | 2 | <img src="docs/images/logo.png" alt="LMeterX Logo" width="400"/> |
| 3 | + <p> |
| 4 | + <a href="https://github.com/MigoXLab/LMeterX/blob/main/LICENSE"><img src="https://img.shields.io/github/license/MigoXLab/LMeterX" alt="License"></a> |
| 5 | + <a href="https://github.com/MigoXLab/LMeterX/stargazers"><img src="https://img.shields.io/github/stars/MigoXLab/LMeterX" alt="GitHub stars"></a> |
| 6 | + <a href="https://github.com/MigoXLab/LMeterX/network/members"><img src="https://img.shields.io/github/forks/MigoXLab/LMeterX" alt="GitHub forks"></a> |
| 7 | + <a href="https://github.com/MigoXLab/LMeterX/issues"><img src="https://img.shields.io/github/issues/MigoXLab/LMeterX" alt="GitHub issues"></a> |
| 8 | + <a href="https://deepwiki.com/MigoXLab/LMeterX"><img src="https://deepwiki.com/badge.svg" alt="Ask DeepWiki"></a> |
| 9 | + </p> |
3 | 10 | <p> |
4 | 11 | <a href="README_CN.md">简体中文</a> | |
5 | 12 | <strong>English</strong> |
|
10 | 17 |
|
11 | 18 | ## 📋 Project Overview |
12 | 19 |
|
13 | | -LMeterX is a professional large language model performance testing platform that supports comprehensive load testing for any LLM service. Through an intuitive Web interface, users can easily create and manage test tasks, monitor testing processes in real-time, and obtain detailed performance analysis reports, providing reliable data support for model deployment and performance optimization. |
| 20 | +LMeterX is a professional large language model performance testing platform that can be applied to model inference services based on large model inference frameworks (such as LiteLLM, vLLM, TensorRT-LLM, LMDeploy, and others), and also supports performance testing for cloud services like Azure OpenAI, AWS Bedrock, Google Vertex AI, and other major cloud providers. Through an intuitive Web interface, users can easily create and manage test tasks, monitor testing processes in real-time, and obtain detailed performance analysis reports, providing reliable data support for model deployment and performance optimization. |
14 | 21 |
|
15 | 22 | <div align="center"> |
16 | 23 | <img src="docs/images/images.gif" alt="LMeterX Demo" width="800"/> |
17 | 24 | </div> |
18 | 25 |
|
19 | 26 | ## ✨ Core Features |
20 | 27 |
|
| 28 | +- **Universal Framework Support** - Compatible with mainstream inference frameworks (vLLM, LiteLLM, TensorRT-LLM) and cloud services (Azure, AWS, Google Cloud) |
21 | 29 | - **Full Model Compatibility** - Supports mainstream LLMs like GPT, Claude, and Llama with one-click stress testing |
22 | 30 | - **High-Load Stress Testing** - Simulates high-concurrency requests to accurately detect model performance limits |
23 | 31 | - **Multi-Scenario Coverage** - Supports streaming/non-streaming, supports text/multimodal/custom datasets<sup></sup> |
@@ -59,38 +67,52 @@ curl -fsSL https://raw.githubusercontent.com/MigoXLab/LMeterX/main/quick-start.s |
59 | 67 | ``` |
60 | 68 | ### Usage Guide |
61 | 69 |
|
62 | | -1. **Access Web Interface**: http://localhost:8080 |
63 | | -2. **Create Test Task**: |
64 | | - - Configure target API address and model parameters |
65 | | - - Select test type (text conversation/image-text conversation) |
66 | | - - Set concurrent user count and test duration |
67 | | - - Configure other advanced parameters (optional) |
68 | | -3. **Monitor Test Process**: Real-time view of test logs and performance metrics |
69 | | -4. **View and Export Test Results**: View detailed performance results and export reports. |
70 | | -5. **AI Summary**: After configuring the AI service on the System Configuration page, you can perform AI-powered evaluation and summary of performance results on the Task Results page. |
| 70 | +1. **Access Web Interface**: Open http://localhost:8080 |
| 71 | +2. **Create Test Task**: Navigate to Test Tasks → Create Task, configure LLM API request information, test data, and request-response field mapping |
| 72 | + **2.1 Basic Information**: For `/chat/completions` API, you only need to configure API path, model, and response mode. You can also supplement the complete payload in request parameters. |
| 73 | + **2.2 Data Payload**: Select built-in text datasets/multimodal datasets as needed, or upload custom JSONL data files. |
| 74 | + **2.3 Field Mapping**: Configure the prompt field path in payload, and response data paths for model output content, reasoning_content fields, usage fields, etc. This field mapping is crucial for updating request parameters with datasets and correctly parsing streaming/non-streaming responses. |
| 75 | +3. **API Testing**: In Test Tasks → Create Task, click the "Test" button in the Basic Information panel to quickly test API connectivity |
| 76 | + **Note**: For quick API response, it's recommended to use simple prompts when testing API connectivity. |
| 77 | +4. **Real-time Monitoring**: Navigate to Test Tasks → Logs/Monitoring Center to view full-chain test logs and troubleshoot exceptions |
| 78 | +5. **Result Analysis**: Navigate to Test Tasks → Results to view detailed performance results and export reports |
| 79 | +6. **Result Comparison**: Navigate to Model Arena to select multiple models or versions for multi-dimensional performance comparison |
| 80 | +7. **AI Analysis**: In Test Tasks → Results/Model Arena, after configuring AI analysis service, support intelligent performance evaluation for single/multiple tasks |
71 | 81 |
|
72 | 82 | ## 🔧 Configuration |
73 | 83 |
|
74 | 84 | ### Environment Variable Configuration |
75 | 85 |
|
76 | 86 | #### General Configuration |
77 | 87 | ```bash |
78 | | -SECRET_KEY=your_secret_key_here # Application security key |
79 | | -FLASK_DEBUG=false # Debug mode switch |
80 | | -``` |
| 88 | +# ================= Database Configuration ================= |
| 89 | +DB_HOST=mysql # Database host (container name or IP) |
| 90 | +DB_PORT=3306 # Database port |
| 91 | +DB_USER=lmeterx # Database username |
| 92 | +DB_PASSWORD=lmeterx_password # Database password (use secrets management in production) |
| 93 | +DB_NAME=lmeterx # Database name |
| 94 | + |
| 95 | +# ================= Frontend Configuration ================= |
| 96 | +VITE_API_BASE_URL=/api # Base API URL for frontend requests (supports reverse proxy) |
| 97 | + |
| 98 | +# ================= High-Concurrency Load Testing Deployment Requirements ================= |
| 99 | +# When concurrent users exceed this threshold, the system will automatically enable multi-process mode (requires multi-core CPU support) |
| 100 | +MULTIPROCESS_THRESHOLD=1000 |
| 101 | + |
| 102 | +# Minimum number of concurrent users each child process should handle (prevents excessive processes and resource waste) |
| 103 | +MIN_USERS_PER_PROCESS=600 |
| 104 | + |
| 105 | +# ⚠️ IMPORTANT NOTES: |
| 106 | +# - When concurrency ≥ 1000, enabling multi-process mode is strongly recommended for performance. |
| 107 | +# - Multi-process mode requires multi-core CPU resources — ensure your deployment environment meets these requirements. |
| 108 | + |
| 109 | +# ================= Deployment Resource Limits ================= |
| 110 | +deploy: |
| 111 | + resources: |
| 112 | + limits: |
| 113 | + cpus: '2.0' # Recommended minimum: 2 CPU cores (4+ cores recommended for high-concurrency scenarios) |
| 114 | + memory: 2G # Memory limit — adjust based on actual load (minimum recommended: 2G) |
81 | 115 |
|
82 | | -#### Database Configuration |
83 | | -```bash |
84 | | -DB_HOST=mysql # Database host address |
85 | | -DB_PORT=3306 # Database port |
86 | | -DB_USER=lmeterx # Database username |
87 | | -DB_PASSWORD=lmeterx_password # Database password |
88 | | -DB_NAME=lmeterx # Database name |
89 | | -``` |
90 | | - |
91 | | -#### Frontend Configuration |
92 | | -```bash |
93 | | -VITE_API_BASE_URL=/api # API base path |
94 | 116 | ``` |
95 | 117 |
|
96 | 118 | ## 🤝 Development Guide |
@@ -135,6 +157,7 @@ LMeterX/ |
135 | 157 |
|
136 | 158 | ### Planned |
137 | 159 | - [ ] CLI command-line tool |
| 160 | +- [ ] Support for `/v1/embedding` and `/v1/rerank` API stress testing |
138 | 161 |
|
139 | 162 | ## 📚 Related Documentation |
140 | 163 |
|
|
0 commit comments