You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[📚 Related Documentation](#-related-documentation)
41
+
-[👥 Contributors](#-contributors)
42
+
-[📄 Open Source License](#-open-source-license)
43
+
18
44
## 📋 Project Overview
19
45
20
46
LMeterX is a professional large language model performance testing platform that can be applied to model inference services based on large model inference frameworks (such as LiteLLM, vLLM, TensorRT-LLM, LMDeploy, and others), and also supports performance testing for cloud services like Azure OpenAI, AWS Bedrock, Google Vertex AI, and other major cloud providers. Through an intuitive Web interface, users can easily create and manage test tasks, monitor testing processes in real-time, and obtain detailed performance analysis reports, providing reliable data support for model deployment and performance optimization.
@@ -26,11 +52,11 @@ LMeterX is a professional large language model performance testing platform that
26
52
## ✨ Core Features
27
53
28
54
-**Universal Framework Support** - Compatible with mainstream inference frameworks (vLLM, LiteLLM, TensorRT-LLM) and cloud services (Azure, AWS, Google Cloud)
29
-
-**Full Model Compatibility** - Supports mainstream LLMs like GPT, Claude, and Llama with one-click stress testing
55
+
-**Full Model Compatibility** - Supports mainstream LLMs like GPT, Claude, and Llama, also supports large document parsing models such as [MinerU](https://github.com/opendatalab/MinerU) and [dots.ocr](https://github.com/rednote-hilab/dots.ocr).
30
56
-**High-Load Stress Testing** - Simulates high-concurrency requests to accurately detect model performance limits
-**Professional Metrics**- Core performance metrics including first token latency, throughput(RPS、TPS), and success rate
59
+
-**AI Smart Reports** <imgsrc="docs/images/badge-new.svg"alt="NEW"height="16" /> - AI-powered performance analysis, multi-dimensional model comparison and visualization
34
60
-**Web Console** - One-stop management for task creation, stopping, status tracking, and full-chain log monitoring
35
61
-**Enterprise-level Deployment** - Docker containerization with elastic scaling and distributed deployment support
36
62
@@ -58,64 +84,62 @@ LMeterX adopts a microservices architecture design, consisting of four core comp
58
84
59
85
## 🚀 Quick Start
60
86
61
-
### Environment Requirements
62
-
- Docker 20.10.0+
63
-
- Docker Compose 2.0.0+
64
-
- At least 4GB available memory
65
-
- At least 5GB available disk space
66
-
67
-
### One-Click Deployment (Recommended)
87
+
### Environment Checklist
88
+
- Docker 20.10.0+ with the daemon running
89
+
- Docker Compose 2.0.0+ (`docker compose` plugin or standalone `docker-compose`)
90
+
- At least 4GB free memory and 5GB disk space
68
91
69
-
> **Complete Deployment Guide**: See [Complete Deployment Guide](docs/DEPLOYMENT_GUIDE.md) for detailed instructions on all deployment methods
92
+
> **Need more deployment options?** See the [Complete Deployment Guide](docs/DEPLOYMENT_GUIDE.md) for Kubernetes, air-gapped installs, and advanced tuning.
70
93
71
-
Use pre-built Docker images to start all services with one click:
94
+
### One-Click Deployment (Recommended)
72
95
73
96
```bash
74
-
# Download and run one-click deployment script
97
+
# Download and run the one-click deployment script
# Start 2 backends + 2 engines (the number can be adjusted as needed)
85
-
docker compose up -d --scale backend=2 --scale engine=2
86
-
```
107
+
### Data & Volume Layout
108
+
-`./data` → mounted to `/app/data` in the `engine` service (large datasets are **not** baked into the image)
109
+
-`./logs` → shared log output for backend and engine
110
+
-`./upload_files` → user-supplied payloads and exported reports
111
+
112
+
For custom data, please refer to the [Dataset Usage Guide](docs/DATASET_GUIDE.md).
87
113
88
114
### Usage Guide
89
115
90
116
1.**Access Web Interface**: Open http://localhost:8080
91
-
2.**Create Test Task**: Navigate to Test Tasks → Create Task, configure LLM API request information, test data, and request-response field mapping
92
-
- 2.1 Basic Information: For `/chat/completions` API, you only need to configure API path, model, and response mode. You can also supplement the complete payload in request parameters.
93
-
- 2.2 Data Payload: Select built-in text datasets/multimodal datasets as needed, or upload custom JSONL/ShareGPT dataset files.
94
-
- 2.3 Field Mapping: Configure the prompt field path in payload, and response data paths for model output content, reasoning_content fields, usage fields, etc. This field mapping is crucial for updating request parameters with datasets and correctly parsing streaming/non-streaming responses.
117
+
2.**Create Test Task**: Navigate to Test Tasks → Create Task, configure API request information, test data, and request/response field mappings.
118
+
- 2.1 Basic Information: For OpenAI-like and Claude-like APIs, you only need to configure API path, model, and response mode. You can also supplement the complete payload in request parameters.
119
+
- 2.2 Data & load: Select the dataset type, concurrency, load testing time, etc., as needed.
120
+
- 2.3 Field Mapping: For custom APIs, you need to configure the prompt field path in payload, and response data paths for model output fields, usage fields, etc. This field mapping is crucial for updating request parameters with datasets and correctly parsing streaming/non-streaming responses.
95
121
> 💡 **Tip**: For custom multimodal dataset load tests, follow the [Dataset Guide](docs/DATASET_GUIDE.md) for data preparation, mounting, and troubleshooting.
96
-
3.**API Testing**: In Test Tasks → Create Task, click the "Test" button in the Basic Information panel to quickly test API connectivity
97
-
**Note**: For quick API response, it's recommended to use simple prompts when testing API connectivity.
122
+
3.**API Testing**: In Test Tasks → Create Task, click the "Test" button in the Basic Information panel to quickly test API connectivity (use a lightweight prompt for faster feedback).
98
123
4.**Real-time Monitoring**: Navigate to Test Tasks → Logs/Monitoring Center to view full-chain test logs and troubleshoot exceptions
99
124
5.**Result Analysis**: Navigate to Test Tasks → Results to view detailed performance results and export reports
100
125
6.**Result Comparison**: Navigate to Model Arena to select multiple models or versions for multi-dimensional performance comparison
101
126
7.**AI Analysis**: In Test Tasks → Results/Model Arena, after configuring AI analysis service, support intelligent performance evaluation for single/multiple tasks
0 commit comments