You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+20-8Lines changed: 20 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -54,9 +54,9 @@ vibecoding, vibe coding, web evaluation, autonomous exploration, web testing aut
54
54
55
55
### 📋 Feature Highlights
56
56
57
-
-**🤖 AI-Powered Testing**: Performs autonomous website testing—explores pages, plans actions, and executes end-to-end flows without manual scripting.
58
-
-**📊 Multi-Dimensional Observation**: Covers functionality, performance, user experience, and basic security; evaluates load speed, design details, and links to surface issues.
59
-
-**🎯 Actionable Recommendations**: Runs in real browsers and provides concrete suggestions for improvement.
57
+
-**🤖 AI-Powered Testing**: Performs autonomous website testing with intelligent planning and reflection—explores pages, plans actions, and executes end-to-end flows without manual scripting. Features 2-stage architecture (lightweight filtering + comprehensive planning) and dynamic test generation for newly appeared UI elements.
58
+
-**📊 Multi-Dimensional Observation**: Covers functionality, performance, user experience, and basic security; evaluates load speed, design details, and links to surface issues. Uses multi-modal analysis (screenshots + DOM structure + text content) and DOM diff detection to discover new test opportunities.
59
+
-**🎯 Actionable Recommendations**: Runs in real browsers with smart element prioritization and automatic viewport management. Provides concrete suggestions for improvement with adaptive recovery mechanisms for robust test execution.
60
60
-**📈 Visual Reports**: Generates detailed HTML test reports with clear, multi-dimensional views for analysis and tracking.
@@ -145,8 +146,8 @@ test_config: # Test configuration
145
146
business_objectives: example business objectives # Recommended to include test scope, e.g., test search functionality
146
147
dynamic_step_generation: # Optional, configuration for dynamic steps generation
147
148
enabled: True # Optional, default False, recommended to set True to enable dynamic step generation
148
-
max_dynamic_steps: 5# Optional, default 5 test steps generated per trigger
149
-
min_elements_threshold: 2# Optional, default trigger threshold is 2 DOM element differences
149
+
max_dynamic_steps: 10# Optional, default 5, this example uses 10
150
+
min_elements_threshold: 1# Optional, default 2, this example uses 1 for higher sensitivity
150
151
ux_test: # User experience testing
151
152
enabled: True
152
153
performance_test: # Performance analysis
@@ -155,28 +156,39 @@ test_config: # Test configuration
155
156
enabled: False
156
157
157
158
llm_config: # Vision model configuration, currently supports OpenAI SDK compatible format only
158
-
model: gpt-4.1-2025-04-14 # Recommended
159
+
model: gpt-4.1-2025-04-14 # Primary model for Stage 2 test planning (Recommended)
160
+
filter_model: gpt-4o-mini # Lightweight model for Stage 1 element filtering (cost-effective)
159
161
api_key: your_api_key
160
162
base_url: https://api.example.com/v1
163
+
temperature: 0.1# Optional, default 0.1
164
+
# top_p: 0.9 # Optional, if not set, this parameter will not be passed
165
+
# max_tokens: 8192 # Optional, maximum output tokens (supports generating more test cases)
161
166
162
167
browser_config:
163
168
viewport: {"width": 1280, "height": 720}
164
169
headless: False # Automatically overridden to True in Docker environment
165
170
language: zh-CN
166
171
cookies: []
172
+
save_screenshots: False # Whether to save screenshots to local disk (default: False)
173
+
174
+
report:
175
+
language: en-US # zh-CN, en-US
176
+
177
+
log:
178
+
level: info
167
179
```
168
180
169
181
Please note the following important considerations when configuring and running tests:
170
182
171
183
#### 1. Functional Testing Notes
172
184
173
-
- **AI Mode**: When specifying the number of test cases to generate in the configuration file, the system may re-plan based on actual page conditions. This may result in the final number of executed test cases differing from the initial configuration to ensure coverage and effectiveness.
185
+
- **AI Mode**: Uses a 2-stage planning architecture where Stage 1 (filter_model) prioritizes elements for efficient analysis, and Stage 2 (primary model) generates comprehensive test cases. The system may reflect and re-plan based on actual page conditions and test coverage, which may result in the final number of executed test cases differing from the initial configuration to ensure effectiveness. When `dynamic_step_generation` is enabled, the system automatically generates additional test steps for newly appeared UI elements (e.g., dropdowns, modals) detected through DOM diff analysis.
174
186
175
187
- **Default Mode**: The `default` mode focuses on whether UI interactions (e.g., clicks and navigations) complete successfully.
176
188
177
189
#### 2. User Experience Testing Notes
178
190
179
-
UX (User Experience) testing focuses on usability, and user-friendliness. The model output in the results provides suggestions based on best practices to guide optimization.
191
+
UX (User Experience) testing focuses on usability and user-friendliness. Uses multi-modal analysis combining screenshots, DOM structure, and text content to evaluate visual quality, detect typos/grammar issues, and validate layout rendering. The model output in the results provides suggestions based on best practices to guide optimization.
0 commit comments