Skip to content

Commit b6f1fc4

Browse files
arvinxxclaude[bot]
arvinxx
and
claude[bot]
authored
✅ test: add BDD test framework and initial tests with Playwright and Cucumber (lobehub#9843)
* try with bdd test * update * update * add workspace * update * fix * ci * ci * fix * update * update * update parallel * update config * ⚡️ perf: increase e2e timeout to 120 seconds Co-authored-by: Arvin Xu <arvinxx@users.noreply.github.com> * update config * more parallel * fix parallel * fix tests * refactor to improve performance * fix * fix * fix * refactor with tsx --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Arvin Xu <arvinxx@users.noreply.github.com>
1 parent 0dc1124 commit b6f1fc4

File tree

23 files changed

+672
-132
lines changed

23 files changed

+672
-132
lines changed

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,36 @@
1212
- [ ] 📝 docs
1313
- [ ] 🔨 chore
1414

15+
#### 🔗 Related Issue
16+
17+
<!-- Link to the issue that is fixed by this PR -->
18+
19+
<!-- Example: Fixes #123, Closes #456, Related to #789 -->
20+
1521
#### 🔀 Description of Change
1622

1723
<!-- Thank you for your Pull Request. Please provide a description above. -->
1824

25+
#### 🧪 How to Test
26+
27+
<!-- Please describe how you tested your changes -->
28+
29+
<!-- For AI features, please include test prompts or scenarios -->
30+
31+
- [ ] Tested locally
32+
- [ ] Added/updated tests
33+
- [ ] No tests needed
34+
35+
#### 📸 Screenshots / Videos
36+
37+
<!-- If this PR includes UI changes, please provide screenshots or videos -->
38+
39+
| Before | After |
40+
| ------ | ----- |
41+
| ... | ... |
42+
1943
#### 📝 Additional Information
2044

2145
<!-- Add any other context about the Pull Request here. -->
46+
47+
<!-- Breaking changes? Migration guide? Performance impact? -->

‎.github/workflows/e2e.yml‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -35,18 +35,18 @@ jobs:
3535
PORT: 3010
3636
run: bun run e2e
3737

38-
- name: Upload Playwright HTML report (on failure)
38+
- name: Upload Cucumber HTML report (on failure)
3939
if: failure()
4040
uses: actions/upload-artifact@v4
4141
with:
42-
name: playwright-report
43-
path: playwright-report
42+
name: cucumber-report
43+
path: e2e/reports
4444
if-no-files-found: ignore
4545

46-
- name: Upload Playwright traces (on failure)
46+
- name: Upload screenshots (on failure)
4747
if: failure()
4848
uses: actions/upload-artifact@v4
4949
with:
50-
name: test-results
51-
path: test-results
50+
name: test-screenshots
51+
path: e2e/screenshots
5252
if-no-files-found: ignore

‎e2e/README.md‎

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# E2E Tests for LobeChat
2+
3+
This directory contains end-to-end (E2E) tests for LobeChat using Cucumber (BDD) and Playwright.
4+
5+
## Directory Structure
6+
7+
````
8+
e2e/
9+
├── src/ # Source files
10+
│ ├── features/ # Gherkin feature files
11+
│ │ └── discover/ # Discover page tests
12+
│ ├── steps/ # Step definitions
13+
│ │ ├── common/ # Reusable step definitions
14+
│ │ └── discover/ # Discover-specific steps
15+
│ └── support/ # Test support files
16+
│ └── world.ts # Custom World context
17+
├── reports/ # Test reports (generated)
18+
├── cucumber.config.js # Cucumber configuration
19+
├── tsconfig.json # TypeScript configuration
20+
└── package.json # Dependencies and scripts
21+
22+
## Prerequisites
23+
24+
- Node.js 20, 22, or >=24
25+
- Dev server running on `http://localhost:3010` (or set `BASE_URL` env var)
26+
27+
## Installation
28+
29+
Install dependencies:
30+
31+
```bash
32+
cd e2e
33+
pnpm install
34+
````
35+
36+
Install Playwright browsers:
37+
38+
```bash
39+
npx playwright install chromium
40+
```
41+
42+
## Running Tests
43+
44+
Run all tests:
45+
46+
```bash
47+
npm test
48+
```
49+
50+
Run tests in headed mode (see browser):
51+
52+
```bash
53+
npm run test:headed
54+
```
55+
56+
Run only smoke tests:
57+
58+
```bash
59+
npm run test:smoke
60+
```
61+
62+
Run discover tests:
63+
64+
```bash
65+
npm run test:discover
66+
```
67+
68+
## Environment Variables
69+
70+
- `BASE_URL`: Base URL for the application (default: `http://localhost:3010`)
71+
- `PORT`: Port number (default: `3010`)
72+
- `HEADLESS`: Run browser in headless mode (default: `true`, set to `false` to see browser)
73+
74+
Example:
75+
76+
```bash
77+
HEADLESS=false BASE_URL=http://localhost:3000 npm run test:smoke
78+
```
79+
80+
## Writing Tests
81+
82+
### Feature Files
83+
84+
Feature files are written in Gherkin syntax and placed in the `src/features/` directory:
85+
86+
```gherkin
87+
@discover @smoke
88+
Feature: Discover Smoke Tests
89+
Critical path tests to ensure the discover module is functional
90+
91+
@DISCOVER-SMOKE-001 @P0
92+
Scenario: Load discover assistant list page
93+
Given I navigate to "/discover/assistant"
94+
Then the page should load without errors
95+
And I should see the page body
96+
And I should see the search bar
97+
And I should see assistant cards
98+
```
99+
100+
### Step Definitions
101+
102+
Step definitions are TypeScript files in the `src/steps/` directory that implement the steps from feature files:
103+
104+
```typescript
105+
import { Given, Then } from '@cucumber/cucumber';
106+
import { expect } from '@playwright/test';
107+
108+
import { CustomWorld } from '../../support/world';
109+
110+
Given('I navigate to {string}', async function (this: CustomWorld, path: string) {
111+
await this.page.goto(path);
112+
await this.page.waitForLoadState('domcontentloaded');
113+
});
114+
```
115+
116+
## Test Reports
117+
118+
After running tests, HTML and JSON reports are generated in the `reports/` directory:
119+
120+
- `reports/cucumber-report.html` - Human-readable HTML report
121+
- `reports/cucumber-report.json` - Machine-readable JSON report
122+
123+
## Troubleshooting
124+
125+
### Browser not found
126+
127+
If you see errors about missing browser executables:
128+
129+
```bash
130+
npx playwright install chromium
131+
```
132+
133+
### Port already in use
134+
135+
Make sure the dev server is running on the expected port (3010 by default), or set `PORT` or `BASE_URL` environment variable.
136+
137+
### Test timeout
138+
139+
Increase timeout in `cucumber.config.js` or `src/steps/hooks.ts`:
140+
141+
```typescript
142+
setDefaultTimeout(120000); // 2 minutes
143+
```

‎e2e/cucumber.config.js‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
/**
2+
* @type {import('@cucumber/cucumber').IConfiguration}
3+
*/
4+
export default {
5+
format: [
6+
'progress-bar',
7+
'html:reports/cucumber-report.html',
8+
'json:reports/cucumber-report.json',
9+
],
10+
formatOptions: {
11+
snippetInterface: 'async-await',
12+
},
13+
parallel: process.env.CI ? 1 : 4,
14+
paths: ['src/features/**/*.feature'],
15+
publishQuiet: true,
16+
require: ['src/steps/**/*.ts', 'src/support/**/*.ts'],
17+
requireModule: ['tsx/cjs'],
18+
retry: 0,
19+
timeout: 120_000,
20+
};

‎e2e/package.json‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"name": "@lobechat/e2e-tests",
3+
"version": "0.1.0",
4+
"private": true,
5+
"description": "E2E tests for LobeChat using Cucumber and Playwright",
6+
"scripts": {
7+
"test": "cucumber-js --config cucumber.config.js",
8+
"test:discover": "cucumber-js --config cucumber.config.js src/features/discover/",
9+
"test:headed": "HEADLESS=false cucumber-js --config cucumber.config.js",
10+
"test:routes": "cucumber-js --config cucumber.config.js --tags '@routes'",
11+
"test:routes:ci": "cucumber-js --config cucumber.config.js --tags '@routes and not @ci-skip'",
12+
"test:smoke": "cucumber-js --config cucumber.config.js --tags '@smoke'"
13+
},
14+
"dependencies": {
15+
"@cucumber/cucumber": "^12.2.0",
16+
"@playwright/test": "^1.56.1",
17+
"playwright": "^1.56.1"
18+
},
19+
"devDependencies": {
20+
"@types/node": "^22.10.5",
21+
"tsx": "^4.20.6",
22+
"typescript": "^5.7.3"
23+
}
24+
}

‎e2e/routes.spec.ts‎

Lines changed: 0 additions & 73 deletions
This file was deleted.

‎e2e/src/features/discover/smoke.feature‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
@discover @smoke
2+
Feature: Discover Smoke Tests
3+
Critical path tests to ensure the discover module is functional
4+
5+
@DISCOVER-SMOKE-001 @P0
6+
Scenario: Load discover assistant list page
7+
Given I navigate to "/discover/assistant"
8+
Then the page should load without errors
9+
And I should see the page body
10+
And I should see the search bar
11+
And I should see assistant cards

‎e2e/src/features/routes/core-routes.feature‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
@routes @smoke
2+
Feature: Core Routes Accessibility
3+
As a user
4+
I want all core application routes to be accessible
5+
So that I can navigate the application without errors
6+
7+
Background:
8+
Given the application is running
9+
10+
@ROUTES-001 @P0
11+
Scenario Outline: Access core routes without errors
12+
When I navigate to "<route>"
13+
Then the response status should be less than 400
14+
And the page should load without errors
15+
And I should see the page body
16+
And the page title should not contain "error" or "not found"
17+
18+
Examples:
19+
| route |
20+
| / |
21+
| /chat |
22+
| /discover |
23+
| /files |
24+
| /repos |
25+
26+
@ROUTES-002 @P0
27+
Scenario Outline: Access settings routes without errors
28+
When I navigate to "/settings?active=<tab>"
29+
Then the response status should be less than 400
30+
And the page should load without errors
31+
And I should see the page body
32+
And the page title should not contain "error" or "not found"
33+
34+
Examples:
35+
| tab |
36+
| about |
37+
| agent |
38+
| hotkey |
39+
| provider |
40+
| proxy |
41+
| storage |
42+
| system-agent |
43+
| tts |

0 commit comments

Comments
 (0)