test(computer,report): add e2e tests for extension playground and recorder#2195
test(computer,report): add e2e tests for extension playground and recorder#2195
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4b70fa0f77
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,33 @@ | |||
| target: | |||
| serve: ./dist | |||
| url: demo.html | |||
There was a problem hiding this comment.
Running pnpm --filter @midscene/report e2e will try to load ./dist/demo.html here, but a repo-wide check only finds apps/report/dist/index.html and apps/report/template/index.html—there is no demo.html under apps/report. As written, these newly added report suites 404 before any assertions run, so the added coverage cannot actually execute until the target page name or build output is corrected.
Useful? React with 👍 / 👎.
| - name: sidebar task list is rendered | ||
| flow: | ||
| - sleep: 2000 | ||
| - aiAssert: The left sidebar shows a list of task rows with type, time, and intent columns |
There was a problem hiding this comment.
This first assertion expects an intent column on initial load, but apps/report/src/components/sidebar/index.tsx only appends the Intent/Model/Prompt/... columns when proModeEnabled is true via the Model Call Details checkbox. Because the checkbox is still off at this point, the suite fails before it ever reaches the later task that turns those columns on.
Useful? React with 👍 / 👎.
| - ai: Click on the "Param" section header in the right detail side panel to expand it | ||
| - sleep: 1000 | ||
| - aiAssert: The Param section is expanded showing task parameters or input data | ||
| - ai: Click on the "Meta" section header in the right detail side panel to expand it |
There was a problem hiding this comment.
apps/report/src/components/detail-side/index.tsx renders both Param and Meta as <details open>, so clicking their headers here collapses them instead of expanding them. On a normal report page, the subsequent assertions looking for visible parameters and metadata will therefore fail unless some earlier step happened to close those sections first.
Useful? React with 👍 / 👎.
Deploying midscene with
|
| Latest commit: |
b76232f
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://9359bdeb.midscene.pages.dev |
| Branch Preview URL: | https://feat-extension-playground-e2.midscene.pages.dev |
- Add playground advanced tests (aiQuery, aiAssert, multi-todo) - Add recorder mode tests (session, start/stop, code gen) - Add report YAML e2e tests (sidebar, player, detail, theme) - Add parallel CI jobs for new test files
- Reduce playground test steps to avoid timeout - Fix recorder test to match auto-start behavior - Remove report-e2e CI job (demo.html not available)
- Combine playground tests: aiQuery+aiAssert+fail in one it() - Combine recorder tests: full lifecycle in one it() - Add chrome-extension-settings.test.ts: settings modal, 3-mode rotation - Add parallel CI job for settings tests (6 total jobs)
- Bridge: kill port before starting, wait for LISTENING, async kill - Playground: split aiQuery into own it(), increase sleep to 30s
- Increase testTimeout to 480s for playground and recorder - Add 3s sleep after bridge connect for UI update - Split recorder lifecycle into 2 shorter tests - Remove overly complex delete session test
- Settings: increase timeout to 480s, remove extra bridge assert - Recorder: merge close+return into lifecycle test
- Playground: separate tests, remove flaky aiAssert fail scenario - Recorder: remove fragile REC state assertion
- Recorder: increase testTimeout to 600s for long lifecycle test - Bridge: increase post-connect wait to 5s for UI update
The mode switch back to Playground is not core recorder logic and causes intermittent timeout failures.
quanru
force-pushed
the
feat/extension-playground-e2e-tests
branch
from
d741257 to
2e8ba7c
Compare
player-autoplay.yaml and player-settings.yaml from main already cover play/pause and subtitle toggle. The duplicate was flaky.
OPENAI_API_KEY, OPENAI_BASE_URL, MIDSCENE_OPENAI_INIT_CONFIG_JSON are no longer used by the chrome extension. Use MIDSCENE_* vars only.
Simplify assertions to match actual report UI. The sidebar shows task execution rows, not column headers by default.
2 participants
Summary
chrome-extension-playground.test.ts: extended Playground tests covering aiQuery execution, aiAssert validation, and todo operations via aiActchrome-extension-recorder.test.ts: Recorder mode tests covering mode switching, session creation, start/stop recording, and returning to Playgroundheadless-linux.ymlwith 2 new parallel CI jobs (chrome-extension-playground,chrome-extension-recorder) for increased parallelismAll 5 Headless Linux CI jobs pass: https://github.com/web-infra-dev/midscene/actions/runs/23350651881
Test plan
pnpm run lintpasseschrome-extension-playgroundCI job passes (~21min)chrome-extension-recorderCI job passes (~18min)chrome-extensionandchrome-extension-bridgejobs unaffected and passheadless-linux-ai-todojob unaffected and passes