tests: Migrate to @playwright/test and add core behavior coverage#1498
Open
DSingh0304 wants to merge 10 commits intozulip:mainfrom
Open
tests: Migrate to @playwright/test and add core behavior coverage#1498DSingh0304 wants to merge 10 commits intozulip:mainfrom
DSingh0304 wants to merge 10 commits intozulip:mainfrom
Conversation
added 5 commits
March 10, 2026 13:20
This migration is the first step in building an efficient testing process for the app, as discussed in issue #D82. The legacy testing setup relied on 'tape' and 'playwright-core' with manual window state management. By switching to '@playwright/test', we gain: - Native Electron test fixtures and window handling. - Parallel test execution capabilities. - Robust built-in reporter and CI trace viewer. This commit updates the dependencies and the 'test-e2e' script to use the new runner.
This provides a standardized configuration for the new testing framework. We set 'workers: 1' to avoid potential conflicts during Electron application launches in parallel and configure the HTML reporter for better visibility into CI failures.
Refactor the setup utility to use types and methods from the new testing framework. This ensures that our test helpers are correctly typed and provide a stable foundation for launching and closing the Electron instance during tests.
This commit replaces the primitive manual window queue logic with Playwright's automated window handling. We removed 'tests/index.ts' and 'tests/test-new-organization.ts' which were using the old 'tape' runner and replaced the core organization management test with a non-flaky version that reliably handles organization connection flows.
This adds a second 'core behavior' test to demonstrate the stability of the new Playwright framework. It verifies that: 1. The settings button is interactable. 2. The modal successfully transitions the UI to the active state. 3. The functional settings view is rendered correctly. This serves as a template for building further UI coverage for the rest of the desktop client.
added 5 commits
March 10, 2026 13:43
This configuration file sets up the modern testing runner for the project. It defines the test directory, matches test files by pattern, and configures the HTML reporter. Specifically, we set 'workers: 1' to prevent multiple Electron instances from conflicting with each other during parallel testing, which ensures a non-flaky execution environment as requested by the maintainers.
This update adjusts the 'createApp' and 'endTest' helpers to use types from '@playwright/test'. By unifying the framework across the setup and the tests, we ensure better type safety and compatibility with the new runner's fixtures. This provides a clean foundation for launching and cleaning up the Electron instance for every test.
This change removes the old tests that relied on 'tape' and manual window queue management, and replaces them with a modern fixture-based approach in 'test-add-organization.ts'. By using the native Electron window handling from @playwright/test, we can reliably verify the initial screen title and organization connection flow (including 'chat.zulip.org') in a non-flaky manner. This provides the stable, well-written demonstration requested by the maintainers.
This adds a second 'core behavior' test to demonstrate the reliability and extensibility of the new Playwright framework. It verifies that clicking the gear icon correctly opens the settings modal by checking for the active UI class and the presence of the functional settings view. This provides a concrete baseline for expanding UI coverage across the rest of the application.
This change applies Prettier formatting to the 'package.json' file, resolving a lint warning that was causing the PR checks to fail. The file now correctly matches the project's code style standards as verified by the 'node --run test' command.
Collaborator
Author
|
Ready for review! |
Member
|
This PR needs to be cleaned up for review-ability. Check out our GitHub guide and commit guidelines for more details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR serves as a demonstration of a modernized, non-flaky testing infrastructure for the desktop client.
Why this is needed: The current testing setup relies on tape and playwright core, requiring manual window/page queue management which is inefficient and prone to flakiness. Transitioning to @playwright/test provides:
Changes included:
Screenshots and screen captures:
Platforms this PR was tested on:
Self-review checklist
(variable names, code reuse, readability, etc.).
Communicate decisions, questions, and potential concerns.
Individual commits are ready for review (see commit discipline).
Completed manual review and testing of the following: