Conversation
This commit adds comprehensive visual regression testing infrastructure using Playwright, with full support for both light and dark themes. Features: - Visual regression tests for key sections (homepage, presets, general, compute, storage, results) in both light and dark modes - Playwright configuration optimized for visual testing - GitHub Actions workflows for automated testing and baseline generation - Comprehensive documentation and usage guides Setup includes: - playwright.config.ts: Configuration for light/dark mode projects - e2e/visual-regression.spec.ts: Complete test suite covering all major sections - .github/workflows/visual-regression.yml: Main CI workflow - .github/workflows/visual-regression-init.yml: Baseline generation workflow - e2e/README.md: Detailed test documentation - VISUAL_REGRESSION_TESTING.md: Complete setup and usage guide Testing workflow: 1. Tests run automatically on push and PRs 2. Baselines can be generated via GitHub Actions 3. Test reports and screenshots uploaded as artifacts 4. Support for local development with npm scripts npm scripts added: - test:e2e: Run all visual regression tests - test:e2e:ui: Run tests with interactive UI - test:e2e:debug: Run tests in debug mode - test:e2e:update: Update baseline screenshots - test:e2e:report: View HTML test report Dependencies added: - @playwright/test: Testing framework - playwright: Browser automation The baselines should be generated using the "Initialize Visual Regression Baselines" workflow in GitHub Actions for consistency across environments.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughThis change introduces a comprehensive visual regression testing framework using Playwright to the project. It includes GitHub Actions workflows for baseline initialization and continuous testing, a Playwright configuration supporting light and dark themes, an extensive test suite covering multiple UI sections, npm scripts for test execution, and detailed documentation for setup and usage. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User
participant GHA as GitHub Actions
participant Build as Build Process
participant E2E as Playwright E2E
participant Snapshots as Snapshot Storage
participant PR as Pull Request
User->>GHA: Trigger visual-regression-init workflow
GHA->>Build: Checkout & setup Node.js
Build->>Build: npm ci & playwright install
Build->>Build: npm run build (production)
GHA->>E2E: npm run test:e2e:update (CI=true)
E2E->>Snapshots: Generate baseline screenshots
Snapshots-->>E2E: Baseline images created
GHA->>PR: Create PR with baselines
PR-->>User: Review & merge baselines
rect rgb(220, 240, 255)
Note over GHA,E2E: Continuous Testing Flow
User->>GHA: Push to main or open PR
GHA->>Build: Checkout & setup
Build->>Build: npm ci & build
GHA->>E2E: npm run test:e2e
E2E->>Snapshots: Compare against baselines
Snapshots-->>E2E: Pass/Fail results
GHA-->>User: Test results & artifacts
end
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~28 minutes
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
VISUAL_REGRESSION_TESTING.md (1)
98-118: Add language hint for the project tree block.Markdownlint (MD040) flags fences without a language. Tag this one as plain text so docs CI stays green.
-``` +```text Infra-Wise/ ├── e2e/ │ ├── README.md # Detailed test documentation │ ├── visual-regression.spec.ts # Test suite │ └── visual-regression.spec.ts-snapshots/ # Baseline images ... -``` +```e2e/README.md (1)
90-103: Label the project tree fence.Same MD040 warning here—declare the fence as
text(or similar) to satisfy markdownlint.-``` +```text e2e/ ├── README.md # This file ├── visual-regression.spec.ts # Visual regression test suite └── visual-regression.spec.ts-snapshots/ # Baseline screenshots (git-tracked) ... -``` +```
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (8)
.github/workflows/visual-regression-init.yml(1 hunks).github/workflows/visual-regression.yml(1 hunks).gitignore(1 hunks)VISUAL_REGRESSION_TESTING.md(1 hunks)e2e/README.md(1 hunks)e2e/visual-regression.spec.ts(1 hunks)package.json(3 hunks)playwright.config.ts(1 hunks)
🧰 Additional context used
🪛 LanguageTool
e2e/README.md
[grammar] ~135-~135: Ensure spelling is correct
Context: ...2e:update` ### Tests Timeout If tests timeout during execution: - Ensure the develop...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🪛 markdownlint-cli2 (0.18.1)
e2e/README.md
98-98: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: test (20.x)
Comment on lines
+252
to
+281
| // Click on Results section in sidebar | ||
| const resultsButton = page.getByRole('button', { name: /results/i }).first() | ||
| if (await resultsButton.isVisible()) { | ||
| await resultsButton.click() | ||
| await page.waitForTimeout(500) | ||
|
|
||
| await expect(page).toHaveScreenshot('results-section-light.png', { | ||
| fullPage: true, | ||
| }) | ||
| } | ||
| }) | ||
|
|
||
| test('results should match snapshot in dark mode', async ({ page }) => { | ||
| await page.evaluate(() => { | ||
| document.documentElement.classList.remove('light') | ||
| document.documentElement.classList.add('dark') | ||
| }) | ||
| await page.waitForTimeout(500) | ||
|
|
||
| // Click on Results section in sidebar | ||
| const resultsButton = page.getByRole('button', { name: /results/i }).first() | ||
| if (await resultsButton.isVisible()) { | ||
| await resultsButton.click() | ||
| await page.waitForTimeout(500) | ||
|
|
||
| await expect(page).toHaveScreenshot('results-section-dark.png', { | ||
| fullPage: true, | ||
| }) | ||
| } | ||
| }) |
There was a problem hiding this comment.
Stop skipping the Results screenshot when the section is missing.
Because the screenshot assertion lives inside if (await resultsButton.isVisible()), the test happily passes even when the Results entry disappears—precisely the regression we want to catch. Assert visibility, always click it, and always compare the screenshot so the suite fails loudly when Results vanish.
- const resultsButton = page.getByRole('button', { name: /results/i }).first()
- if (await resultsButton.isVisible()) {
- await resultsButton.click()
- await page.waitForTimeout(500)
-
- await expect(page).toHaveScreenshot('results-section-light.png', {
- fullPage: true,
- })
- }
+ const resultsButton = page.getByRole('button', { name: /results/i }).first()
+ await expect(resultsButton).toBeVisible()
+ await resultsButton.click()
+ await page.waitForTimeout(500)
+
+ await expect(page).toHaveScreenshot('results-section-light.png', {
+ fullPage: true,
+ })
@@
- const resultsButton = page.getByRole('button', { name: /results/i }).first()
- if (await resultsButton.isVisible()) {
- await resultsButton.click()
- await page.waitForTimeout(500)
-
- await expect(page).toHaveScreenshot('results-section-dark.png', {
- fullPage: true,
- })
- }
+ const resultsButton = page.getByRole('button', { name: /results/i }).first()
+ await expect(resultsButton).toBeVisible()
+ await resultsButton.click()
+ await page.waitForTimeout(500)
+
+ await expect(page).toHaveScreenshot('results-section-dark.png', {
+ fullPage: true,
+ })🤖 Prompt for AI Agents
In e2e/visual-regression.spec.ts around lines 252 to 281, the screenshot
assertion is gated by an if-check on resultsButton.isVisible(), which lets the
test pass when the Results entry is missing; change this to assert the button is
visible (e.g., await expect(resultsButton).toBeVisible()), then always perform
the click and await necessary waits, and always call the screenshot assertion so
the test fails loudly if the Results section is absent or not rendered.
Sagyam
deleted the
claude/add-visual-regression-and-linter-fixes-011CV49AGiBFQddYG4cXHTFd
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit adds comprehensive visual regression testing infrastructure using Playwright, with full support for both light and dark themes.
Features:
Setup includes:
Testing workflow:
npm scripts added:
Dependencies added:
The baselines should be generated using the "Initialize Visual Regression Baselines" workflow in GitHub Actions for consistency across environments.
Summary by CodeRabbit
New Features
Documentation
Chores