GH#681: Add Playwright E2E tests for benchmark page - #682
Merged
Conversation
- Register ModelBenchmarkPage under Tools menu so the standalone React
SPA is accessible at tools.php?page=gratis-ai-agent-benchmark
- Remove the legacy redirect for gratis-ai-agent-benchmark from
UnifiedAdminMenu::handleLegacyRedirects() so the page is not
intercepted before it can render
- Add goToBenchmarkPage() helper to tests/e2e/utils/wp-admin.js
- Create tests/e2e/benchmark-page.spec.js with 19 test cases across
4 describe blocks covering:
- Page render: wrap/heading, React app mount, tab panel, form card
- Form inputs: Run Name, Description, Test Suite, Model Selector,
Start Benchmark button, model selection, Select/Deselect All
- Run list: History tab, Benchmark History heading, run name,
View/Delete buttons, status badge
- Empty run list: empty state message
REST API responses are mocked via page.route() for deterministic tests.
Closes #681
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughThis PR registers the Model Benchmark admin page as a standalone Tools menu item, removes an old legacy redirect for the benchmark page, and adds Playwright E2E tests plus a test helper to exercise rendering, form interactions, model selection, and run history behaviors. Changes
Sequence Diagram(s)sequenceDiagram
participant Browser as "Admin Browser\n(runs Playwright)"
participant WP as "WordPress Admin\nPHP (menu registration)"
participant REST as "WP REST API\n(gratis-ai-agent endpoints)"
participant DB as "Database / Storage"
rect rgba(135,206,250,0.5)
Browser->>WP: GET /wp-admin/tools.php?page=gratis-ai-agent-benchmark
WP->>Browser: Serve admin page + React app root
end
rect rgba(144,238,144,0.5)
Browser->>REST: GET /wp/v2/gratis-ai-agent/suites (mocked in tests)
REST->>Browser: 200 { suites: [...] }
Browser->>REST: GET /.../providers (mocked)
REST->>Browser: 200 { providers: [...] }
end
rect rgba(255,182,193,0.5)
Browser->>Browser: User fills form, selects models
Browser->>REST: POST /.../runs (creates benchmark run)
REST->>DB: persist run
DB->>REST: persisted run
REST->>Browser: 201 { run: {...} }
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/e2e/benchmark-page.spec.js (1)
414-466: Consider extracting a parameterized mock helper to reduce duplication.The
beforeEachblock duplicates most ofmockBenchmarkApi(), only differing in the runs response. You could parameterize the helper:Optional refactor
-async function mockBenchmarkApi( page ) { +async function mockBenchmarkApi( page, { runs = MOCK_RUNS } = {} ) { // ... suites and providers intercepts unchanged ... await page.route( ( url ) => { const decoded = decodeURIComponent( url.toString() ); return ( decoded.includes( 'gratis-ai-agent/v1/benchmark/runs' ) && ! decoded.includes( '/run-next' ) ); }, async ( route ) => { if ( route.request().method() === 'GET' ) { await route.fulfill( { status: 200, contentType: 'application/json', - body: JSON.stringify( MOCK_RUNS ), + body: JSON.stringify( runs ), } ); } else { await route.continue(); } } ); }Then in Empty Run List:
test.beforeEach( async ( { page } ) => { await loginToWordPress( page ); await mockBenchmarkApi( page, { runs: { runs: [] } } ); await goToBenchmarkPage( page ); } );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/benchmark-page.spec.js` around lines 414 - 466, The beforeEach in the "Benchmark Page - Empty Run List" test duplicates the routes from mockBenchmarkApi() except for the runs response; refactor by extracting a parameterized helper mockBenchmarkApi(page, overrides) that registers the same routes (suites -> MOCK_SUITES, providers -> MOCK_PROVIDERS, runs -> default or overridden response) and accepts an overrides object (e.g., { runs: { runs: [] } }) to replace the runs body, then update this test's test.beforeEach to call loginToWordPress(page); await mockBenchmarkApi(page, { runs: { runs: [] } }); and await goToBenchmarkPage(page) so duplication is removed and the runs response is customized via the helper.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/benchmark-page.spec.js`:
- Around line 262-274: Rename the test title to match the assertion and
behavior: update the test(...) description currently "Start Benchmark button is
disabled when no model is selected" to something like "Start Benchmark button is
enabled when no model is selected (clicking shows notice)"; leave the assertion
that startBtn (obtained via page.getByRole('button', { name: /start benchmark/i
})) is enabled and keep the surrounding comment about handleCreateRun showing a
notice instead of submitting.
---
Nitpick comments:
In `@tests/e2e/benchmark-page.spec.js`:
- Around line 414-466: The beforeEach in the "Benchmark Page - Empty Run List"
test duplicates the routes from mockBenchmarkApi() except for the runs response;
refactor by extracting a parameterized helper mockBenchmarkApi(page, overrides)
that registers the same routes (suites -> MOCK_SUITES, providers ->
MOCK_PROVIDERS, runs -> default or overridden response) and accepts an overrides
object (e.g., { runs: { runs: [] } }) to replace the runs body, then update this
test's test.beforeEach to call loginToWordPress(page); await
mockBenchmarkApi(page, { runs: { runs: [] } }); and await
goToBenchmarkPage(page) so duplication is removed and the runs response is
customized via the helper.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 31fd01d1-add7-4a21-984f-44d45fc3c027
📒 Files selected for processing (4)
gratis-ai-agent.phpincludes/Admin/UnifiedAdminMenu.phptests/e2e/benchmark-page.spec.jstests/e2e/utils/wp-admin.js
💤 Files with no reviewable changes (1)
- includes/Admin/UnifiedAdminMenu.php
The test name said 'disabled when no model is selected' but the assertion was toBeEnabled(). The button is always enabled — validation fires at runtime via a notice, not by disabling the button. Addresses CodeRabbit review on PR #682.
…ators Use exact: true for 'Select All' and 'Deselect All' button locators to prevent Playwright strict mode violations. The regex /select all/i matched both buttons because 'Deselect All' contains 'select all'. Fixes CI failure on PR #682.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ModelBenchmarkPageunder the Tools menu so the standalone benchmark React SPA is accessible attools.php?page=gratis-ai-agent-benchmark(fixes regression from UnifiedAdminMenu PR docs: update AGENTS.md #679)gratis-ai-agent-benchmarkfromUnifiedAdminMenu::handleLegacyRedirects()so the page renders instead of redirecting to#/settingsgoToBenchmarkPage()navigation helper totests/e2e/utils/wp-admin.jstests/e2e/benchmark-page.spec.jswith 19 test cases across 4 describe blocksTest Coverage
REST API responses (
/benchmark/suites,/providers,/benchmark/runs) are mocked viapage.route()for deterministic, provider-independent tests.Runtime Testing
node --check; PHP validated with PHPCS (0 violations)Files Changed
gratis-ai-agent.phpModelBenchmarkPage::register()hook — restores benchmark page access removed in #679includes/Admin/UnifiedAdminMenu.phpgratis-ai-agent-benchmarkfrom legacy redirect map — allows page to rendertests/e2e/utils/wp-admin.jsgoToBenchmarkPage()helper + exporttests/e2e/benchmark-page.spec.jsActionable Findings
actionable_findings_total=0fixed_in_pr=0deferred_tasks_created=0coverage=100%Closes #681
aidevops.sh v3.5.126 plugin for OpenCode v1.3.0 with claude-sonnet-4-6 spent 24m and 24,372 tokens on this as a headless worker.
Summary by CodeRabbit
New Features
Bug Fixes
Tests