Skip to content

Commit e0eb1a0

Browse files
nirukk52nirukk52
committed
feat: update webapp testing skill documentation and add bug report for run page regressions
- Revise SKILL.md to reflect Playwright E2E testing integration and pre-push hook automation. - Introduce new bug report for BUG-010 detailing critical regressions on the /run page, including issues with graph events and screenshot visibility. - Provide steps to reproduce, expected vs actual results, and proposed fixes to guide resolution efforts.
1 parent c54e30d commit e0eb1a0

File tree

3 files changed

+180
-3
lines changed

3 files changed

+180
-3
lines changed

‎.DS_Store‎

0 Bytes
Binary file not shown.

‎.claude-skills/webapp-testing_skill/SKILL.md‎

Lines changed: 61 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,73 @@
11
---
22
name: webapp-testing
3-
description: Playwright-first testing playbook for ScreenGraph. Automate the run page, capture screenshots, verify graph events, and fall back to Cursor browser tools when you need quick manual checks.
3+
description: Playwright E2E testing for ScreenGraph. Tests use .env configuration (package name is the main key). Automated tests run in pre-push hook. Use Cursor browser tools for quick manual inspection.
44
---
55

66
# ScreenGraph Web App Testing (Playwright + Cursor Tools)
77

8-
This skill consolidates the former `cursor-browser-mastery`, `cursor-chrome-window-mastery`, and legacy webapp testing guidance into a **Playwright-first** workflow. Use Playwright (via MCP or local scripts) for reliable automation, and keep Cursor’s built-in browser tools handy for fast inspection or ad-hoc verification.
8+
**Updated 2025-11-08**: FR-020 implemented unified Playwright test suite integrated into pre-push hook. All tests use `.env` file as single source of truth.
99

1010
---
1111

12-
## 1. Standard Setup (Do This First)
12+
## 1. Automated E2E Tests (Primary Method)
13+
14+
### Quick Start
15+
```bash
16+
# Headed mode (visual debugging with slowMo)
17+
bun run test:e2e:headed
18+
19+
# CI mode (headless, fast)
20+
bun run test:e2e:ci
21+
22+
# Interactive UI mode
23+
bun run test:e2e:ui
24+
25+
# Run from root or frontend directory
26+
```
27+
28+
### Test Configuration
29+
- **Package Name**: Read from `VITE_PACKAGE_NAME` in `.env` (currently: `com.jetbrains.kotlinconf`)
30+
- **Location**: `frontend/tests/e2e/`
31+
- **Config**: `frontend/playwright.config.ts` (environment-aware)
32+
- **Helpers**: `frontend/tests/e2e/helpers.ts` (reusable utilities)
33+
34+
### Current Tests
35+
1. **Landing Page Load** - Verifies frontend health
36+
2. **Run Page Navigation** - Clicks "Detect My First Drift" → verifies Run Timeline heading
37+
38+
### Pre-Push Hook Integration
39+
E2E tests run automatically before every push:
40+
```bash
41+
git push # Runs smoke tests + E2E tests automatically
42+
```
43+
44+
---
45+
46+
## 2. Manual Playwright Testing (For New Tests)
47+
48+
When creating new regression tests, place them in `frontend/tests/e2e/`:
49+
50+
```typescript
51+
import { test, expect } from "@playwright/test";
52+
import { TEST_PACKAGE_NAME, TEST_APP_CONFIG } from "./helpers";
53+
54+
test("my new test", async ({ page }) => {
55+
await page.goto("/");
56+
57+
// Your test logic here
58+
// All config comes from .env via helpers
59+
console.log(`Testing package: ${TEST_PACKAGE_NAME}`);
60+
});
61+
```
62+
63+
**Key Principles:**
64+
- Use `.env` for all configuration (package name, Appium URL, APK path)
65+
- Use `helpers.ts` utilities for common operations
66+
- Write environment-aware tests (work in both headed and headless modes)
67+
68+
---
69+
70+
## 3. Cursor Browser Tools (Quick Inspection)
1371
1. **Start services**
1472
```bash
1573
.cursor/commands/start-services

‎jira/bugs/BUG-010-run-page-regressions/BUG-010-main.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# BUG-010: Run Page Regressions - Graph Events & Screenshots Not Visible
2+
3+
> **Line Limit:** 150 lines max (enforced)
4+
> **Priority**: P1
5+
> **Status**: 🔴 Open
6+
7+
---
8+
9+
## Summary
10+
The `/run` page has three critical regressions preventing proper visualization of agent activity:
11+
1. **Graph events not visible** - Timeline events are not rendering
12+
2. **Screenshots not visible** - Discovered screenshot gallery is empty
13+
3. **Stop node not called** - Run completion indicator never fires
14+
15+
**Evidence**: `.playwright-mcp/drift-detection-with-screenshot.png`
16+
17+
---
18+
19+
## Severity / Impact
20+
- **Severity**: High
21+
- **Impact**: Complete loss of run visualization - users cannot verify agent behavior. Affects all users attempting drift detection. No workaround available.
22+
23+
---
24+
25+
## Environment
26+
- **Backend**: `encore run` (local) - Port 4000
27+
- **Frontend**: `localhost:5173`
28+
- **Browser/OS**: All browsers, all OS
29+
- **Package**: `com.jetbrains.kotlinconf` (from `.env`)
30+
31+
---
32+
33+
## Steps to Reproduce
34+
1. Start services: `bun run dev`
35+
2. Ensure Appium/device running
36+
3. Navigate to `http://localhost:5173`
37+
4. Click "Detect My First Drift" button
38+
5. Wait for navigation to `/run/{runId}` page
39+
6. **Observe**: Page loads but timeline empty, no screenshots, no stop node
40+
41+
---
42+
43+
## Expected Result
44+
- Timeline events appear in real-time (SSE updates)
45+
- Screenshot gallery populates with captured screens
46+
- Stop node fires when agent completes exploration
47+
- Run state transitions visible in UI
48+
49+
---
50+
51+
## Actual Result
52+
- Run Timeline heading visible ✅
53+
- Cancel Run button present ✅
54+
- **BUT**: No timeline events render
55+
- **BUT**: Screenshot gallery remains empty
56+
- **BUT**: Stop node never appears
57+
58+
---
59+
60+
## Root Cause
61+
**Investigation required**. Potential causes:
62+
1. SSE streams not connecting (check Network tab for `/run/stream`, `/graph/stream`)
63+
2. Backend not publishing events (check `run_event_outbox` table)
64+
3. Frontend not rendering events (Svelte reactivity issue)
65+
4. Agent not running (check backend logs for agent activity)
66+
67+
**Diagnostic Commands** (see `.playwright-mcp/BUG-010-INVESTIGATION.md`):
68+
```bash
69+
# Check SSE streams
70+
curl -N http://localhost:4000/run/stream/{runId}
71+
72+
# Check backend logs
73+
task backend:logs | grep -i "sse\|event"
74+
75+
# Check browser DevTools → Network tab → filter "stream"
76+
```
77+
78+
---
79+
80+
## Proposed Fix
81+
1. **Investigate**: Check browser DevTools Network/Console tabs
82+
2. **Verify**: Backend logs show event publications
83+
3. **Test**: SSE endpoints directly with curl
84+
4. **Fix**: Address root cause (SSE connection, event publishing, or rendering)
85+
5. **Test**: Add automated tests for event/screenshot rendering
86+
87+
**Suggested Test Additions** (`frontend/tests/e2e/run-page.spec.ts`):
88+
- Wait for graph events: `waitForGraphEvent(page, "agent.event.screenshot_captured")`
89+
- Verify screenshot count: `expect(countDiscoveredScreenshots(page)).toBeGreaterThan(0)`
90+
- Check stop node: `waitForStopNode(page, { timeout: 120000 })`
91+
92+
---
93+
94+
## Attachments / Logs
95+
- **Evidence**: `.playwright-mcp/drift-detection-with-screenshot.png`
96+
- **Investigation Guide**: `.playwright-mcp/BUG-010-INVESTIGATION.md`
97+
- **Test Helpers**: `frontend/tests/e2e/helpers.ts`
98+
99+
---
100+
101+
## Owner / Priority
102+
- **Reported by**: Founder (via Playwright analysis)
103+
- **Assigned to**: Next available frontend/fullstack engineer
104+
- **Priority**: P1 (High impact, core feature broken)
105+
106+
---
107+
108+
## Related Items
109+
- **Testing Infrastructure**: FR-020 (Run Page Regression Harness)
110+
- **Testing Guide**: `.claude-skills/webapp-testing_skill/SKILL.md`
111+
- **Original Report**: Request `f0164999-3a34-4705-bd7c-e426eff61c6f`
112+
113+
---
114+
115+
## Notes
116+
- Current E2E tests only verify page load, not event rendering
117+
- Test package: `com.jetbrains.kotlinconf` (defined in `.env`)
118+
- Manual reproduction required with Appium/device services running
119+
- See investigation guide for detailed diagnostic steps

0 commit comments

Comments
 (0)