feat: enable parallel Playwright workers for E2E tests by kdaviduik · Pull Request #3509 · Shopify/hydrogen

kdaviduik · 2026-02-26T04:30:44Z

Summary

E2E tests were limited to 1 Playwright worker because every DevServer defaulted to port 3100, causing bind failures under parallel execution
Enables 3 workers in CI (4 locally) by using OS-assigned ports, fixing process tree cleanup, and bypassing codegen conflicts

Changes

e2e/fixtures/server.ts (main changes):

Default port changed from 3100 to 0 (OS-assigned) — port collision is now structurally impossible
Spawn shopify hydrogen dev directly instead of npm run dev to avoid --codegen file write conflicts in shared templates/skeleton/ directory
Process group kill via detached: true + negative PID ensures child processes (vite, workerd) are terminated, not just the npm parent
stop() hardened: handles already-dead processes, captures PID upfront to avoid races, SIGKILL timeout always resolves the promise
getUrl() throws clear error instead of returning nonsensical http://localhost:0
Server startup timeout increased from 60s to 120s for CPU contention during parallel startup
Stdout URL capture rejects port 0 (guards against pre-binding echo)

playwright.config.ts:

workers: process.env.CI ? 3 : 4 (was 1)
Comment explains CI runner constraints (2 vCPUs, 7GB RAM)

.github/workflows/ci.yml:

Removed --workers=1 override (config is now single source of truth)
E2E job timeout increased from 15 to 20 minutes

Test plan

Smoke tests pass locally with 4 workers (3 servers on ports 5173, 5174, 5175)
Full E2E suite passes in CI with 3 workers
Monitor first 10-20 CI runs for startup timeouts, OOM kills, or flakiness

shopify · 2026-02-26T04:31:00Z

Oxygen deployed a preview of your kd-workers-4-playwright branch. Details:

Storefront	Status	Preview link	Deployment details	Last update (UTC)
Skeleton (skeleton.hydrogen.shop)	✅ Successful (Logs)	Preview deployment	Inspect deployment	February 27, 2026 3:57 AM

Learn more about Hydrogen's GitHub integration.

kdaviduik · 2026-02-26T04:58:36Z

.github/workflows/ci.yml


      - name: 🧪 Run E2E tests
-        run: pnpm exec playwright test --workers=1
+        run: pnpm exec playwright test


we want to have the workers number have a single source of truth: the playwright config

frandiox

Funny because I actually tried spinning up multiple servers with different ports (that's why capturedUrl already existed) and it failed miserably 😂 (blaming Opus 4.5)

fredericoo · 2026-02-26T11:24:15Z

e2e/fixtures/server.ts

    this.id = options.id;
    this.storeKey = options.storeKey;
-    this.port = options.port ?? 3100;
+    this.port = options.port ?? 0;


is zero really a good fallback? undefined is easier to check for later

fredericoo · 2026-02-26T11:24:55Z

e2e/fixtures/server.ts

  getUrl() {
-    return this.capturedUrl || `http://localhost:${this.port}`;
+    if (this.capturedUrl) return this.capturedUrl;
+    if (this.port === 0) {


like here, if we had undefined as a possible value, this would be clearer to check, zero feels a lil magic numbery

Using port 0 is a POSIX convention, but yeah definitely magic numbery, thanks for catching that :) I've updated it here to not be a magic number

fredericoo

left some non blocking comments but looks great! lfg!!!

E2E tests were limited to 1 worker because every DevServer defaulted to port 3100, causing bind failures under parallel execution. The skeleton's npm script also includes --codegen which causes file write conflicts when multiple servers share the same directory. Key changes: - Default port changed from 3100 to 0 (OS-assigned) so port collision is structurally impossible - Spawn shopify hydrogen dev directly (not via npm run dev) to avoid --codegen file write conflicts - Process group kill via detached spawn + negative PID ensures child processes (vite, workerd) are terminated, not just the npm parent - stop() hardened: handles already-dead processes, captures PID upfront to avoid races, SIGKILL timeout always resolves the promise - getUrl() throws clear error instead of returning nonsensical http://localhost:0 - Server startup timeout increased from 60s to 120s for CPU contention during parallel startup - Workers: 3 in CI (ubuntu-latest: 2 vCPUs, 7GB RAM), 4 locally - CI job timeout increased from 15 to 20 minutes

…T constant Addresses PR feedback from Freddie: port 0 was doing double duty as both 'no port specified' (application concept) and 'let the OS pick' (POSIX convention). Now undefined means 'not specified' at the TypeScript level, and OS_ASSIGNED_PORT = 0 is only used at the spawn boundary where it has actual POSIX semantics. This eliminates the magic number and makes getUrl() checks more idiomatic TypeScript (=== undefined vs === 0).

binks-code-reviewer · 2026-02-27T04:04:00Z

⚠️ Findings outside the diff

These findings are in files not modified by this PR and cannot be posted as inline comments.

e2e/fixtures/server.ts:109 — Dev server start detection depends on output containing "success" (flaky/can hang)

The server is considered “started” only when the CLI output contains the substring "success": if (!started && output.includes('success')) {}. This is brittle because it depends on a specific log message that can change across Shopify CLI/Hydrogen versions, locales, log levels, or can be printed before the server is actually reachable. If that exact substring is not emitted, started never flips to true → the Promise never resolves → tests wait until the 120s timeout and fail (or worse: if stop() doesn’t actually terminate the process group reliably on the platform, you can leave orphan processes behind).

e2e/fixtures/server.ts:116 — URL/port extraction may pick wrong port when tunnel URL exists

When started, code sets this.capturedUrl = tunnelUrl || localUrl; then parses port via this.capturedUrl?.match(/:(\d+)/)?.[1]. If tunnelUrl is used (https), the regex can match unexpected parts of the string and/or capture a port from the tunnel URL that may not correspond to the local dev server port. Even for localUrl, :(\d+) isn’t anchored and could match other “:123” fragments if output format changes.

binks-code-reviewer · 2026-02-27T04:04:01Z

🤖 Code Review · #projects-dev-ai for questions
React with 👍/👎 or reply — all feedback helps improve the agent.

✅ Complete - No issues

📋 History

❌ Failed → ✅ No issues

kdaviduik requested a review from a team as a code owner February 26, 2026 04:30

kdaviduik commented Feb 26, 2026

View reviewed changes

frandiox reviewed Feb 26, 2026

View reviewed changes

fredericoo reviewed Feb 26, 2026

View reviewed changes

fredericoo approved these changes Feb 26, 2026

View reviewed changes

andguy95 approved these changes Feb 26, 2026

View reviewed changes

kdaviduik force-pushed the kd-workers-4-playwright branch from 849a85e to 2502880 Compare February 27, 2026 03:44

kdaviduik force-pushed the kd-workers-4-playwright branch from 2502880 to 4a724dc Compare February 27, 2026 03:47

kdaviduik force-pushed the kd-workers-4-playwright branch from 4a724dc to 1aaced3 Compare February 27, 2026 03:55

kdaviduik merged commit 9386b50 into main Feb 27, 2026
14 checks passed

kdaviduik deleted the kd-workers-4-playwright branch February 27, 2026 04:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: enable parallel Playwright workers for E2E tests#3509

feat: enable parallel Playwright workers for E2E tests#3509
kdaviduik merged 2 commits intomainfrom
kd-workers-4-playwright

kdaviduik commented Feb 26, 2026 •

edited

Loading

Uh oh!

shopify bot commented Feb 26, 2026 •

edited

Loading

Uh oh!

kdaviduik Feb 26, 2026

Uh oh!

frandiox left a comment

Uh oh!

fredericoo Feb 26, 2026

Uh oh!

kdaviduik Feb 27, 2026

Uh oh!

fredericoo Feb 26, 2026

Uh oh!

kdaviduik Feb 27, 2026

Uh oh!

fredericoo left a comment

Uh oh!

binks-code-reviewer bot commented Feb 27, 2026

Uh oh!

binks-code-reviewer bot commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

kdaviduik commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Uh oh!

shopify bot commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kdaviduik Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

frandiox left a comment

Choose a reason for hiding this comment

Uh oh!

fredericoo Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

kdaviduik Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

fredericoo Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

kdaviduik Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

fredericoo left a comment

Choose a reason for hiding this comment

Uh oh!

binks-code-reviewer bot commented Feb 27, 2026

Uh oh!

binks-code-reviewer bot commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kdaviduik commented Feb 26, 2026 •

edited

Loading

shopify bot commented Feb 26, 2026 •

edited

Loading