Skip to content

Add Shopify checkout Playwright integration test #461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shayancoin merged 1 commit into from
Oct 20, 2025
Merged

Add Shopify checkout Playwright integration test #461

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/release-checklist.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ This checklist ties together continuous integration signal, Grafana alerting, an
## 1. Verify CI Observability Gates

- Check the **performance-budget** job in GitHub Actions CI. This job runs the Playwright-based budget defined in `perf-budget.yml` and publishes a JUnit report that Grafana can ingest. If it fails, fix the regression before proceeding.
- Confirm the **shopify-integration** Playwright job passes. It executes `tests/shopify-checkout.spec.ts`, which provisions Shopify configuration, drives a Storefront test checkout, and waits for the backend to ingest the order. Configure the following GitHub Actions secrets so the run can talk to Shopify and Supabase:
- `SHOPIFY_STORE_DOMAIN`
- `SHOPIFY_STOREFRONT_ACCESS_TOKEN`
- `SHOPIFY_STOREFRONT_HELPER_URL`
- `SHOPIFY_STOREFRONT_HELPER_TOKEN` (if the helper requires bearer auth)
- `SHOPIFY_TEST_VARIANT_ID`
- `SHOPIFY_TEST_CURRENCY` (defaults to `USD` if omitted)
- `SUPABASE_URL`
- `SUPABASE_SERVICE_ROLE_KEY`
- Optional: `SHOPIFY_MODULE_CONFIG_TABLE` and `SHOPIFY_ORDER_TABLE` when the Supabase schema uses custom table names.
Comment on lines +8 to +17
Copy link
Contributor

@coderabbitai coderabbitai bot Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Harden secrets handling for CI (service-role key).

Document safeguards for SUPABASE_SERVICE_ROLE_KEY and other tokens:

  • Use GitHub Environments with required reviewers and secret scoping to shopify-integration only.
  • Ensure logs/artifacts never echo secrets; mask them and avoid attaching request/response bodies that may include auth headers.
  • Prefer short‑lived tokens if possible; rotate on a schedule and after test failures/incidents.

Adding these points here prevents accidental leakage during integration runs.

🤖 Prompt for AI Agents 
In docs/release-checklist.md around lines 8–17, the checklist currently lists
GitHub Actions secrets for the shopify-integration job but lacks CI
secret-handling safeguards; update this section to document concrete protections
for SUPABASE_SERVICE_ROLE_KEY and other tokens by: add bullets instructing use
of GitHub Environments scoped to the shopify-integration workflow with required
reviewers, restrict secret access to the job only, ensure masking of secrets and
avoid logging or artifacting request/response bodies that may contain auth
headers, recommend short‑lived tokens plus scheduled rotation and rotation after
failures/incidents, and optionally call out using separate least-privilege
service role keys for CI versus production.

- Confirm that the **observability-budgets** job has passed. It queries Prometheus and Tempo spanmetrics using `observability-budgets.yml` and fails when P95 latency or error-rate thresholds are exceeded compared to the previous day.
- Export any new failure signatures into the on-call runbook.

Expand Down
129 changes: 101 additions & 28 deletions playwright.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import path from 'path';
import { Module } from 'module';
Copy link
Contributor

@coderabbitai coderabbitai bot Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Make NODE_PATH handling cross‑platform and fix private API typing.

  • Use path.delimiter instead of hardcoded : for Windows support.
  • Module._initPaths() is private and not typed; cast to any or avoid it.
  • Prefer node: specifier for core module import.

Apply this diff:

-import { Module } from 'module';
+import { Module } from 'node:module';
@@
-const nodePathEntries = [path.resolve(__dirname, 'frontend/node_modules'), process.env.NODE_PATH].filter(Boolean) as string[];
+const nodePathEntries = [path.resolve(__dirname, 'frontend/node_modules'), process.env.NODE_PATH]
+  .filter(Boolean) as string[];
 if (nodePathEntries.length > 0) {
-  process.env.NODE_PATH = nodePathEntries.join(':');
-  Module._initPaths();
+  process.env.NODE_PATH = nodePathEntries.join(path.delimiter);
+  // eslint-disable-next-line @typescript-eslint/no-explicit-any
+  (Module as any)._initPaths?.();
 }

Alternatively, avoid NODE_PATH entirely by using TS path aliases + bundler resolution.

Also applies to: 34-38

🤖 Prompt for AI Agents 
In playwright.config.ts around lines 2 and 34-38, the code imports the private
Module API and assumes POSIX NODE_PATH separator; change the import to use the
core specifier (import Module from 'node:module' or import * as Module from
'node:module') and replace hardcoded ':' with path.delimiter from the path
module to make NODE_PATH handling cross‑platform; avoid directly calling/typing
Module._initPaths() by casting Module to any when invoking it or refactor to not
call it at all (e.g., rebuild NODE_PATH into process.env.NODE_PATH or rely on TS
path aliases + bundler), and ensure any modifications to NODE_PATH
append/prepend using path.delimiter consistently for Windows support.

import type { PlaywrightTestConfig } from '@playwright/test';

const testDir = path.resolve(__dirname, 'frontend/tests/e2e');
const frontendTestDir = path.resolve(__dirname, 'frontend/tests/e2e');
const shopifyTestDir = path.resolve(__dirname, 'tests');
const junitOutputFile = path.resolve(__dirname, 'artifacts/junit.xml');
const launchArgs = [
'--ignore-gpu-blocklist',
Expand All @@ -10,9 +12,32 @@ const launchArgs = [
'--disable-gpu-sandbox',
];

const frontendEnv = {
NEXT_PUBLIC_BACKEND_HOST: process.env.NEXT_PUBLIC_BACKEND_HOST || 'localhost',
NEXT_PUBLIC_BACKEND_PORT: process.env.NEXT_PUBLIC_BACKEND_PORT || '3000',
NEXT_PUBLIC_DISABLE_ENVIRONMENT: 'true',
NEXT_PUBLIC_DISABLE_MODEL_LOADING: 'true',
};

const sharedUseOptions = {
headless: true,
trace: 'retain-on-failure' as const,
launchOptions: {
args: launchArgs,
},
};

const requestedProjects = collectRequestedProjects(process.argv);
const skipWebServer =
requestedProjects.length > 0 && requestedProjects.every((project) => project === 'shopify-integration');

const nodePathEntries = [path.resolve(__dirname, 'frontend/node_modules'), process.env.NODE_PATH].filter(Boolean) as string[];
if (nodePathEntries.length > 0) {
process.env.NODE_PATH = nodePathEntries.join(':');
Module._initPaths();
}

const config: PlaywrightTestConfig = {
testDir,
timeout: 60_000,
fullyParallel: true,
expect: {
timeout: 15_000,
Expand All @@ -22,33 +47,81 @@ const config: PlaywrightTestConfig = {
['junit', { outputFile: junitOutputFile }],
],
globalSetup: path.resolve(__dirname, 'playwright.global-setup.ts'),
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
headless: true,
trace: 'retain-on-failure',
launchOptions: {
args: launchArgs,
projects: [
{
name: 'frontend-e2e',
testDir: frontendTestDir,
timeout: 60_000,
use: {
...sharedUseOptions,
baseURL: process.env.BASE_URL || 'http://localhost:3000',
env: frontendEnv,
},
Comment on lines +55 to +59
Copy link
Contributor

@coderabbitai coderabbitai bot Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

use.env is not a supported Playwright option (TS/type error and no effect).

This will fail type‑checking under PlaywrightTestConfig and won’t set process.env for tests. Tests already read process.env.* directly, and webServer.env correctly seeds Next’s dev server.

  • Remove env under use for both projects.
  • Keep env only where supported (webServer.env).
  • Also avoid actionTimeout: 0; prefer a bounded value to prevent hangs.

Apply this diff:

@@
-      use: {
-        ...sharedUseOptions,
-        baseURL: process.env.BASE_URL || 'http://localhost:3000',
-        env: frontendEnv,
-      },
+      use: {
+        ...sharedUseOptions,
+        baseURL: process.env.BASE_URL || 'http://localhost:3000',
+      },
@@
-      use: {
-        ...sharedUseOptions,
-        actionTimeout: 0,
-        baseURL: process.env.BACKEND_BASE_URL || 'http://localhost:8787',
-        env: {
-          SHOPIFY_STORE_DOMAIN: process.env.SHOPIFY_STORE_DOMAIN || '',
-          SHOPIFY_STOREFRONT_ACCESS_TOKEN: process.env.SHOPIFY_STOREFRONT_ACCESS_TOKEN || '',
-          SHOPIFY_STOREFRONT_HELPER_URL: process.env.SHOPIFY_STOREFRONT_HELPER_URL || '',
-          SHOPIFY_STOREFRONT_HELPER_TOKEN: process.env.SHOPIFY_STOREFRONT_HELPER_TOKEN || '',
-          SHOPIFY_TEST_VARIANT_ID: process.env.SHOPIFY_TEST_VARIANT_ID || '',
-          SHOPIFY_TEST_CURRENCY: process.env.SHOPIFY_TEST_CURRENCY || 'USD',
-          SHOPIFY_MODULE_CONFIG_TABLE: process.env.SHOPIFY_MODULE_CONFIG_TABLE || 'shopify_module_configurations',
-          SHOPIFY_ORDER_TABLE: process.env.SHOPIFY_ORDER_TABLE || 'shopify_orders',
-          SUPABASE_URL: process.env.SUPABASE_URL || '',
-          SUPABASE_SERVICE_ROLE_KEY: process.env.SUPABASE_SERVICE_ROLE_KEY || '',
-          NODE_PATH: [path.resolve(__dirname, 'frontend/node_modules'), process.env.NODE_PATH]
-            .filter(Boolean)
-            .join(':'),
-        },
-      },
+      use: {
+        ...sharedUseOptions,
+        actionTimeout: process.env.CI ? 30_000 : 45_000,
+        baseURL: process.env.BACKEND_BASE_URL || 'http://localhost:8787',
+      },

Also applies to: 67-86

🤖 Prompt for AI Agents 
In playwright.config.ts around lines 55–59 (and similarly lines 67–86), the
configuration incorrectly sets use.env (unsupported by Playwright types and has
no effect) and sets actionTimeout: 0; remove the env property from each use
block so you only seed environment variables via webServer.env, and replace
actionTimeout: 0 with a reasonable bounded timeout (e.g., a nonzero millisecond
value) to avoid potential test hangs.

},
env: {
NEXT_PUBLIC_BACKEND_HOST: process.env.NEXT_PUBLIC_BACKEND_HOST || 'localhost',
NEXT_PUBLIC_BACKEND_PORT: process.env.NEXT_PUBLIC_BACKEND_PORT || '3000',
NEXT_PUBLIC_DISABLE_ENVIRONMENT: 'true',
NEXT_PUBLIC_DISABLE_MODEL_LOADING: 'true',
{
name: 'shopify-integration',
testDir: shopifyTestDir,
testMatch: /shopify-checkout\.spec\.ts/,
timeout: process.env.CI ? 300_000 : 180_000,
retries: process.env.CI ? 2 : 0,
use: {
...sharedUseOptions,
actionTimeout: 0,
baseURL: process.env.BACKEND_BASE_URL || 'http://localhost:8787',
env: {
SHOPIFY_STORE_DOMAIN: process.env.SHOPIFY_STORE_DOMAIN || '',
SHOPIFY_STOREFRONT_ACCESS_TOKEN: process.env.SHOPIFY_STOREFRONT_ACCESS_TOKEN || '',
SHOPIFY_STOREFRONT_HELPER_URL: process.env.SHOPIFY_STOREFRONT_HELPER_URL || '',
SHOPIFY_STOREFRONT_HELPER_TOKEN: process.env.SHOPIFY_STOREFRONT_HELPER_TOKEN || '',
SHOPIFY_TEST_VARIANT_ID: process.env.SHOPIFY_TEST_VARIANT_ID || '',
SHOPIFY_TEST_CURRENCY: process.env.SHOPIFY_TEST_CURRENCY || 'USD',
SHOPIFY_MODULE_CONFIG_TABLE: process.env.SHOPIFY_MODULE_CONFIG_TABLE || 'shopify_module_configurations',
SHOPIFY_ORDER_TABLE: process.env.SHOPIFY_ORDER_TABLE || 'shopify_orders',
SUPABASE_URL: process.env.SUPABASE_URL || '',
SUPABASE_SERVICE_ROLE_KEY: process.env.SUPABASE_SERVICE_ROLE_KEY || '',
NODE_PATH: [path.resolve(__dirname, 'frontend/node_modules'), process.env.NODE_PATH]
.filter(Boolean)
.join(':'),
},
},
},
},
webServer: {
command: 'npm run dev -- --hostname 0.0.0.0',
url: 'http://127.0.0.1:3000',
reuseExistingServer: !process.env.CI,
timeout: 120_000,
cwd: path.resolve(__dirname, 'frontend'),
env: {
NEXT_PUBLIC_BACKEND_HOST: process.env.NEXT_PUBLIC_BACKEND_HOST || 'localhost',
NEXT_PUBLIC_BACKEND_PORT: process.env.NEXT_PUBLIC_BACKEND_PORT || '3000',
NEXT_PUBLIC_DISABLE_ENVIRONMENT: 'true',
NEXT_PUBLIC_DISABLE_MODEL_LOADING: 'true',
},
},
],
webServer: skipWebServer
? undefined
: {
command: 'npm run dev -- --hostname 0.0.0.0',
url: 'http://127.0.0.1:3000',
reuseExistingServer: !process.env.CI,
timeout: 120_000,
cwd: path.resolve(__dirname, 'frontend'),
env: frontendEnv,
},
};

export default config;

function collectRequestedProjects(argv: string[]): string[] {
const names: string[] = [];
for (let index = 0; index < argv.length; index += 1) {
const arg = argv[index];
if (!arg.startsWith('--project')) {
continue;
}

if (arg === '--project') {
const value = argv[index + 1];
if (value) {
names.push(value);
}
index += 1;
continue;
}

const [, value] = arg.split('=');
if (value) {
names.push(value);
}
}

return names;
}
Loading
Loading