Skip to content

[E2E] Re-enable Woocommerce Blocks e2e tests #11005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 22 commits into
base: dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocommerce-and-php
Choose a base branch
from
+46 −33
Open

[E2E] Re-enable Woocommerce Blocks e2e tests #11005

Changes from 4 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c8d87ff
WIP
mgascam Aug 8, 2025
b04bb13
Update e2e-test workflow to include blocks test group and adjust job …
mgascam Aug 8, 2025
ef41d8d
Refine e2e-test workflow comments to clarify blocks tests strategy fo…
mgascam Aug 11, 2025
1d8b8b1
Update e2e-test workflow to include skip_blocks parameter for legacy …
mgascam Aug 11, 2025
1e8d58d
Refine comments in e2e-test workflow to clarify block-based checkout …
mgascam Aug 12, 2025
c40d544
Update e2e-test workflow to include block-based checkout tests for L-…
mgascam Aug 12, 2025
fcfbd76
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 13, 2025
ae31c90
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 13, 2025
518769e
Add support for 'blocks' test group in E2E pull request workflow
mgascam Aug 13, 2025
e2c46cd
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 13, 2025
a2220a1
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 13, 2025
38c89e1
Amend docs
mgascam Aug 13, 2025
3f3e1c6
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 13, 2025
b9e6b4e
Update e2e-test.yml to simplify SKIP_WC_BLOCKS_TESTS condition
mgascam Aug 13, 2025
c504b85
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 13, 2025
530a068
Remove skips_blocks field
mgascam Aug 14, 2025
70199e9
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 14, 2025
0712bdf
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 14, 2025
ff14dc3
Merge remote-tracking branch 'origin/dev/woopmnt-5249-e2e-ensure-vers…
mgascam Aug 14, 2025
fd59905
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 14, 2025
22f78b5
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Aug 14, 2025
c4ea9b6
Merge branch 'dev/woopmnt-5249-e2e-ensure-version-coverage-for-woocom…
mgascam Sep 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 33 additions & 39 deletions .github/workflows/e2e-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,58 +94,70 @@ jobs:

# Create optimized matrix with selective PHP version testing
# Build matrix dynamically based on WC versions
#
# BLOCKS TESTS STRATEGY:
# - Run blocks tests on current stable and newer WC versions where blocks support is mature
# - WC 7.7.0: NO blocks tests (too old, limited support) - uses skip_blocks=1
# - WC L-1: NO blocks tests (previous major, potential compatibility issues) - uses skip_blocks=1
# - WC latest: YES blocks tests (current stable, full support)
# - WC beta: YES blocks tests (newer version, test latest features)
# - WC RC: YES blocks tests (newest version, test upcoming release)

# Define common values to reduce repetition
PHP_LEGACY="7.3"
PHP_STABLE="8.3"
PHP_LATEST="8.4"
TEST_GROUPS_WCPAY="wcpay"
TEST_GROUPS_SUBSCRIPTIONS="subscriptions"
TEST_GROUPS_BLOCKS="blocks"
TEST_BRANCHES_MERCHANT="merchant"
TEST_BRANCHES_SHOPPER="shopper"

# Initialize empty matrix array
MATRIX_ENTRIES=()

# Add WC 7.7.0 with PHP 7.3 only (legacy)
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
# Add WC 7.7.0 with PHP 7.3 only (legacy - NO blocks tests)
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\",\"skip_blocks\":\"1\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\",\"skip_blocks\":\"1\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\",\"skip_blocks\":\"1\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"7.7.0\",\"php\":\"$PHP_LEGACY\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\",\"skip_blocks\":\"1\"}")

# Add L-1 version with PHP 8.3 only
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
# Add L-1 version with PHP 8.3 only (NO blocks tests - older version)
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\",\"skip_blocks\":\"1\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\",\"skip_blocks\":\"1\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\",\"skip_blocks\":\"1\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$L1_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\",\"skip_blocks\":\"1\"}")

# Add latest with PHP 8.3 only
# Add latest with PHP 8.3 (INCLUDE blocks tests - current stable, full support)
MATRIX_ENTRIES+=("{\"woocommerce\":\"latest\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"latest\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"latest\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"latest\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"latest\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_BLOCKS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")

# Add beta with PHP 8.3 only (only if beta version is available)
# Add beta with PHP 8.3 (INCLUDE blocks tests - newer version)
if [[ -n "$BETA_VERSION" ]]; then
MATRIX_ENTRIES+=("{\"woocommerce\":\"$BETA_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$BETA_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$BETA_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$BETA_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$BETA_VERSION\",\"php\":\"$PHP_STABLE\",\"test_groups\":\"$TEST_GROUPS_BLOCKS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
fi

# Add rc with PHP 8.4 only
# Add rc with PHP 8.4 (INCLUDE blocks tests - newest version)
MATRIX_ENTRIES+=("{\"woocommerce\":\"$RC_VERSION\",\"php\":\"$PHP_LATEST\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$RC_VERSION\",\"php\":\"$PHP_LATEST\",\"test_groups\":\"$TEST_GROUPS_WCPAY\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$RC_VERSION\",\"php\":\"$PHP_LATEST\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_MERCHANT\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$RC_VERSION\",\"php\":\"$PHP_LATEST\",\"test_groups\":\"$TEST_GROUPS_SUBSCRIPTIONS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")
MATRIX_ENTRIES+=("{\"woocommerce\":\"$RC_VERSION\",\"php\":\"$PHP_LATEST\",\"test_groups\":\"$TEST_GROUPS_BLOCKS\",\"test_branches\":\"$TEST_BRANCHES_SHOPPER\"}")

# Convert array to JSON
MATRIX_INCLUDE=$(printf '%s\n' "${MATRIX_ENTRIES[@]}" | jq -s . -c)

echo "matrix={\"include\":$MATRIX_INCLUDE}" >> $GITHUB_OUTPUT

# Run WCPay & subscriptions tests against specific WC versions with PHP variations
wcpay-subscriptions-tests:
# Run WCPay, subscriptions & blocks tests against specific WC versions with PHP variations
wcpay-subscriptions-blocks-tests:
runs-on: ubuntu-latest
needs: generate-matrix
strategy:
Expand All @@ -160,7 +172,9 @@ jobs:
E2E_PHP_VERSION: ${{ matrix.php }}
E2E_GROUP: ${{ matrix.test_groups }}
E2E_BRANCH: ${{ matrix.test_branches }}
SKIP_WC_BLOCKS_TESTS: 1 # skip running blocks tests
SKIP_WC_SUBSCRIPTIONS_TESTS: ${{ matrix.test_groups == 'blocks' && '1' || '' }}
SKIP_WC_ACTION_SCHEDULER_TESTS: ${{ matrix.test_groups == 'blocks' && '1' || '' }}
SKIP_WC_BLOCKS_TESTS: ${{ matrix.skip_blocks == '1' && '1' || '' }}

steps:
- name: Checkout WCPay repository
Expand All @@ -176,38 +190,18 @@ jobs:
- name: Run tests, upload screenshots & logs
uses: ./.github/actions/e2e/run-log-tests

# Run tests against WC Checkout blocks & WC latest
# [TODO] Unskip blocks tests after investigating constant failures.
# blocks-tests:
# runs-on: ubuntu-latest
# name: WC - latest | blocks - shopper

# env:
# E2E_WP_VERSION: 'latest'
# E2E_WC_VERSION: 'latest'
# E2E_GROUP: 'blocks'
# E2E_BRANCH: 'shopper'
# SKIP_WC_SUBSCRIPTIONS_TESTS: 1 #skip installing & running subscriptions tests
# SKIP_WC_ACTION_SCHEDULER_TESTS: 1 #skip installing & running action scheduler tests

# steps:
# - name: Checkout WCPay repository
# uses: actions/checkout@v4

# - name: Setup E2E environment
# uses: ./.github/actions/e2e/env-setup

# - name: Run tests, upload screenshots & logs
# uses: ./.github/actions/e2e/run-log-tests

# Run tests against WP Nightly & WC latest
# This section tests cutting-edge WordPress features with latest WC
# Blocks tests are included here since we're testing against WP nightly + WC latest
wp-nightly-tests:
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
test_groups: [ 'wcpay', 'subscriptions' ] # [TODO] Unskip blocks tests after investigating constant failures.
test_groups: [ 'wcpay', 'subscriptions', 'blocks' ]
test_branches: [ 'merchant', 'shopper' ]

name: WP - nightly | WC - latest | ${{ matrix.test_groups }} - ${{ matrix.test_branches }}
Expand Down
Loading