Skip to content

Commit ff5850e

Browse files
desrosjdesrosj
committed
Build/Test Tools: Trim down the upgrade testing matrix.
The upgrade testing workflow is currently at ~978 jobs spawned from the strategy matrix. While it's great to test all possible combinations, GitHub's UI cannot keep up with tracking that number of jobs, often taking 30-45 minutes to accurately report the outcome even though the jobs themselves all complete in under 5 minutes. This is 2x the total number of concurrent jobs allowed for the entire organization (which creates a backlog and slows everything down even more). This trims down the number of combinations included in the testing matrices to be a bit more thoughtful following the following methodology: - The last two releases of WordPress are tested against all PHP and MySQL LTS version combinations and the most recent innovation release. - The next 6 oldest versions of WordPress are tested against both the oldest and newest releases of PHP currently supported for both PHP 7 & 8 along with the oldest and newest MySQL LTS versions currently supported (no innovation releases). - For the remaining versions of WordPress receiving security updates, they are only included if the database version was different that the previous major release. - The oldest version of WordPress receiving security updates should always be tested against the same full list of combinations as the last two releases. When choosing which MySQL versions to test against: - Only the most recent innovation release should be included in testing. - Even though MySQL >= 5.5.5 is currently supported, there are no 5.5.x Docker containers available that work on modern architectures. - 5.6.x Docker containers are available and work, but 5.6 only accounts for ~2.3% of installs as of 12/6/2024.defaults: - 5.7.x accounts for ~20% of installs, so this is used below instead. With these changes, the total number of jobs is reduced by ~58%. Props johnbillion, mukesh27. See #62221. git-svn-id: https://develop.svn.wordpress.org/trunk@59508 602fd350-edb4-49c9-b593-d223f7449a82
1 parent b4969d4 commit ff5850e

File tree

1 file changed

+97
-26
lines changed

1 file changed

+97
-26
lines changed

‎.github/workflows/upgrade-testing.yml‎

Lines changed: 97 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# Confirms that updating WordPress using WP-CLI works successfully.
2+
#
3+
# This workflow is not meant to test wordpress-develop checkouts, but rather tagged versions officially available on WordPress.org.
14
name: Upgrade Tests
25

36
on:
@@ -31,9 +34,28 @@ concurrency:
3134
# Any needed permissions should be configured at the job level.
3235
permissions: {}
3336

37+
# Because the number of jobs spawned can quickly balloon out of control, the following methodology is applied when
38+
# building out the matrix below:
39+
#
40+
# - The last two releases of WordPress are tested against all PHP/MySQL LTS version combinations and the most recent
41+
# innovation release.
42+
# - The next 6 oldest versions of WordPress are tested against both the oldest and newest releases of PHP currently
43+
# supported for both PHP 7 & 8 along with the oldest and newest MySQL LTS versions currently supported (no innovation
44+
# releases). At the current 3 releases per year pace, this accounts for 2 additional years worth of releases.
45+
# - Of the remaining versions of WordPress still receiving security updates, only test the ones where the database
46+
# version was updated since the previous major release.
47+
# - The oldest version of WordPress receiving security updates should always be tested against the same combinations as
48+
# detailed for the last two releases.
49+
50+
# Notes about chosen MySQL versions:
51+
# - Only the most recent innovation release should be included in testing.
52+
# - Even though MySQL >= 5.5.5 is currently supported, there are no 5.5.x Docker containers available that work on
53+
# modern architectures.
54+
# - 5.6.x Docker containers are available and work, but 5.6 only accounts for ~2.3% of installs as of 12/6/2024.defaults:
55+
# - 5.7.x accounts for ~20% of installs, so this is used below instead.
3456
jobs:
35-
# Spawns upgrade testing from WordPress 6.x versions on PHP 8.x with MySQL.
36-
upgrade-tests-wp-6x-php-8x-mysql:
57+
# Tests the full list of PHP/MySQL combinations for the last two versions of WordPress.
58+
upgrade-tests-last-two-releases:
3759
name: ${{ matrix.wp }} to ${{ inputs.new-version && inputs.new-version || 'latest' }}
3860
uses: WordPress/wordpress-develop/.github/workflows/reusable-upgrade-testing.yml@trunk
3961
if: ${{ github.repository == 'WordPress/wordpress-develop' || github.event_name == 'pull_request' }}
@@ -43,12 +65,22 @@ jobs:
4365
fail-fast: false
4466
matrix:
4567
os: [ 'ubuntu-latest' ]
46-
php: [ '8.0', '8.1', '8.2', '8.3', '8.4' ]
68+
php: [ '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3', '8.4' ]
4769
db-type: [ 'mysql' ]
48-
db-version: [ '5.7', '8.0', '8.4' ]
49-
wp: [ '6.0', '6.1', '6.2', '6.3', '6.4', '6.5', '6.6', '6.7' ]
70+
db-version: [ '5.7', '8.0', '8.4', '9.1' ]
71+
wp: [ '6.6', '6.7' ]
5072
multisite: [ false, true ]
5173

74+
exclude:
75+
- php: '7.2'
76+
db-version: '8.4'
77+
- php: '7.3'
78+
db-version: '8.4'
79+
# MySQL 9.0+ will not work on PHP 7.2 & 7.3
80+
- php: '7.2'
81+
db-version: '9.1'
82+
- php: '7.3'
83+
db-version: '9.1'
5284
with:
5385
os: ${{ matrix.os }}
5486
php: ${{ matrix.php }}
@@ -58,8 +90,8 @@ jobs:
5890
new-version: ${{ inputs.new-version && inputs.new-version || 'latest' }}
5991
multisite: ${{ matrix.multisite }}
6092

61-
# Spawns upgrade testing from WordPress 6.x versions on PHP 7.x with MySQL.
62-
upgrade-tests-wp-6x-php-7x-mysql:
93+
# Tests the remaining 6.x releases on the oldest and newest supported versions of PHP 7 & 8.
94+
upgrade-tests-wp-6x-mysql:
6395
name: ${{ matrix.wp }} to ${{ inputs.new-version && inputs.new-version || 'latest' }}
6496
uses: WordPress/wordpress-develop/.github/workflows/reusable-upgrade-testing.yml@trunk
6597
if: ${{ github.repository == 'WordPress/wordpress-develop' || github.event_name == 'pull_request' }}
@@ -69,10 +101,10 @@ jobs:
69101
fail-fast: false
70102
matrix:
71103
os: [ 'ubuntu-latest' ]
72-
php: [ '7.2', '7.3', '7.4' ]
104+
php: [ '7.2', '7.4', '8.0', '8.4' ]
73105
db-type: [ 'mysql' ]
74-
db-version: [ '5.7', '8.0', '8.4' ]
75-
wp: [ '6.0', '6.1', '6.2', '6.3', '6.4', '6.5', '6.6', '6.7' ]
106+
db-version: [ '5.7', '8.4' ]
107+
wp: [ '6.0', '6.1', '6.2', '6.3', '6.4', '6.5' ]
76108
multisite: [ false, true ]
77109

78110
exclude:
@@ -89,7 +121,7 @@ jobs:
89121
new-version: ${{ inputs.new-version && inputs.new-version || 'latest' }}
90122
multisite: ${{ matrix.multisite }}
91123

92-
# Spawns upgrade testing from WordPress 5.x versions on PHP 7.x with MySQL.
124+
# Tests 5.x releases where the WordPress database version changed on the oldest and newest supported versions of PHP 7.
93125
upgrade-tests-wp-5x-php-7x-mysql:
94126
name: ${{ matrix.wp }} to ${{ inputs.new-version && inputs.new-version || 'latest' }}
95127
uses: WordPress/wordpress-develop/.github/workflows/reusable-upgrade-testing.yml@trunk
@@ -98,10 +130,10 @@ jobs:
98130
fail-fast: false
99131
matrix:
100132
os: [ 'ubuntu-latest' ]
101-
php: [ '7.2', '7.3', '7.4' ]
133+
php: [ '7.2', '7.4' ]
102134
db-type: [ 'mysql' ]
103-
db-version: [ '5.7', '8.0', '8.4' ]
104-
wp: [ '5.0', '5.1', '5.2', '5.3', '5.4', '5.5', '5.6', '5.7', '5.8', '5.9' ]
135+
db-version: [ '5.7', '8.4' ]
136+
wp: [ '5.0', '5.1', '5.3', '5.4', '5.5', '5.6', '5.9' ]
105137
multisite: [ false, true ]
106138

107139
exclude:
@@ -118,7 +150,7 @@ jobs:
118150
new-version: ${{ inputs.new-version && inputs.new-version || 'latest' }}
119151
multisite: ${{ matrix.multisite }}
120152

121-
# Spawns upgrade testing from WordPress 5.x versions on PHP 8.x with MySQL.
153+
# Tests 5.x releases where the WordPress database version changed on the oldest and newest supported versions of PHP 8.
122154
#
123155
# WordPress 5.0-5.2 are excluded from PHP 8+ testing because of the following fatal errors:
124156
# - Use of __autoload().
@@ -131,10 +163,10 @@ jobs:
131163
fail-fast: false
132164
matrix:
133165
os: [ 'ubuntu-latest' ]
134-
php: [ '8.0', '8.1', '8.2', '8.3', '8.4' ]
166+
php: [ '8.0', '8.4' ]
135167
db-type: [ 'mysql' ]
136-
db-version: [ '5.7', '8.0', '8.4' ]
137-
wp: [ '5.3', '5.4', '5.5', '5.6', '5.7', '5.8', '5.9' ]
168+
db-version: [ '5.7', '8.4' ]
169+
wp: [ '5.3', '5.4', '5.5', '5.6', '5.9' ]
138170
multisite: [ false, true ]
139171
with:
140172
os: ${{ matrix.os }}
@@ -145,7 +177,9 @@ jobs:
145177
new-version: ${{ inputs.new-version && inputs.new-version || 'latest' }}
146178
multisite: ${{ matrix.multisite }}
147179

148-
# Spawns upgrade testing from WordPress 4.x versions on PHP 7.x with MySQL.
180+
# Tests 4.x releases where the WordPress database version changed on the oldest and newest supported versions of PHP 7.
181+
#
182+
# The oldest version of WordPress receiving security updates should always be tested.
149183
upgrade-tests-wp-4x-php-7x-mysql:
150184
name: ${{ matrix.wp }} to ${{ inputs.new-version && inputs.new-version || 'latest' }}
151185
uses: WordPress/wordpress-develop/.github/workflows/reusable-upgrade-testing.yml@trunk
@@ -154,10 +188,10 @@ jobs:
154188
fail-fast: false
155189
matrix:
156190
os: [ 'ubuntu-latest' ]
157-
php: [ '7.2', '7.3', '7.4' ]
191+
php: [ '7.2', '7.4' ]
158192
db-type: [ 'mysql' ]
159-
db-version: [ '5.7', '8.0', '8.4' ]
160-
wp: [ '4.1', '4.2', '4.3', '4.4', '4.5', '4.6', '4.7', '4.8', '4.9' ]
193+
db-version: [ '5.7', '8.4' ]
194+
wp: [ '4.1', '4.2', '4.3', '4.4', '4.5', '4.6', '4.7' ]
161195
multisite: [ false, true ]
162196

163197
exclude:
@@ -174,7 +208,9 @@ jobs:
174208
new-version: ${{ inputs.new-version && inputs.new-version || 'latest' }}
175209
multisite: ${{ matrix.multisite }}
176210

177-
# Spawns upgrade testing from WordPress 4.x versions on PHP 8.x with MySQL.
211+
# Tests 4.x releases where the WordPress database version changed on the oldest and newest supported versions of PHP 8.
212+
#
213+
# The oldest version of WordPress receiving security updates should always be tested.
178214
#
179215
# WordPress 4.6-4.9 are excluded from PHP 8+ testing because of the following fatal errors:
180216
# - Use of __autoload().
@@ -187,9 +223,9 @@ jobs:
187223
fail-fast: false
188224
matrix:
189225
os: [ 'ubuntu-latest' ]
190-
php: [ '8.0', '8.1', '8.2', '8.3', '8.4' ]
226+
php: [ '8.0', '8.4' ]
191227
db-type: [ 'mysql' ]
192-
db-version: [ '5.7', '8.0', '8.4' ]
228+
db-version: [ '5.7', '8.4' ]
193229
wp: [ '4.1', '4.2', '4.3', '4.4', '4.5' ]
194230
multisite: [ false, true ]
195231
with:
@@ -201,13 +237,48 @@ jobs:
201237
new-version: ${{ inputs.new-version && inputs.new-version || 'latest' }}
202238
multisite: ${{ matrix.multisite }}
203239

240+
# The oldest version of WordPress receiving security updates should always be tested against
241+
# the full list of PHP/MySQL combinations.
242+
upgrade-tests-oldest-wp-mysql:
243+
name: ${{ matrix.wp }} to ${{ inputs.new-version && inputs.new-version || 'latest' }}
244+
uses: WordPress/wordpress-develop/.github/workflows/reusable-upgrade-testing.yml@trunk
245+
if: ${{ github.repository == 'WordPress/wordpress-develop' || github.event_name == 'pull_request' }}
246+
strategy:
247+
fail-fast: false
248+
matrix:
249+
os: [ 'ubuntu-latest' ]
250+
php: [ '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3', '8.4' ]
251+
db-type: [ 'mysql' ]
252+
db-version: [ '5.7', '8.0', '8.4', '9.1' ]
253+
wp: [ '4.1' ]
254+
multisite: [ false, true ]
255+
256+
exclude:
257+
- php: '7.2'
258+
db-version: '8.4'
259+
- php: '7.3'
260+
db-version: '8.4'
261+
# MySQL 9.0+ will not work on PHP 7.2 & 7.3
262+
- php: '7.2'
263+
db-version: '9.1'
264+
- php: '7.3'
265+
db-version: '9.1'
266+
with:
267+
os: ${{ matrix.os }}
268+
php: ${{ matrix.php }}
269+
db-type: ${{ matrix.db-type }}
270+
db-version: ${{ matrix.db-version }}
271+
wp: ${{ matrix.wp }}
272+
new-version: ${{ inputs.new-version && inputs.new-version || 'latest' }}
273+
multisite: ${{ matrix.multisite }}
274+
204275
slack-notifications:
205276
name: Slack Notifications
206277
uses: WordPress/wordpress-develop/.github/workflows/slack-notifications.yml@trunk
207278
permissions:
208279
actions: read
209280
contents: read
210-
needs: [ upgrade-tests-wp-6x-php-8x-mysql, upgrade-tests-wp-6x-php-7x-mysql, upgrade-tests-wp-5x-php-7x-mysql, upgrade-tests-wp-5x-php-8x-mysql, upgrade-tests-wp-4x-php-7x-mysql, upgrade-tests-wp-4x-php-8x-mysql ]
281+
needs: [ upgrade-tests-last-two-releases, upgrade-tests-wp-6x-mysql, upgrade-tests-wp-5x-php-7x-mysql, upgrade-tests-wp-5x-php-8x-mysql, upgrade-tests-wp-4x-php-7x-mysql, upgrade-tests-wp-4x-php-8x-mysql, upgrade-tests-oldest-wp-mysql ]
211282
if: ${{ github.repository == 'WordPress/wordpress-develop' && github.event_name != 'pull_request' && always() }}
212283
with:
213284
calling_status: ${{ contains( needs.*.result, 'cancelled' ) && 'cancelled' || contains( needs.*.result, 'failure' ) && 'failure' || 'success' }}

0 commit comments

Comments
 (0)