Skip to content

Commit ca3f00e

Browse files
jrfnljrfnl
committed
Tests: allow for PHPUnit 10 and 11
This commit makes the necessary changes in the CI wiring to allow for running the tests on PHPUnit 8, 9, 10 and 11 and still failing them on PHP deprecations and notices like before. The PHPUnit configuration file specification has undergone changes in PHPUnit 9.3, 10.0, 10.1 and 10.5.32/11.3.3. Most notably: * In PHPUnit 9.3, the manner of specifying the code coverage configuration has changed, though the "old" way is still supported on PHPUnit 9, but not on PHPUnit 10. * In PHPUnit 10.0, a significant number of attributes of the `<phpunit>` element were removed or renamed and failing test runs on PHP deprecations or notices was no longer possible. * In PHPUnit 10.1 a further iteration was released on the way to specify code coverage, which now expects a `<source>` element. Additionally, it is now possible again to fail a test run on PHP deprecations/notices. * In PHPUnit 10.5.32/11.3.3 new `displayDetailsOnPhpunitDeprecations` and `failOnPhpunitDeprecation` options were introduced. As of PHPUnit 10.0, the option to fail tests on (PHP) deprecation notices would **_also_** fail the tests on PHPUnit deprecation notices, while those are only relevant when upgrading to the _next_ PHPUnit version. This made things a bit awkward. This was, at long last, fixed via the new options in PHPUnit 10.5.32/11.3.3. While the `--migrate-configuration` command can upgrade a configuration for the changes in the format made in PHPUnit 9.3, some of the changes in the configuration format in PHPUnit 10 don't have one-on-one replacements and/or are not taken into account. With that in mind, I deem it more appropriate to have a dedicated PHPUnit configuration file for PHPUnit 10/11 to ensure the test runs will behave as intended. To be forward compatible, the default PHPUnit config file `phpunit.xml.dist` will now use the PHPUnit 10+ format, while a separate `phpunit-lte9.xml.dist` file will be used for PHPUnit 8 and 9. > ☝️ This means that contributors who run on recent PHP versions, should just be able to run `vendor/bin/phpunit`. Only contributors who run on PHP < 8.1 and therefore run on PHPUnit < 10, will need to be aware of the difference in the config files. All in all, this commit makes the following changes: * Rename the "old" PHPUnit configuration file to `phpunit-lte9.xml.dist` - This file has also been added to `.gitattributes` to ignore for distribution packages. - An entry for `phpunit-lte9.xml` has been added to the `.gitignore` file. While a contributor can choose the name for a config overload file for the `phpunit-lte9.xml.dist` file freely, it is a good convention to use the same name as the file being overloaded, minus the `.dist`, so ignoring that filename by default seems appropriate. - Composer scripts have been added to run the tests with the `phpunit-lte9.xml.dist` file and these new scripts are also mentioned in the `CONTRIBUTING` file. * The `phpunit.xml.dist` now contains the PHPUnit configuration for running the tests on PHPUnit 10.5+. This includes the appropriate settings to fail the test runs on any PHP deprecations, notices or warnings. Take note that the `cacheDirectory` option has been set to a custom directory which is already git ignored. * The GH Actions scripts have been updated to include a PHPUnit version based toggle and run the tests with the correct configuration file depending on the PHPUnit version being used. * Composer: allow for PHPUnit 10 on version 10.5.32 or higher and PHPUnit 11 on v 11.3.3 or higher. The 10.5.32/11.3.3 version are deliberately set as the minimum as: - the `failOnNotice` and `failOnDeprecation` attributes can now be set in the XML config file (PHPUnit 10.1+). - the `source` element for the XML config file was introduced in 10.1. - the `displayDetailsOnPhpunitDeprecations` and `failOnPhpunitDeprecation` attributes can be set (PHPUnit 10.5.32/11.3.3+). Refs: * https://phpunit.de/announcements/phpunit-10.html * sebastianbergmann/phpunit 5196 * sebastianbergmann/phpunit@fb6673f * sebastianbergmann/phpunit 5937 * sebastianbergmann/phpunit@1a5d2b8
1 parent 78e5d09 commit ca3f00e

File tree

9 files changed

+186
-34
lines changed

9 files changed

+186
-34
lines changed

‎.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ tests/Core/**/ export-ignore
1919
phpcs.xml.dist export-ignore
2020
phpstan.neon.dist export-ignore
2121
phpunit.xml.dist export-ignore
22+
phpunit-lte9.xml.dist export-ignore
2223
tests/Core/ErrorSuppressionTest.php export-ignore
2324

2425
#

‎.github/CONTRIBUTING.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -259,10 +259,14 @@ To help you with this, a number of convenience scripts are available:
259259
* `composer check-all` will run the `cs` + `test` checks in one go.
260260
* `composer cs` will check for code style violations.
261261
* `composer cbf` will run the autofixers for code style violations.
262-
* `composer test` will run the unit tests.
263-
* `composer coverage` will run the unit tests with code coverage and show a text summary.
262+
* `composer test` will run the unit tests when using PHP 8.1+/PHPUnit 10+.
263+
* `composer test-lte9` will run the unit tests when using PHP < 8.1/PHPUnit <= 9.
264+
* `composer coverage` will run the unit tests with code coverage and show a text summary (PHP 8.1+/PHPUnit 10+).
265+
* `composer coverage-lte9` will run the unit tests with code coverage and show a text summary (PHP < 8.1/PHPUnit <= 9).
264266
* `composer coverage-local` will run the unit tests with code coverage and generate an HTML coverage report,
265-
which will be placed in a `build/coverage-html` subdirectory.
267+
which will be placed in a `build/coverage-html` subdirectory (PHP 8.1+/PHPUnit 10+).
268+
* `composer coverage-lte9-local` will run the unit tests with code coverage and generate an HTML coverage report,
269+
which will be placed in a `build/coverage-html` subdirectory (PHP < 8.1/PHPUnit <= 9).
266270
* `composer build` will build the phpcs.phar and phpcbf.phar files.
267271

268272

‎.github/workflows/quicktest.yml‎

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,19 +55,44 @@ jobs:
5555
# Bust the cache at least once a month - output format: YYYY-MM.
5656
custom-cache-suffix: $(date -u "+%Y-%m")
5757

58+
- name: Grab PHPUnit version
59+
id: phpunit_version
60+
shell: bash
61+
# yamllint disable-line rule:line-length
62+
run: echo "VERSION=$(php "vendor/bin/phpunit" --version | grep --only-matching --max-count=1 --extended-regexp '\b[0-9]+\.[0-9]+')" >> "$GITHUB_OUTPUT"
63+
64+
- name: "DEBUG: Show grabbed version"
65+
run: echo ${{ steps.phpunit_version.outputs.VERSION }}
66+
67+
- name: Determine PHPUnit config file to use
68+
id: phpunit_config
69+
shell: bash
70+
run: |
71+
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
72+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
73+
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) }}" == "true" ]; then
74+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
75+
else
76+
echo 'FILE=phpunit-lte9.xml.dist' >> "$GITHUB_OUTPUT"
77+
fi
78+
5879
- name: 'PHPCS: set the path to PHP'
5980
run: php "bin/phpcs" --config-set php_path php
6081

6182
- name: 'PHPUnit: run the full test suite'
6283
if: ${{ matrix.os != 'windows-latest' }}
63-
run: php "vendor/bin/phpunit" --no-coverage
84+
run: php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
6485

6586
- name: 'PHPUnit: run tests which may have different outcomes on Windows'
6687
if: ${{ matrix.os == 'windows-latest' }}
67-
run: php "vendor/bin/phpunit" --group Windows --no-coverage
88+
run: >
89+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
90+
--group Windows
6891
6992
- name: 'PHPUnit: run select tests in CBF mode'
70-
run: php "vendor/bin/phpunit" --group CBF --exclude-group nothing --no-coverage
93+
run: >
94+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
95+
--group CBF --exclude-group nothing
7196
env:
7297
PHP_CODESNIFFER_CBF: '1'
7398

‎.github/workflows/test.yml‎

Lines changed: 44 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -197,18 +197,41 @@ jobs:
197197
composer-options: ${{ matrix.php == '8.5' && '--ignore-platform-req=php+' || '' }}
198198
custom-cache-suffix: $(date -u "+%Y-%m")
199199

200+
- name: Grab PHPUnit version
201+
id: phpunit_version
202+
shell: bash
203+
# yamllint disable-line rule:line-length
204+
run: echo "VERSION=$(php "vendor/bin/phpunit" --version | grep --only-matching --max-count=1 --extended-regexp '\b[0-9]+\.[0-9]+')" >> "$GITHUB_OUTPUT"
205+
206+
- name: "DEBUG: Show grabbed version"
207+
run: echo ${{ steps.phpunit_version.outputs.VERSION }}
208+
209+
- name: Determine PHPUnit config file to use
210+
id: phpunit_config
211+
shell: bash
212+
run: |
213+
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
214+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
215+
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) }}" == "true" ]; then
216+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
217+
else
218+
echo 'FILE=phpunit-lte9.xml.dist' >> "$GITHUB_OUTPUT"
219+
fi
220+
200221
# Note: The code style check is run multiple times against every PHP version
201222
# as it also acts as an integration test.
202223
- name: 'PHPCS: set the path to PHP'
203224
run: php "bin/phpcs" --config-set php_path php
204225

205226
- name: 'PHPUnit: run the full test suite without code coverage'
206227
if: ${{ matrix.skip_tests != true }}
207-
run: php "vendor/bin/phpunit" --no-coverage
228+
run: php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
208229

209230
- name: 'PHPUnit: run select tests in CBF mode'
210231
if: ${{ matrix.skip_tests != true }}
211-
run: php "vendor/bin/phpunit" --group CBF --exclude-group nothing --no-coverage
232+
run: >
233+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
234+
--group CBF --exclude-group nothing
212235
env:
213236
PHP_CODESNIFFER_CBF: '1'
214237

@@ -293,6 +316,18 @@ jobs:
293316
- name: "DEBUG: Show grabbed version"
294317
run: echo ${{ steps.phpunit_version.outputs.VERSION }}
295318

319+
- name: Determine PHPUnit config file to use
320+
id: phpunit_config
321+
shell: bash
322+
run: |
323+
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
324+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
325+
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) }}" == "true" ]; then
326+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
327+
else
328+
echo 'FILE=phpunit-lte9.xml.dist' >> "$GITHUB_OUTPUT"
329+
fi
330+
296331
- name: 'PHPCS: set the path to PHP'
297332
run: php "bin/phpcs" --config-set php_path php
298333

@@ -302,18 +337,20 @@ jobs:
302337
# Using that option prevents issues with PHP-Parser backfilling PHP tokens during our test runs.
303338
- name: "Warm the PHPUnit cache (PHPUnit 9.3+)"
304339
if: ${{ steps.phpunit_version.outputs.VERSION >= '9.3' }}
305-
run: php "vendor/bin/phpunit" --coverage-cache ./build/phpunit-cache --warm-coverage-cache
340+
run: >
341+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
342+
--coverage-cache ./build/phpunit-cache --warm-coverage-cache
306343
307344
- name: "Run the unit tests with code coverage"
308345
if: ${{ matrix.os != 'windows-latest' }}
309346
run: >
310-
php "vendor/bin/phpunit"
347+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
311348
${{ steps.phpunit_version.outputs.VERSION >= '9.3' && '--coverage-cache ./build/phpunit-cache' || '' }}
312349
313350
- name: "Run select tests in CBF mode with code coverage"
314351
if: ${{ matrix.os != 'windows-latest' }}
315352
run: >
316-
php "vendor/bin/phpunit"
353+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
317354
${{ steps.phpunit_version.outputs.VERSION >= '9.3' && '--coverage-cache ./build/phpunit-cache' || '' }}
318355
--group CBF --exclude-group nothing --coverage-clover build/logs/clover-cbf.xml
319356
env:
@@ -322,8 +359,9 @@ jobs:
322359
- name: "Run the unit tests which may have different outcomes on Windows with code coverage"
323360
if: ${{ matrix.os == 'windows-latest' }}
324361
run: >
325-
php "vendor/bin/phpunit" --group Windows
362+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
326363
${{ steps.phpunit_version.outputs.VERSION >= '9.3' && '--coverage-cache ./build/phpunit-cache' || '' }}
364+
--group Windows
327365
328366
- name: "Upload coverage results to Coveralls (normal run)"
329367
if: ${{ success() }}

‎.github/workflows/validate.yml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,12 @@ jobs:
5656
pattern: "phpcs.xml.dist"
5757
xsd-file: "phpcs.xsd"
5858

59-
- name: "Validate PHPUnit config for well-formedness"
59+
- name: "Validate PHPUnit <= 9 config for well-formedness"
60+
uses: phpcsstandards/xmllint-validate@v1
61+
with:
62+
pattern: "phpunit-lte9.xml.dist"
63+
64+
- name: "Validate PHPUnit 10+ config for well-formedness"
6065
uses: phpcsstandards/xmllint-validate@v1
6166
with:
6267
pattern: "phpunit.xml.dist"

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
/CodeSniffer.conf
22
/phpcs.xml
33
/phpunit.xml
4+
/phpunitlte9.xml
5+
/phpunit-lte9.xml
46
.phpunit.result.cache
57
/build/
68
.idea/*

‎composer.json‎

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
"ext-xmlwriter": "*"
3737
},
3838
"require-dev": {
39-
"phpunit/phpunit": "^8.0 || ^9.3.4"
39+
"phpunit/phpunit": "^8.0 || ^9.3.4 || ^10.5.32 || ^11.3.3"
4040
},
4141
"bin": [
4242
"bin/phpcbf",
@@ -61,14 +61,26 @@
6161
"Composer\\Config::disableProcessTimeout",
6262
"@php ./vendor/phpunit/phpunit/phpunit --no-coverage"
6363
],
64+
"test-lte9": [
65+
"Composer\\Config::disableProcessTimeout",
66+
"@php ./vendor/phpunit/phpunit/phpunit -c phpunit-lte9.xml.dist --no-coverage"
67+
],
6468
"coverage": [
6569
"Composer\\Config::disableProcessTimeout",
6670
"@php ./vendor/phpunit/phpunit/phpunit -d max_execution_time=0"
6771
],
72+
"coverage-lte9": [
73+
"Composer\\Config::disableProcessTimeout",
74+
"@php ./vendor/phpunit/phpunit/phpunit -c phpunit-lte9.xml.dist -d max_execution_time=0"
75+
],
6876
"coverage-local": [
6977
"Composer\\Config::disableProcessTimeout",
7078
"@php ./vendor/phpunit/phpunit/phpunit --coverage-html ./build/coverage-html -d max_execution_time=0"
7179
],
80+
"coverage-lte9-local": [
81+
"Composer\\Config::disableProcessTimeout",
82+
"@php ./vendor/phpunit/phpunit/phpunit -c phpunit-lte9.xml.dist --coverage-html ./build/coverage-html -d max_execution_time=0"
83+
],
7284
"build": [
7385
"Composer\\Config::disableProcessTimeout",
7486
"@php -d phar.readonly=0 -f ./scripts/build-phar.php"
@@ -81,9 +93,12 @@
8193
"scripts-descriptions": {
8294
"cs": "Check for code style violations.",
8395
"cbf": "Fix code style violations.",
84-
"test": "Run the unit tests without code coverage.",
85-
"coverage": "Run the unit tests with code coverage.",
86-
"coverage-local": "Run the unit tests with code coverage and generate an HTML report in a 'build' directory.",
96+
"test": "PHPUnit 10+: Run the unit tests without code coverage.",
97+
"test-lte9": "PHPUnit <= 9: Run the unit tests without code coverage.",
98+
"coverage": "PHPUnit 10+: Run the unit tests with code coverage.",
99+
"coverage-lte9": "PHPUnit <= 9: Run the unit tests with code coverage.",
100+
"coverage-local": "PHPUnit 10+: Run the unit tests with code coverage and generate an HTML report in a 'build' directory.",
101+
"coverage-lte9-local": "PHPUnit <= 9: Run the unit tests with code coverage and generate an HTML report in a 'build' directory.",
87102
"build": "Create PHAR files for PHPCS and PHPCBF.",
88103
"check-all": "Run all checks (phpcs, tests)."
89104
}

‎phpunit-lte9.xml.dist‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<phpunit
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.2/phpunit.xsd"
5+
backupGlobals="true"
6+
beStrictAboutOutputDuringTests="true"
7+
beStrictAboutTestsThatDoNotTestAnything="false"
8+
bootstrap="tests/bootstrap.php"
9+
convertErrorsToExceptions="true"
10+
convertWarningsToExceptions="true"
11+
convertNoticesToExceptions="true"
12+
convertDeprecationsToExceptions="true"
13+
forceCoversAnnotation="true"
14+
>
15+
<testsuites>
16+
<testsuite name="PHPCS_Core">
17+
<directory>./tests/Core/</directory>
18+
</testsuite>
19+
<testsuite name="PHPCS_Sniffs">
20+
<directory>./src/Standards/Generic/Tests/</directory>
21+
<directory>./src/Standards/PEAR/Tests/</directory>
22+
<directory>./src/Standards/PSR1/Tests/</directory>
23+
<directory>./src/Standards/PSR2/Tests/</directory>
24+
<directory>./src/Standards/PSR12/Tests/</directory>
25+
<directory>./src/Standards/Squiz/Tests/</directory>
26+
<directory>./src/Standards/Zend/Tests/</directory>
27+
</testsuite>
28+
</testsuites>
29+
30+
<groups>
31+
<exclude>
32+
<group>CBF</group>
33+
</exclude>
34+
</groups>
35+
36+
<filter>
37+
<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
38+
<directory suffix=".php">./src</directory>
39+
<file>./autoload.php</file>
40+
<exclude>
41+
<directory suffix="UnitTest.php">./src/Standards</directory>
42+
</exclude>
43+
</whitelist>
44+
</filter>
45+
46+
<logging>
47+
<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
48+
<log type="coverage-clover" target="build/logs/clover.xml"/>
49+
</logging>
50+
51+
<php>
52+
<env name="PHP_CODESNIFFER_CBF" value="0"/>
53+
</php>
54+
</phpunit>

‎phpunit.xml.dist‎

Lines changed: 25 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,22 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<phpunit
33
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4-
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.2/phpunit.xsd"
4+
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.5/phpunit.xsd"
55
backupGlobals="true"
66
beStrictAboutOutputDuringTests="true"
77
beStrictAboutTestsThatDoNotTestAnything="false"
88
bootstrap="tests/bootstrap.php"
9-
convertErrorsToExceptions="true"
10-
convertWarningsToExceptions="true"
11-
convertNoticesToExceptions="true"
12-
convertDeprecationsToExceptions="true"
13-
forceCoversAnnotation="true"
9+
cacheDirectory="build/phpunit-cache"
10+
displayDetailsOnTestsThatTriggerErrors="true"
11+
displayDetailsOnTestsThatTriggerWarnings="true"
12+
displayDetailsOnTestsThatTriggerNotices="true"
13+
displayDetailsOnTestsThatTriggerDeprecations="true"
14+
displayDetailsOnPhpunitDeprecations="false"
15+
failOnWarning="true"
16+
failOnNotice="true"
17+
failOnDeprecation="true"
18+
failOnPhpunitDeprecation="false"
19+
requireCoverageMetadata="true"
1420
>
1521
<testsuites>
1622
<testsuite name="PHPCS_Core">
@@ -33,20 +39,22 @@
3339
</exclude>
3440
</groups>
3541

36-
<filter>
37-
<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
42+
<source>
43+
<include>
3844
<directory suffix=".php">./src</directory>
3945
<file>./autoload.php</file>
40-
<exclude>
41-
<directory suffix="UnitTest.php">./src/Standards</directory>
42-
</exclude>
43-
</whitelist>
44-
</filter>
46+
</include>
47+
<exclude>
48+
<directory suffix="UnitTest.php">./src/Standards</directory>
49+
</exclude>
50+
</source>
4551

46-
<logging>
47-
<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
48-
<log type="coverage-clover" target="build/logs/clover.xml"/>
49-
</logging>
52+
<coverage includeUncoveredFiles="true">
53+
<report>
54+
<clover outputFile="build/logs/clover.xml"/>
55+
<text outputFile="php://stdout" showOnlySummary="true"/>
56+
</report>
57+
</coverage>
5058

5159
<php>
5260
<env name="PHP_CODESNIFFER_CBF" value="0"/>

0 commit comments

Comments
 (0)