Skip to content

Commit 268a4b5

Browse files
jrfnljrfnl
committed
Tests: allow for PHPUnit 10 and 11
This commit makes the necessary changes in the CI wiring to allow for running the tests on PHPUnit 8, 9, 10 and 11 and still failing them on PHP deprecations and notices like before. The PHPUnit configuration file specification has undergone changes in PHPUnit 9.3, 10.0, 10.1 and 10.5.32/11.3.3. Most notably: * In PHPUnit 9.3, the manner of specifying the code coverage configuration has changed, though the "old" way is still supported on PHPUnit 9, but not on PHPUnit 10. * In PHPUnit 10.0, a significant number of attributes of the `<phpunit>` element were removed or renamed and failing test runs on PHP deprecations or notices was no longer possible. * In PHPUnit 10.1 a further iteration was released on the way to specify code coverage, which now expects a `<source>` element. Additionally, it is now possible again to fail a test run on PHP deprecations/notices. * In PHPUnit 10.5.32/11.3.3 new `displayDetailsOnPhpunitDeprecations` and `failOnPhpunitDeprecation` options were introduced. As of PHPUnit 10.0, the option to fail tests on (PHP) deprecation notices would **_also_** fail the tests on PHPUnit deprecation notices, while those are only relevant when upgrading to the _next_ PHPUnit version. This made things a bit awkward. This was, at long last, fixed via the new options in PHPUnit 10.5.32/11.3.3. While the `--migrate-configuration` command can upgrade a configuration for the changes in the format made in PHPUnit 9.3, some of the changes in the configuration format in PHPUnit 10 don't have one-on-one replacements and/or are not taken into account. With that in mind, I deem it more appropriate to have a dedicated PHPUnit configuration file for PHPUnit 10/11 to ensure the test runs will behave as intended. To be forward compatible, the default PHPUnit config file `phpunit.xml.dist` will now use the PHPUnit 10+ format, while a separate `phpunitlt10.xml.dist` file will be used for PHPUnit 8 and 9. > ☝️ This means that contributors who run on recent PHP versions, should just be able to run `vendor/bin/phpunit`. Only contributors who run on PHP < 8.1 and therefore run on PHPUnit < 10, will need to be aware of the difference in the config files. All in all, this commit makes the following changes: * Rename the "old" PHPUnit configuration file to `phpunitlt10.xml.dist` - This file has also been added to `.gitattributes` to ignore for distribution packages. - An entry for `phpunitlt10.xml` has been added to the `.gitignore` file. While a contributor can choose the name for a config overload file for the `phpunitlt10.xml.dist` file freely, it is a good convention to use the same name as the file being overloaded, minus the `.dist`, so ignoring that filename by default seems appropriate. - Composer scripts have been added to run the tests with the `phpunitlt10.xml.dist` file and these new scripts are also mentioned in the `CONTRIBUTING` file. * The `phpunit.xml.dist` now contains the PHPUnit configuration for running the tests on PHPUnit 10.5+. This includes the appropriate settings to fail the test runs on any PHP deprecations, notices or warnings. Take note that the `cacheDirectory` option has been set to a custom directory which is already git ignored. * The GH Actions scripts have been updated to include a PHPUnit version based toggle and run the tests with the correct configuration file depending on the PHPUnit version being used. * Composer: allow for PHPUnit 10 on version 10.5.32 or higher and PHPUnit 11 on v 11.3.3 or higher. The 10.5.32/11.3.3 version are deliberately set as the minimum as: - the `failOnNotice` and `failOnDeprecation` attributes can now be set in the XML config file (PHPUnit 10.1+). - the `source` element for the XML config file was introduced in 10.1. - the `displayDetailsOnPhpunitDeprecations` and `failOnPhpunitDeprecation` attributes can be set (PHPUnit 10.5.32/11.3.3+). Refs: * https://phpunit.de/announcements/phpunit-10.html * sebastianbergmann/phpunit 5196 * sebastianbergmann/phpunit@fb6673f * sebastianbergmann/phpunit 5937 * sebastianbergmann/phpunit@1a5d2b8
1 parent 78e5d09 commit 268a4b5

File tree

9 files changed

+180
-34
lines changed

9 files changed

+180
-34
lines changed

‎.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ tests/Core/**/ export-ignore
1919
phpcs.xml.dist export-ignore
2020
phpstan.neon.dist export-ignore
2121
phpunit.xml.dist export-ignore
22+
phpunitlt10.xml.dist export-ignore
2223
tests/Core/ErrorSuppressionTest.php export-ignore
2324

2425
#

‎.github/CONTRIBUTING.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -259,10 +259,14 @@ To help you with this, a number of convenience scripts are available:
259259
* `composer check-all` will run the `cs` + `test` checks in one go.
260260
* `composer cs` will check for code style violations.
261261
* `composer cbf` will run the autofixers for code style violations.
262-
* `composer test` will run the unit tests.
263-
* `composer coverage` will run the unit tests with code coverage and show a text summary.
262+
* `composer test` will run the unit tests when using PHP 8.1+/PHPUnit 10+.
263+
* `composer testlt10` will run the unit tests when using PHP < 8.1/PHPUnit < 10.
264+
* `composer coverage` will run the unit tests with code coverage and show a text summary (PHP 8.1+/PHPUnit 10+).
265+
* `composer coveragelt10` will run the unit tests with code coverage and show a text summary (PHP < 8.1/PHPUnit < 10).
264266
* `composer coverage-local` will run the unit tests with code coverage and generate an HTML coverage report,
265-
which will be placed in a `build/coverage-html` subdirectory.
267+
which will be placed in a `build/coverage-html` subdirectory (PHP 8.1+/PHPUnit 10+).
268+
* `composer coveragelt10-local` will run the unit tests with code coverage and generate an HTML coverage report,
269+
which will be placed in a `build/coverage-html` subdirectory (PHP < 8.1/PHPUnit < 10).
266270
* `composer build` will build the phpcs.phar and phpcbf.phar files.
267271

268272

‎.github/workflows/quicktest.yml‎

Lines changed: 26 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,19 +55,42 @@ jobs:
5555
# Bust the cache at least once a month - output format: YYYY-MM.
5656
custom-cache-suffix: $(date -u "+%Y-%m")
5757

58+
- name: Grab PHPUnit version
59+
id: phpunit_version
60+
# yamllint disable-line rule:line-length
61+
run: echo "VERSION=$(vendor/bin/phpunit --version | grep --only-matching --max-count=1 --extended-regexp '\b[0-9]+\.[0-9]+')" >> $GITHUB_OUTPUT
62+
63+
- name: "DEBUG: Show grabbed version"
64+
run: echo ${{ steps.phpunit_version.outputs.VERSION }}
65+
66+
- name: Determine PHPUnit config file to use
67+
id: phpunit_config
68+
run: |
69+
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
70+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
71+
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) }}" == "true" ]; then
72+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
73+
else
74+
echo 'FILE=phpunitlt10.xml.dist' >> "$GITHUB_OUTPUT"
75+
fi
76+
5877
- name: 'PHPCS: set the path to PHP'
5978
run: php "bin/phpcs" --config-set php_path php
6079

6180
- name: 'PHPUnit: run the full test suite'
6281
if: ${{ matrix.os != 'windows-latest' }}
63-
run: php "vendor/bin/phpunit" --no-coverage
82+
run: php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
6483

6584
- name: 'PHPUnit: run tests which may have different outcomes on Windows'
6685
if: ${{ matrix.os == 'windows-latest' }}
67-
run: php "vendor/bin/phpunit" --group Windows --no-coverage
86+
run: >
87+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
88+
--group Windows
6889
6990
- name: 'PHPUnit: run select tests in CBF mode'
70-
run: php "vendor/bin/phpunit" --group CBF --exclude-group nothing --no-coverage
91+
run: >
92+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
93+
--group CBF --exclude-group nothing
7194
env:
7295
PHP_CODESNIFFER_CBF: '1'
7396

‎.github/workflows/test.yml‎

Lines changed: 41 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -197,18 +197,39 @@ jobs:
197197
composer-options: ${{ matrix.php == '8.5' && '--ignore-platform-req=php+' || '' }}
198198
custom-cache-suffix: $(date -u "+%Y-%m")
199199

200+
- name: Grab PHPUnit version
201+
id: phpunit_version
202+
# yamllint disable-line rule:line-length
203+
run: echo "VERSION=$(vendor/bin/phpunit --version | grep --only-matching --max-count=1 --extended-regexp '\b[0-9]+\.[0-9]+')" >> $GITHUB_OUTPUT
204+
205+
- name: "DEBUG: Show grabbed version"
206+
run: echo ${{ steps.phpunit_version.outputs.VERSION }}
207+
208+
- name: Determine PHPUnit config file to use
209+
id: phpunit_config
210+
run: |
211+
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
212+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
213+
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) }}" == "true" ]; then
214+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
215+
else
216+
echo 'FILE=phpunitlt10.xml.dist' >> "$GITHUB_OUTPUT"
217+
fi
218+
200219
# Note: The code style check is run multiple times against every PHP version
201220
# as it also acts as an integration test.
202221
- name: 'PHPCS: set the path to PHP'
203222
run: php "bin/phpcs" --config-set php_path php
204223

205224
- name: 'PHPUnit: run the full test suite without code coverage'
206225
if: ${{ matrix.skip_tests != true }}
207-
run: php "vendor/bin/phpunit" --no-coverage
226+
run: php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
208227

209228
- name: 'PHPUnit: run select tests in CBF mode'
210229
if: ${{ matrix.skip_tests != true }}
211-
run: php "vendor/bin/phpunit" --group CBF --exclude-group nothing --no-coverage
230+
run: >
231+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }} --no-coverage
232+
--group CBF --exclude-group nothing
212233
env:
213234
PHP_CODESNIFFER_CBF: '1'
214235

@@ -293,6 +314,17 @@ jobs:
293314
- name: "DEBUG: Show grabbed version"
294315
run: echo ${{ steps.phpunit_version.outputs.VERSION }}
295316

317+
- name: Determine PHPUnit config file to use
318+
id: phpunit_config
319+
run: |
320+
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
321+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
322+
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) }}" == "true" ]; then
323+
echo 'FILE=phpunit.xml.dist' >> "$GITHUB_OUTPUT"
324+
else
325+
echo 'FILE=phpunitlt10.xml.dist' >> "$GITHUB_OUTPUT"
326+
fi
327+
296328
- name: 'PHPCS: set the path to PHP'
297329
run: php "bin/phpcs" --config-set php_path php
298330

@@ -302,18 +334,20 @@ jobs:
302334
# Using that option prevents issues with PHP-Parser backfilling PHP tokens during our test runs.
303335
- name: "Warm the PHPUnit cache (PHPUnit 9.3+)"
304336
if: ${{ steps.phpunit_version.outputs.VERSION >= '9.3' }}
305-
run: php "vendor/bin/phpunit" --coverage-cache ./build/phpunit-cache --warm-coverage-cache
337+
run: >
338+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
339+
--coverage-cache ./build/phpunit-cache --warm-coverage-cache
306340
307341
- name: "Run the unit tests with code coverage"
308342
if: ${{ matrix.os != 'windows-latest' }}
309343
run: >
310-
php "vendor/bin/phpunit"
344+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
311345
${{ steps.phpunit_version.outputs.VERSION >= '9.3' && '--coverage-cache ./build/phpunit-cache' || '' }}
312346
313347
- name: "Run select tests in CBF mode with code coverage"
314348
if: ${{ matrix.os != 'windows-latest' }}
315349
run: >
316-
php "vendor/bin/phpunit"
350+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
317351
${{ steps.phpunit_version.outputs.VERSION >= '9.3' && '--coverage-cache ./build/phpunit-cache' || '' }}
318352
--group CBF --exclude-group nothing --coverage-clover build/logs/clover-cbf.xml
319353
env:
@@ -322,8 +356,9 @@ jobs:
322356
- name: "Run the unit tests which may have different outcomes on Windows with code coverage"
323357
if: ${{ matrix.os == 'windows-latest' }}
324358
run: >
325-
php "vendor/bin/phpunit" --group Windows
359+
php "vendor/bin/phpunit" -c ${{ steps.phpunit_config.outputs.FILE }}
326360
${{ steps.phpunit_version.outputs.VERSION >= '9.3' && '--coverage-cache ./build/phpunit-cache' || '' }}
361+
--group Windows
327362
328363
- name: "Upload coverage results to Coveralls (normal run)"
329364
if: ${{ success() }}

‎.github/workflows/validate.yml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,12 @@ jobs:
5656
pattern: "phpcs.xml.dist"
5757
xsd-file: "phpcs.xsd"
5858

59-
- name: "Validate PHPUnit config for well-formedness"
59+
- name: "Validate PHPUnit < 10 config for well-formedness"
60+
uses: phpcsstandards/xmllint-validate@v1
61+
with:
62+
pattern: "phpunitlt10.xml.dist"
63+
64+
- name: "Validate PHPUnit 10+ config for well-formedness"
6065
uses: phpcsstandards/xmllint-validate@v1
6166
with:
6267
pattern: "phpunit.xml.dist"

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
/CodeSniffer.conf
22
/phpcs.xml
33
/phpunit.xml
4+
/phpunitlt10.xml
45
.phpunit.result.cache
56
/build/
67
.idea/*

‎composer.json‎

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
"ext-xmlwriter": "*"
3737
},
3838
"require-dev": {
39-
"phpunit/phpunit": "^8.0 || ^9.3.4"
39+
"phpunit/phpunit": "^8.0 || ^9.3.4 || ^10.5.32 || ^11.3.3"
4040
},
4141
"bin": [
4242
"bin/phpcbf",
@@ -61,14 +61,26 @@
6161
"Composer\\Config::disableProcessTimeout",
6262
"@php ./vendor/phpunit/phpunit/phpunit --no-coverage"
6363
],
64+
"testlt10": [
65+
"Composer\\Config::disableProcessTimeout",
66+
"@php ./vendor/phpunit/phpunit/phpunit -c phpunitlt10.xml.dist --no-coverage"
67+
],
6468
"coverage": [
6569
"Composer\\Config::disableProcessTimeout",
6670
"@php ./vendor/phpunit/phpunit/phpunit -d max_execution_time=0"
6771
],
72+
"coveragelt10": [
73+
"Composer\\Config::disableProcessTimeout",
74+
"@php ./vendor/phpunit/phpunit/phpunit -c phpunitlt10.xml.dist -d max_execution_time=0"
75+
],
6876
"coverage-local": [
6977
"Composer\\Config::disableProcessTimeout",
7078
"@php ./vendor/phpunit/phpunit/phpunit --coverage-html ./build/coverage-html -d max_execution_time=0"
7179
],
80+
"coveragelt10-local": [
81+
"Composer\\Config::disableProcessTimeout",
82+
"@php ./vendor/phpunit/phpunit/phpunit -c phpunitlt10.xml.dist --coverage-html ./build/coverage-html -d max_execution_time=0"
83+
],
7284
"build": [
7385
"Composer\\Config::disableProcessTimeout",
7486
"@php -d phar.readonly=0 -f ./scripts/build-phar.php"
@@ -81,9 +93,12 @@
8193
"scripts-descriptions": {
8294
"cs": "Check for code style violations.",
8395
"cbf": "Fix code style violations.",
84-
"test": "Run the unit tests without code coverage.",
85-
"coverage": "Run the unit tests with code coverage.",
86-
"coverage-local": "Run the unit tests with code coverage and generate an HTML report in a 'build' directory.",
96+
"test": "PHPUnit 10+: Run the unit tests without code coverage.",
97+
"testlt10": "PHPUnit < 10: Run the unit tests without code coverage.",
98+
"coverage": "PHPUnit 10+: Run the unit tests with code coverage.",
99+
"coveragelt10": "PHPUnit < 10: Run the unit tests with code coverage.",
100+
"coverage-local": "PHPUnit 10+: Run the unit tests with code coverage and generate an HTML report in a 'build' directory.",
101+
"coveragelt10-local": "PHPUnit < 10: Run the unit tests with code coverage and generate an HTML report in a 'build' directory.",
87102
"build": "Create PHAR files for PHPCS and PHPCBF.",
88103
"check-all": "Run all checks (phpcs, tests)."
89104
}

‎phpunit.xml.dist‎

Lines changed: 25 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,22 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<phpunit
33
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4-
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.2/phpunit.xsd"
4+
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.5/phpunit.xsd"
55
backupGlobals="true"
66
beStrictAboutOutputDuringTests="true"
77
beStrictAboutTestsThatDoNotTestAnything="false"
88
bootstrap="tests/bootstrap.php"
9-
convertErrorsToExceptions="true"
10-
convertWarningsToExceptions="true"
11-
convertNoticesToExceptions="true"
12-
convertDeprecationsToExceptions="true"
13-
forceCoversAnnotation="true"
9+
cacheDirectory="build/phpunit-cache"
10+
displayDetailsOnTestsThatTriggerErrors="true"
11+
displayDetailsOnTestsThatTriggerWarnings="true"
12+
displayDetailsOnTestsThatTriggerNotices="true"
13+
displayDetailsOnTestsThatTriggerDeprecations="true"
14+
displayDetailsOnPhpunitDeprecations="false"
15+
failOnWarning="true"
16+
failOnNotice="true"
17+
failOnDeprecation="true"
18+
failOnPhpunitDeprecation="false"
19+
requireCoverageMetadata="true"
1420
>
1521
<testsuites>
1622
<testsuite name="PHPCS_Core">
@@ -33,20 +39,22 @@
3339
</exclude>
3440
</groups>
3541

36-
<filter>
37-
<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
42+
<source>
43+
<include>
3844
<directory suffix=".php">./src</directory>
3945
<file>./autoload.php</file>
40-
<exclude>
41-
<directory suffix="UnitTest.php">./src/Standards</directory>
42-
</exclude>
43-
</whitelist>
44-
</filter>
46+
</include>
47+
<exclude>
48+
<directory suffix="UnitTest.php">./src/Standards</directory>
49+
</exclude>
50+
</source>
4551

46-
<logging>
47-
<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
48-
<log type="coverage-clover" target="build/logs/clover.xml"/>
49-
</logging>
52+
<coverage includeUncoveredFiles="true">
53+
<report>
54+
<clover outputFile="build/logs/clover.xml"/>
55+
<text outputFile="php://stdout" showOnlySummary="true"/>
56+
</report>
57+
</coverage>
5058

5159
<php>
5260
<env name="PHP_CODESNIFFER_CBF" value="0"/>

‎phpunitlt10.xml.dist‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<phpunit
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.2/phpunit.xsd"
5+
backupGlobals="true"
6+
beStrictAboutOutputDuringTests="true"
7+
beStrictAboutTestsThatDoNotTestAnything="false"
8+
bootstrap="tests/bootstrap.php"
9+
convertErrorsToExceptions="true"
10+
convertWarningsToExceptions="true"
11+
convertNoticesToExceptions="true"
12+
convertDeprecationsToExceptions="true"
13+
forceCoversAnnotation="true"
14+
>
15+
<testsuites>
16+
<testsuite name="PHPCS_Core">
17+
<directory>./tests/Core/</directory>
18+
</testsuite>
19+
<testsuite name="PHPCS_Sniffs">
20+
<directory>./src/Standards/Generic/Tests/</directory>
21+
<directory>./src/Standards/PEAR/Tests/</directory>
22+
<directory>./src/Standards/PSR1/Tests/</directory>
23+
<directory>./src/Standards/PSR2/Tests/</directory>
24+
<directory>./src/Standards/PSR12/Tests/</directory>
25+
<directory>./src/Standards/Squiz/Tests/</directory>
26+
<directory>./src/Standards/Zend/Tests/</directory>
27+
</testsuite>
28+
</testsuites>
29+
30+
<groups>
31+
<exclude>
32+
<group>CBF</group>
33+
</exclude>
34+
</groups>
35+
36+
<filter>
37+
<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
38+
<directory suffix=".php">./src</directory>
39+
<file>./autoload.php</file>
40+
<exclude>
41+
<directory suffix="UnitTest.php">./src/Standards</directory>
42+
</exclude>
43+
</whitelist>
44+
</filter>
45+
46+
<logging>
47+
<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
48+
<log type="coverage-clover" target="build/logs/clover.xml"/>
49+
</logging>
50+
51+
<php>
52+
<env name="PHP_CODESNIFFER_CBF" value="0"/>
53+
</php>
54+
</phpunit>

0 commit comments

Comments
 (0)