Tests: allow the test suite to run on PHPUnit 8.x and 9.x

Includes:
* `composer.json`: widening the PHPUnit requirement to allow for PHPUnit 8.x and PHPUnit 9.x.
    Note: The recently released PHPUnit 10.x is not (yet) supported as it no longer supports the old-style test suite runner setup.
    It also would require for the abstract base test cases to be renamed as those classes are no longer allowed to end on `Test`.
    Refactoring the test suite to allow for PHPUnit 10.x is for a future PR.
* `composer.json`: remove the script which was specific for PHP 8.1+
* Adjusting the PHPUnit configuration to ensure the tests are run in the same way and show all notices/warnings/deprecations on all PHPUnit versions.
    The default value for a number of configuration options has changed over time.
    This makes sure they are consistently set to values which are sensible for this codebase, independently of the PHPUnit version on which the tests are run.
    Includes adding a schema annotation (set to PHPUnit 9.2 as the schema has changed in PHPUnit 9.3, though that won't prevent the tests from running correctly).
* GH Actions `test`/`quicktest` workflow: removing work-arounds which were in place related to running PHPUnit 7.x on PHP 8.x.
* GH Actions `phpstan` workflow: let PHPStan run on the latest PHP version, now there's no need anymore to run it against a lower PHP version to prevent deprecation notices related to the use of an outdated PHPUnit version.
* `AllTests`: Adjusting the condition which determines which `TestSuite` file to load to allow for PHPUnit 8.x and 9.x.
* Adding the `.phpunit.result.cache` file to `.gitignore`.
    PHPUnit has a caching feature build in as of PHPUnit 8, so ignore the file that generates to prevent it from being committed.
* CONTRIBUTING: remove references to the remove Composer script + instructions which have now become redundant.

Related to 3395

Author: jrfnl

.github/CONTRIBUTING.md

@@ -137,7 +137,6 @@ Note: There may be an issue or PR open already. If so, please join the discussio
 
 1. Fork/clone the repository.
 2. Run `composer install`.
-    When installing on PHP >= 8.0, use `composer install --ignore-platform-req=php+`.
 3. Create a new branch off the `master` branch to hold your patch.
     If there is an open issue associated with your patch, including the issue number in the branch name is good practice.
 
@@ -152,10 +151,8 @@ To help you with this, a number of convenience scripts are available:
 * `composer check-all` will run the `cs` + `test` checks in one go.
 * `composer cs` will check for code style violations.
 * `composer cbf` will run the autofixers for code style violations.
-* `composer test` will run the unit tests (only works when on PHP < 8.1).
-* `composer test-php8` will run the unit tests when you are working on PHP 8.1+.
-    Please note that using a `phpunit.xml` overload config file will not work with this script!
-* `composer coverage` will run the unit tests with code coverage (only works when on PHP < 8.1).
+* `composer test` will run the unit tests.
+* `composer coverage` will run the unit tests with code coverage.
     Note: you may want to use a custom `phpunit.xml` overload config file to tell PHPUnit where to place an HTML report.
     Alternative run it like so: `composer coverage -- --coverage-html /path/to/report-dir/` to specify the location for the HTML report on the command line.
 * `composer build` will build the phpcs.phar and phpcbf.phar files.

.github/workflows/phpstan.yml

@@ -30,7 +30,7 @@ jobs:
       - name: Install PHP
         uses: shivammathur/setup-php@v2
         with:
-          php-version: '7.4'
+          php-version: 'latest'
           coverage: none
           tools: phpstan
 

.github/workflows/test.yml

@@ -131,37 +131,20 @@ jobs:
 
       # Install dependencies and handle caching in one go.
       # @link https://github.com/marketplace/actions/install-php-dependencies-with-composer
-      - name: Install Composer dependencies - normal
-        if: ${{ matrix.php < '8.0' }}
+      - name: Install Composer dependencies
         uses: "ramsey/composer-install@v2"
         with:
           # Bust the cache at least once a month - output format: YYYY-MM.
           custom-cache-suffix: $(date -u "+%Y-%m")
 
-      # For PHP 8.0+, we need to install with ignore platform reqs as PHPUnit 7 is still used.
-      - name: Install Composer dependencies - with ignore platform
-        if: ${{ matrix.php >= '8.0' }}
-        uses: "ramsey/composer-install@v2"
-        with:
-          composer-options: --ignore-platform-req=php+
-          custom-cache-suffix: $(date -u "+%Y-%m")
-
       # Note: The code style check is run multiple times against every PHP version
       # as it also acts as an integration test.
       - name: 'PHPCS: set the path to PHP'
         run: php bin/phpcs --config-set php_path php
 
       - name: 'PHPUnit: run the tests'
-        if: ${{ matrix.php != '8.1' && matrix.php != '8.2' }}
         run: vendor/bin/phpunit tests/AllTests.php
 
-      # We need to ignore the config file so that PHPUnit doesn't try to read it.
-      # The config file causes an error on PHP 8.1+ with PHPunit 7, but it's not needed here anyway
-      # as we can pass all required settings in the phpunit command.
-      - name: 'PHPUnit: run the tests on PHP > 8.0'
-        if: ${{ matrix.php == '8.1' || matrix.php == '8.2' }}
-        run: vendor/bin/phpunit tests/AllTests.php --no-configuration --bootstrap=tests/bootstrap.php --dont-report-useless-tests
-
       - name: 'PHPCS: check code style without cache, no parallel'
         if: ${{ matrix.custom_ini == false && matrix.php != '7.4' }}
         run: php bin/phpcs --no-cache --parallel=1

.gitignore

@@ -1,6 +1,7 @@
 /CodeSniffer.conf
 /phpcs.xml
 /phpunit.xml
+.phpunit.result.cache
 .idea/*
 /vendor/
 composer.lock

CHANGELOG.md

@@ -36,6 +36,9 @@ The file documents changes to the PHP_CodeSniffer project.
     - PSR2.Methods.FunctionCallSignature
     - PSR2.Methods.FunctionClosingBrace
     - Thanks to Atsushi Okui (@blue32a) for the patch
+- Support for PHPUnit 8 and 9 to the test suite.
+    - Test suites for external standards which run via the PHPCS native test suite can now run on PHPUnit 4-9 (was 4-7).
+    - If any of these tests use the PHPUnit `setUp()`/`tearDown()` methods or overload the `setUp()` in the `AbstractSniffUnitTest` test case, they will need to be adjusted. See the [PR details for further information](https://github.com/PHPCSStandards/PHP_CodeSniffer/pull/59/commits/26384ebfcc0b1c1651b0e1e40c9b6c8c22881832).
 
 ### Changed
 - Changes have been made to the way PHPCS handles invalid sniff properties being set in a custom ruleset

composer.json

@@ -41,7 +41,7 @@
         "ext-simplexml": "*"
     },
     "require-dev": {
-        "phpunit/phpunit": "^4.0 || ^5.0 || ^6.0 || ^7.0"
+        "phpunit/phpunit": "^4.0 || ^5.0 || ^6.0 || ^7.0 || ^8.0 || ^9.0"
     },
     "replace": {
         "squizlabs/php_codesniffer": "> 2.0"
@@ -61,10 +61,6 @@
             "Composer\\Config::disableProcessTimeout",
             "@php ./vendor/phpunit/phpunit/phpunit tests/AllTests.php --no-coverage"
         ],
-        "test-php8": [
-            "Composer\\Config::disableProcessTimeout",
-            "@php ./vendor/phpunit/phpunit/phpunit tests/AllTests.php --no-configuration --bootstrap=tests/bootstrap.php --dont-report-useless-tests --no-coverage"
-        ],
         "coverage": [
             "Composer\\Config::disableProcessTimeout",
             "@php ./vendor/phpunit/phpunit/phpunit tests/AllTests.php -d max_execution_time=0"
@@ -82,7 +78,6 @@
         "cs": "Check for code style violations.",
         "cbf": "Fix code style violations.",
         "test": "Run the unit tests without code coverage.",
-        "test-php8": "Run the unit tests without code coverage on PHP 8.1 or higher.",
         "coverage": "Run the unit tests with code coverage.",
         "build": "Create PHAR files for PHPCS and PHPCBF.",
         "check-all": "Run all checks (phpcs, tests)."

phpunit.xml.dist

@@ -1,5 +1,15 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<phpunit beStrictAboutTestsThatDoNotTestAnything="false" bootstrap="tests/bootstrap.php">
+<phpunit
+    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+    xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.2/phpunit.xsd"
+    backupGlobals="true"
+    beStrictAboutTestsThatDoNotTestAnything="false"
+    bootstrap="tests/bootstrap.php"
+    convertErrorsToExceptions="true"
+    convertWarningsToExceptions="true"
+    convertNoticesToExceptions="true"
+    convertDeprecationsToExceptions="true"
+    >
     <testsuites>
         <testsuite name="PHP_CodeSniffer Test Suite">
             <file>tests/AllTests.php</file>

tests/AllTests.php

@@ -18,7 +18,7 @@
 $phpunit7 = false;
 if (class_exists('\PHPUnit\Runner\Version') === true) {
     $version = \PHPUnit\Runner\Version::id();
-    if ($version[0] === '7') {
+    if (version_compare($version, '7.0', '>=') === true) {
         $phpunit7 = true;
     }
 }