PHP 8.2 | Ruleset: prevent notices about dynamic properties being set

The BC-breaks can be broken down as follows:

If a property is set via a ruleset for an **_individual_** sniff and that sniff does not have the property explicitly declared, nor does the sniff declare the PHP magic `__set()` method or `extend stdClass`, the ruleset will now be regarded as invalid.

In practice, this means the PHPCS run will halt and will show an informative notice to the end-user about the invalid property setting and the PHPCS run will exit with exit code `3`.

For properties being set for a complete category of sniffs or a complete standard, the PHPCS run will not halt, nor show an error when the property isn't supported for all sniffs in the category/standard.
In that case, the property will only be set on those sniffs which support it (i.e. either have the property explicitly declared on the sniff, or have the magic `__set()` method declared on the sniff, or `extend stdClass`) and will be silently ignored for all other sniffs.

:point_right: Aside from possibly surfacing some incidental typos in property settings in rulesets, I expect this change to go mostly unnoticed by end-users.

1. The format of the `$ruleset['sniff code']['properties']['property name']` sub-array has changed from a `array|string` type property value to a sub-array containing two entries:
    1. `'value'` - containing the `array|string` type property value.
    2. `'scope'` - containing a string, either `sniff` or `standard`, to indicate whether the ruleset property directive was for an individual sniff or for a whole standard/category of sniffs.
2. The third parameter for the `Ruleset::setSniffProperty()` method has been changed to expect the above mentioned array of `$settings` instead of just a property value.

This change is necessary to allow for throwing the error notice when an invalid property is being set on an invalid sniff versus silently ignoring the invalid property if the ruleset specified it on a standard/category.

:point_right: The impact of this BC-break is expected to be small, as it would require for an external standard or integration of PHPCS to:
* `extend` the `Ruleset` class **and** overload the `setSniffProperty()` method to have any impact;
* or to call the `setSniffProperty()` method on the `Ruleset` object directly (which _may_ happen in custom test frameworks for external standards).

Note:
* The change to the `processRule()` method cannot be overloaded as it is a `private` method.
* The change in the `populateTokenListeners()` method is only cosmetic and has no functional impact.

---

For consistent handling of properties set in a (custom) ruleset across PHP versions, I have chosen to only support setting properties when:
* A property is explicitly declared on a sniff class.
* A sniff class explicitly declares (or inherits) the magic `__set()` method.
* A sniff class `extends` `stdClass`.

Note: no additional check has been added to verify that a declared property is `public`. If that is not the case a PHP native error would be thrown previously and still will be now. This behaviour has not changed.

I have chosen **not** to respect the `#[\AllowDynamicProperties]` attribute as it is not possible (without tokenizing the sniff files which would create a lot of overhead) to determine whether that attribute exists on a class when PHPCS runs on PHP < 8.0 as the Reflection method needed to detect the attribute is only available on PHP 8.0+.

In other words, adding support for the attribute would introduce an inconsistency in how properties set in a ruleset are handled based on the PHP version on which PHPCS is being run.
It would also mean that the error notice for invalid properties set on individual sniffs would only be available on PHP 8.2+, which would greatly reduce the usefulness of the error notice.

In my opinion, consistency here is more important than supporting the attribute, especially as there are three fully supported ways to allow for supporting properties to be set from a ruleset.
The three above mentioned ways to allow for setting properties from a ruleset for a sniff are all fully supported on all PHP versions currently supported by PHPCS, so there is no compelling reason to also allow for the attribute.

> PHP 8.2: prevent deprecation notices for properties set in a (custom) ruleset for complete standards/complete sniff categories
> - Invalid properties set for individual sniffs will now result in an error and halt the execution of PHPCS.
> - Properties set for complete standards/complete sniff categories will now only be set on sniffs which explicitly support the property.
>       The property will be silently ignored for those sniffs which do not support the property.
> - For sniff developers: it is strongly recommended for sniffs to explicitly declare any user-adjustable `public` properties.
>    If dynamic properties need to be supported for a sniff, either declare the magic `__set()`/`__get()`/`__isset()`/`__unset()` methods on the sniff or let the sniff extend `stdClass`.
>    Note: The `#[\AllowDynamicProperties]` attribute will have **no effect** for properties which are being set in rulesets.
> - Sniff developers/integrators of PHPCS may need to make some small adjustments to allow for changes to the PHPCS internals. See the description of PR 3629 for full details.

The changes are accompanied by a full set of tests covering the change.
There is a little overlap between the `RuleInclusionTest` class and the new `SetSniffPropertyTest` class, but for the most part, the tests compliment each other.

Includes:
* Correct handling of properties being set via `phpcs:set` annotations.
* The `RuleInclusionTest` XML fixture and test expectations have been adjusted to allow for these changes.
    1. As we now need "settable" properties to exist on sniffs to use for the `testSettingProperties()` test, the `PSR1` ruleset is not sufficient as the sniffs in that ruleset don't have public properties.
        For that reason, the "set property for complete standard" test has been changed to use `PSR2` instead and set the `indent` property.
        The test expectations has been adjusted to match.
    2. Along the same lines, the `Zend.NamingConventions` sniffs don't have public properties, so the "setting property for complete category" test has been switched to `PSR12.Operators` and set the `ignoreSpacingBeforeAssignments` property.
        The test expectations has been adjusted to match.
    3. In both these cases, the XML `<rule>` now contains both a property which is valid for at least some sniffs in the standard/category and a property which is invalid in for all sniffs.
        For the _invalid_ properties, a separate `testSettingInvalidPropertiesOnStandardsAndCategoriesSilentlyFails()` test with data provider has been added to verify these will not be set.
    4. As the ruleset has changed, the expectations for the `testHasSniffCodes()` test and the `testRegisteredSniffCodes()` test have been updated to match.
    5. The test descriptions for the test cases in the `dataSettingProperties()` have also been updated to make it more explicit what each test case is testing.
* The PHPCS ruleset has been updated to exclude sniffs which function as test fixtures from the PHPCS check for this repo.
* A minor tweak has been made to the `ValidatePEARPackageXML` script to prevent a warning about sniffs which function as test fixtures.
    These test fixtures should receive the `role="test"` attribute, not a `role="php"` attribute.

Fixes 3489

Author: jrfnl

phpcs.xml.dist

@@ -11,6 +11,7 @@
 
     <exclude-pattern>*/src/Standards/*/Tests/*\.(inc|css|js)$</exclude-pattern>
     <exclude-pattern>*/tests/Core/*/*\.(inc|css|js)$</exclude-pattern>
+    <exclude-pattern>*/tests/Core/*/Fixtures/*\.php$</exclude-pattern>
 
     <arg name="basepath" value="."/>
     <arg name="colors"/>

src/Files/File.php

@@ -361,9 +361,12 @@ public function process()
                         $listenerCode = $token['sniffCode'];
                         if (isset($this->ruleset->sniffCodes[$listenerCode]) === true) {
                             $propertyCode  = $token['sniffProperty'];
-                            $propertyValue = $token['sniffPropertyValue'];
+                            $settings      = [
+                                'value' => $token['sniffPropertyValue'],
+                                'scope' => 'sniff',
+                            ];
                             $listenerClass = $this->ruleset->sniffCodes[$listenerCode];
-                            $this->ruleset->setSniffProperty($listenerClass, $propertyCode, $propertyValue);
+                            $this->ruleset->setSniffProperty($listenerClass, $propertyCode, $settings);
                         }
                     }
                 }//end if

src/Ruleset.php

@@ -14,6 +14,7 @@
 use PHP_CodeSniffer\Exceptions\RuntimeException;
 use PHP_CodeSniffer\Util\Common;
 use PHP_CodeSniffer\Util\Standards;
+use stdClass;
 
 class Ruleset
 {
@@ -932,6 +933,11 @@ private function processRule($rule, $newSniffs, $depth=0)
             if (isset($rule->properties) === true
                 && $this->shouldProcessElement($rule->properties) === true
             ) {
+                $propertyScope = 'standard';
+                if ($code === $ref || substr($ref, -9) === 'Sniff.php') {
+                    $propertyScope = 'sniff';
+                }
+
                 foreach ($rule->properties->property as $prop) {
                     if ($this->shouldProcessElement($prop) === false) {
                         continue;
@@ -952,9 +958,9 @@ private function processRule($rule, $newSniffs, $depth=0)
                         $values = [];
                         if (isset($prop['extend']) === true
                             && (string) $prop['extend'] === 'true'
-                            && isset($this->ruleset[$code]['properties'][$name]) === true
+                            && isset($this->ruleset[$code]['properties'][$name]['value']) === true
                         ) {
-                            $values = $this->ruleset[$code]['properties'][$name];
+                            $values = $this->ruleset[$code]['properties'][$name]['value'];
                         }
 
                         if (isset($prop->element) === true) {
@@ -978,7 +984,10 @@ private function processRule($rule, $newSniffs, $depth=0)
                             $printValue = rtrim($printValue, ',');
                         }
 
-                        $this->ruleset[$code]['properties'][$name] = $values;
+                        $this->ruleset[$code]['properties'][$name] = [
+                            'value' => $values,
+                            'scope' => $propertyScope,
+                        ];
                         if (PHP_CODESNIFFER_VERBOSITY > 1) {
                             $statusMessage = "=> array property \"$name\" set to \"$printValue\"";
                             if ($code !== $ref) {
@@ -988,7 +997,10 @@ private function processRule($rule, $newSniffs, $depth=0)
                             Common::printStatusMessage($statusMessage, ($depth + 2));
                         }
                     } else {
-                        $this->ruleset[$code]['properties'][$name] = (string) $prop['value'];
+                        $this->ruleset[$code]['properties'][$name] = [
+                            'value' => (string) $prop['value'],
+                            'scope' => $propertyScope,
+                        ];
                         if (PHP_CODESNIFFER_VERBOSITY > 1) {
                             $statusMessage = "=> property \"$name\" set to \"".(string) $prop['value'].'"';
                             if ($code !== $ref) {
@@ -1177,8 +1189,8 @@ public function populateTokenListeners()
 
             // Set custom properties.
             if (isset($this->ruleset[$sniffCode]['properties']) === true) {
-                foreach ($this->ruleset[$sniffCode]['properties'] as $name => $value) {
-                    $this->setSniffProperty($sniffClass, $name, $value);
+                foreach ($this->ruleset[$sniffCode]['properties'] as $name => $settings) {
+                    $this->setSniffProperty($sniffClass, $name, $settings);
                 }
             }
 
@@ -1234,18 +1246,48 @@ public function populateTokenListeners()
      *
      * @param string $sniffClass The class name of the sniff.
      * @param string $name       The name of the property to change.
-     * @param string $value      The new value of the property.
+     * @param array  $settings   Array with the new value of the property and the scope of the property being set.
      *
      * @return void
+     *
+     * @throws \PHP_CodeSniffer\Exceptions\RuntimeException When attempting to set a non-existent property on a sniff
+     *                                                      which doesn't declare the property or explicitly supports
+     *                                                      dynamic properties.
      */
-    public function setSniffProperty($sniffClass, $name, $value)
+    public function setSniffProperty($sniffClass, $name, $settings)
     {
         // Setting a property for a sniff we are not using.
         if (isset($this->sniffs[$sniffClass]) === false) {
             return;
         }
 
-        $name = trim($name);
+        $name         = trim($name);
+        $propertyName = $name;
+        if (substr($propertyName, -2) === '[]') {
+            $propertyName = substr($propertyName, 0, -2);
+        }
+
+        $isSettable  = false;
+        $sniffObject = $this->sniffs[$sniffClass];
+        if (property_exists($sniffObject, $propertyName) === true
+            || ($sniffObject instanceof stdClass) === true
+            || method_exists($sniffObject, '__set') === true
+        ) {
+            $isSettable = true;
+        }
+
+        if ($isSettable === false) {
+            if ($settings['scope'] === 'sniff') {
+                $notice  = "Ruleset invalid. Property \"$propertyName\" does not exist on sniff ";
+                $notice .= array_search($sniffClass, $this->sniffCodes, true);
+                throw new RuntimeException($notice);
+            }
+
+            return;
+        }
+
+        $value = $settings['value'];
+
         if (is_string($value) === true) {
             $value = trim($value);
         }
@@ -1260,7 +1302,7 @@ public function setSniffProperty($sniffClass, $name, $value)
         } else if ($value === 'false') {
             $value = false;
         } else if (substr($name, -2) === '[]') {
-            $name   = substr($name, 0, -2);
+            $name   = $propertyName;
             $values = [];
             if ($value !== null) {
                 foreach (explode(',', $value) as $val) {
@@ -1276,7 +1318,7 @@ public function setSniffProperty($sniffClass, $name, $value)
             $value = $values;
         }
 
-        $this->sniffs[$sniffClass]->$name = $value;
+        $sniffObject->$name = $value;
 
     }//end setSniffProperty()
 

tests/Core/Ruleset/Fixtures/Sniffs/Category/SetPropertyAllowedAsDeclaredSniff.php

@@ -0,0 +1,28 @@
+<?php
+/**
+ * Test fixture.
+ *
+ * @see \PHP_CodeSniffer\Tests\Core\Ruleset\SetSniffPropertyTest
+ */
+
+namespace Fixtures\Sniffs\Category;
+
+use PHP_CodeSniffer\Files\File;
+use PHP_CodeSniffer\Sniffs\Sniff;
+
+class SetPropertyAllowedAsDeclaredSniff implements Sniff
+{
+
+    public $arbitrarystring;
+    public $arbitraryarray;
+
+    public function register()
+    {
+        return [T_WHITESPACE];
+    }
+
+    public function process(File $phpcsFile, $stackPtr)
+    {
+        // Do something.
+    }
+}

tests/Core/Ruleset/Fixtures/Sniffs/Category/SetPropertyAllowedViaMagicMethodSniff.php

@@ -0,0 +1,40 @@
+<?php
+/**
+ * Test fixture.
+ *
+ * @see \PHP_CodeSniffer\Tests\Core\Ruleset\SetSniffPropertyTest
+ */
+
+namespace Fixtures\Sniffs\Category;
+
+use PHP_CodeSniffer\Files\File;
+use PHP_CodeSniffer\Sniffs\Sniff;
+
+class SetPropertyAllowedViaMagicMethodSniff implements Sniff
+{
+    private $magic = [];
+
+    public function __set($name, $value)
+    {
+        $this->magic[$name] = $value;
+    }
+
+    public function __get($name)
+    {
+        if (isset($this->magic[$name])) {
+            return $this->magic[$name];
+        }
+
+        return null;
+    }
+
+    public function register()
+    {
+        return [T_WHITESPACE];
+    }
+
+    public function process(File $phpcsFile, $stackPtr)
+    {
+        // Do something.
+    }
+}

tests/Core/Ruleset/Fixtures/Sniffs/Category/SetPropertyAllowedViaStdClassSniff.php

@@ -0,0 +1,26 @@
+<?php
+/**
+ * Test fixture.
+ *
+ * @see \PHP_CodeSniffer\Tests\Core\Ruleset\SetSniffPropertyTest
+ */
+
+namespace Fixtures\Sniffs\Category;
+
+use PHP_CodeSniffer\Files\File;
+use PHP_CodeSniffer\Sniffs\Sniff;
+use stdClass;
+
+class SetPropertyAllowedViaStdClassSniff extends stdClass implements Sniff
+{
+
+    public function register()
+    {
+        return [T_WHITESPACE];
+    }
+
+    public function process(File $phpcsFile, $stackPtr)
+    {
+        // Do something.
+    }
+}

tests/Core/Ruleset/Fixtures/Sniffs/Category/SetPropertyNotAllowedViaAttributeSniff.php

@@ -0,0 +1,27 @@
+<?php
+/**
+ * Test fixture.
+ *
+ * @see \PHP_CodeSniffer\Tests\Core\Ruleset\SetSniffPropertyTest
+ */
+
+namespace Fixtures\Sniffs\Category;
+
+use AllowDynamicProperties;
+use PHP_CodeSniffer\Files\File;
+use PHP_CodeSniffer\Sniffs\Sniff;
+
+#[AllowDynamicProperties]
+class SetPropertyNotAllowedViaAttributeSniff implements Sniff
+{
+
+    public function register()
+    {
+        return [T_WHITESPACE];
+    }
+
+    public function process(File $phpcsFile, $stackPtr)
+    {
+        // Do something.
+    }
+}