Upgrade to PHPParser 5 and PHPStan 2 (#6431)

* Upgrade to PHPParser 5 and PHPStan 2

* patch finally stmt

* patch function stmt

* patch do stmt

* patch catch stmt

* patch trycatch stmt

* patch for stmt

* patch classmethod stmt

* patch else stmt

* patch while stmt

* patch foreach stmt

* patch case stmt

* patch case stmt

* patch elseif stmt

* patch namespace stmt

* patch node traverser stmt

* regenerate preload

* fix build preload generator

* trigger patch

* fix override param method pExpr_ArrowFunction

* fix override param method pExpr_Ternary

* fix override param method pExpr_Yield

* fix override param method p

* update parameters of RichParser

* Fix IsSuperTypeOfResult usage on ShortenedObjectType

* disable temporary phpstan extension config

* use ParserFactory::createForVersion()

* upgrade to phpstan/phpdoc-parser 2.0

* register ParserConfig as service

* set lines and indexes config

* remove ltrim on FileProcessor, PhpParser printformat preserving no longer ltrim

* remove ltrim on FileProcessor, PhpParser printformat preserving no longer ltrim

* Fix PhpDocTypeMapperTest

* make PrettyPrinter.php higher priority than PrettyPrinterAbstract.php

* add ConstantArrayTypeMapper

* Fix native return type on non native call detection

* use parent default indent parameter on pFileWithoutNamespace

* override pExpr_ArrowFunction method

* remove override pModifiers

* fix CallableTypeNode parameter construct

* fix ParamTagValueNode construct parameter

* fix use Token as array on CommonNotEqualRector

* ConstantArrayType is sub of root PHPStan type

* fix cannot set null of node on PhpDocNodeTraverser

* fix use Token as array on VariableInStringInterpolationFixerRector

* fix use Token as array on CompleteMissingIfElseBracketRector

* fix IsSuperTypeOfResult usage on ShortenedGenericObjectType

* fix pArray() override parameters

* Fix InstalledPackageResolverTest

* temporary remove --error-format symplify as phpstan symplify rules temporary disabled

* fix SmartPhpParserFactory setup

* fix compatible phpstan 2.0.1 on build

* Fix Throw_ from Expr on ThrowWithPreviousExceptionRector

* fix ConstantArrayType usage on ArraySpreadInsteadOfArrayMergeRector

* fix use Throw_ on Expr on TerminatedNodeAnalyzer

* clean up prop

* fix pass ArrayItem on Foreach_

* fix ArrayItem instance

* Fix ClosureUse from Node on CallUserFuncWithArrowFunctionToInlineRector

* Fix union normalize of float, string, integer

* Fix union normalize of float, string, integer

* remove return by ref param, no longer supported on phpstan 2

* Fix use of DeclareDeclare to DeclareItem on DeclareStrictTypesRector and IncreaseDeclareStrictTypesRector

* fix replace PropertyProperty to PropertyItem

* fix deprecated traverse constant

* fix removed FunctionVariantWithPhpDocs over ExtendedFunctionVariant

* fix use ExtendedNativeParameterReflection over NativeParameterWithPhpDocsReflection

* replace PropertyProperty with PropertyItem

* fix namespacedName init on ClassDependencyManipulatorTest

* Fix Throw_ on expr on RemoveAlwaysElseRector

* fix PropertyItem over PropertyProperty on MissingPropertiesFactory

* fix replace traverser on docs

* Fix Throw_ stmt to Expr Throw on ChangeSwitchToMatchRector

* fix Throw_ from expr on SwitchExprsResolver

* fix next throw to use Expression with Throw_ as expr on MatchSwitchAnalyzer

* fix sort MatchArm conds

* fix magic __CLASS__ get value

* fix use token as array on ParenthesizedNestedTernaryAnalyzer

* [ci-review] Rector Rectify

* replace deprecated class node to new node

* [ci-review] Rector Rectify

* fix test on FixtureJsonThrowCaseSensitiveConstFetch

* Fix Throw_ from Expr on ConsecutiveNullCompareReturnsToNullCoalesceQueueRector

* Fix Throw_ on SilentVoidResolver

* fix node visitor stop

* [ci-review] Rector Rectify

* fix Throw_ on Expr

* fix undefined ContantType

* fix undefined ContantType

* fix addVisitor() after call parent::construct()

* fix check has Throw on AddNeverReturnType

* [ci-review] Rector Rectify

* fix check has Throw on AddNeverReturnType

* fix test on JsonThrowCaseSensitiveConstFetchTest

* fix arrow function text spacing

* Fix ArrayItem resolve on normalize item value

* prepare for doctrine value migration

* setup doctrine

* [ci-review] Rector Rectify

* use PhpDocTagNode

* clean up debug

* fix ConstantArrayType

* import

* use createForNewestSupportedVersion

* fix ConstantArrayType cast on RecastingRemovalRector

* skip magic call on RecastingRemovalRector

* fix use List_ node over Array_ on Foreach_ valueVar on UnusedForeachValueToArrayKeysRector

* [ci-review] Rector Rectify

* fix List_ on Foreach_ valueVar on PHPStanNodeScopeResolver

* fix Array items always ArrayItem

* skip instanceof check

* skip instanceof check

* fix mirror comment on StmtsManipulator

* fix wrap heredoc print

* fix return null next print on ExplicitReturnNullRector

* fix print float/int override pScalar_int and pScalar_float

* re-run rector

* fix test with remove unused use on RenameClassRector test

* fix ArrowFunction unnecessary parentheses

* fix print in test for logical bool

* fix binary op parentheses

* fix on operator string + or -

* fix on /, *, ** as well

* add note for next todo binary op handling

* fix pow opration with binary op parentheses handling

* [ci-review] Rector Rectify

* pull constant from NodeVisitor

* only support php 7.4+ since phpstan 2 and php-parser 5 require php 7.4+

* downgrade to php 7.4

* remove php 7.2 e2e

* use simple parser for defaultAnalysisParser, with native parser php 7.0

* Revert "use simple parser for defaultAnalysisParser, with native parser php 7.0"

This reverts commit b3d6c45.

* Fix parsing php 7

* fix Throw_ from Expr on AddOverrideAttributeToOverriddenMethodsRector

* using inner try catch to ensure everything catched from root try

* use constant from NodeVisitor

* temporary handle later for doctrine, now focus on php-parser print

* fix reprint cond on ParenthesizeNestedTernaryRector

* PhpDocParser parent __construct() only have require 3 params now

* disable native phpdoc-parser doctrine detection

* Fix doctrine detection on ReadonlyPropertyRector

* differentiate doctrine

* later patch

* fix InterpolatedStringPart detection for CreateFunctionToAnonymousFunctionRector on ValueResolver

* print arrow function

* print ArrowFunction on Carbon

* fix set orig node on nested ternary on SwitchNegatedTernaryRector

* Fix BinaryOp right origNode on CombineIfRector (part 1)

* fix print on CombineIfRector (part 2)

* Fix orig node set on ChangeNestedForeachIfsToEarlyContinueRector

* [ci-review] Rector Rectify

* fix set orig node on SimplifyIfReturnBoolRector

* Add ReprintNodeVisitor, avoid add complexity to AbstractRector

* fix reprint boolean not on SimplifyEmptyTest

* fix right binary op

* require --dev phpstan-phpunit 2 and phpstan-webmozart-assert 2

* ignore instanceof phpstanAPi.class notice

* ignore instanceof phpstanApi.classConstant notice

* ignore assert notice in test on purpose

* php runtime check notice

* set treatPhpDocTypesAsCertain: false

* fix usage of ExtendedParametersAcceptor

* ignore notice on provideMinPhpVersion()

* ignore notice instanceof.alwaysTrue

* fix List_ node usage on UnusedForeachValueToArrayKeysRector

* ignore phpstanApi.constructor

* ignore return.unusedType as follow parent signature

* ignore varTag.nativeType as enforce native type

* ignore phpstan notice: deadCode.unreachable as unreliable on runtime

* fix always falsy on RemoveAlwaysElseRector

* ignore argumet.type and return.type identifier

* fix use Token as array on File value object

* fix invalid ConstantScalar type usage on ValueResolver

* ignore runtime comparison

* fix empty array on TernaryToNullCoalescingRector

* fix empty array on JsonThrowOnErrorRector

* fix never null on RemoveDeadIfForeachForRector

* fix useless array_values() on OverridenExistingNamesResolver

* fix variable assign

* fix ArrayType on TypeNormalizer

* ignore phpstanApi.varTagAssumption

* fix phpdoc parse error

* fix isClassString() usage

* fix phpstan notice

* fix

* fix intersection

* reprint fix

* fallback scope

* fallback scope

* [ci-review] Rector Rectify

* fix phpstan

* fix phpstan ArrayItem

* [ci-review] Rector Rectify

* First step of Doctrine works: detect doctrine class from use statement via ObjectTypeSpecifier service

* fix phpstan

* [ci-review] Rector Rectify

* Fix alias object detection

* Fix alias object detection

* fix skip nette presenter

* no need class exists check on fqcn

* fix

* fix doctrine rename on RenameClassRectorTest

* add JoinColumns stubs

* add InverseJoinColumn stubs

* Fix named arg on NamedArgsFactory on ArrayItem

* [ci-review] Rector Rectify

* fix shortened object from use

* Fix ClassConstFetch as key on AttributeArrayNameInliner

* [ci-review] Rector Rectify

* fix numeric value in attribute

* fix auto import doctrine

* handle numeric int as key

* [ci-review] Rector Rectify

* fix CurlylistNode usage on AnnotationToAttributeRector

* [ci-review] Rector Rectify

* fix nested attribute group apply

* Fix inline array

* Fix inline array

* Fix printing generic

* fix printing generic

* add UniqueEntity stub

* fix from use statement

* remove unused stub

* Fix multiline values

* Fix multiline values

* fix phpstan

* fix requirement as php 7.4

* fix process single quoted

* [ci-review] Rector Rectify

* fix multi line

* fix with description

* use php 7.4 for linter

* temporary update fixture to avoid crash single quote, due to phpstan-phpdoc-parser internal error

* finally, unit test green

* Coalesce print

* fix interpolat print

* [ci-review] Rector Rectify

* patch ArrayItem usge on NodeFactory::createArgs()

* doc indent check

* handle doc indentation

* [ci-review] Rector Rectify

* require-dev phpstan/phpstan-deprecation-rules and fix deprecated method call

* fix variable on full_build.sh

* fix phsptan message return

* Conditionally to not require rector/rector-src on PR 6431 for packages tests CI

* final touch: enable return.type identifier phpstan

* really final touch: comment

* final touch: clean up always true check

* final touch: clean up ignore config

* really really final touch: clean up staticMethod.alreadyNarrowedType notice

---------

Co-authored-by: GitHub Action <[email protected]>

Author: samsonasik

.github/workflows/build_scoped_rector.yaml

@@ -66,11 +66,11 @@ jobs:
             # 3. prefix classes
             -   run: sh build/build-rector-scoped.sh rector-build rector-prefixed-downgraded
 
-            # 4. lint the code for PHP 7.2 - this must happen here, as setup-php allows only one PHP version switch: https://github.com/shivammathur/setup-php/issues/434
+            # 4. lint the code for PHP 7.4 - this must happen here, as setup-php allows only one PHP version switch: https://github.com/shivammathur/setup-php/issues/434
             -
                 uses: shivammathur/setup-php@v2
                 with:
-                    php-version: 7.2
+                    php-version: 7.4
                     coverage: none
             -   run: composer global require php-parallel-lint/php-parallel-lint --ansi
             -   run: /home/runner/.composer/vendor/bin/parallel-lint rector-prefixed-downgraded --exclude rector-prefixed-downgraded/stubs --exclude rector-prefixed-downgraded/vendor/tracy/tracy/examples --exclude rector-prefixed-downgraded/vendor/rector/rector-generator/templates --exclude rector-prefixed-downgraded/vendor/symfony/console/Debug/CliRequest.php

.github/workflows/code_analysis.yaml

@@ -27,7 +27,7 @@ jobs:
 
                     -
                         name: 'PHPStan'
-                        run: vendor/bin/phpstan analyse --ansi --error-format symplify
+                        run: vendor/bin/phpstan analyse --ansi
 
                     -
                         name: 'Commented Code'

.github/workflows/packages_tests.yaml

@@ -50,10 +50,19 @@ jobs:
 
             -   run: composer config minimum-stability dev
 
-            # test with current commit in a pull-request
-            -
-                run: composer require rector/rector-src dev-main#${{github.event.pull_request.head.sha}} --no-update
-                if: ${{ github.event_name == 'pull_request' }}
+            # temporary for PR 6431 as rector/rector-* packages already require this PR branch
+            # so avoid pointed to rector-src main branch composer.json
+            - name: Conditionally to not require rector/rector-src on PR 6431
+              if: ${{ github.event_name == 'pull_request' }}
+              run: |
+                if [ "${{ github.event.pull_request.number }}" != "6431" ]; then
+                  composer require rector/rector-src dev-main#${{ github.event.pull_request.head.sha }} --no-update
+                fi
+
+#            # test with current commit in a pull-request
+#            -
+#                run: composer require rector/rector-src dev-main#${{github.event.pull_request.head.sha}} --no-update
+#                if: ${{ github.event_name == 'pull_request' }}
 
             -   run: composer install --ansi
 

README.md

@@ -11,7 +11,7 @@ Head to [`rectorphp/rector`](http://github.com/rectorphp/rector) for documentati
 
 ## Building `rectorphp/rector`
 
-Code of this repository requires PHP 8. For `rector/rector` package the build downgrades code to PHP 7.2+.
+Code of this repository requires PHP 8. For `rector/rector` package the build downgrades code to PHP 7.4+.
 
 <br>
 

build/build-preload.php

@@ -76,6 +76,7 @@ final class PreloadBuilder
         'TokenEmulator.php',
         'KeywordEmulator.php',
         'Comment.php',
+        'PrettyPrinter.php',
         'PrettyPrinterAbstract.php',
         'Parser.php',
         'ParserAbstract.php',
@@ -104,6 +105,21 @@ final class PreloadBuilder
         'TypeNode.php',
     ];
 
+    /**
+     * The classes are deprecated and moved under Node
+     *
+     * @var string[]
+     */
+    private const IN_USE_CLASS_FILES = [
+        'Node/Expr/ArrayItem.php',
+        'Node/Expr/ClosureUse.php',
+        'Node/Scalar/EncapsedStringPart.php',
+        'Node/Scalar/LNumber.php',
+        'Node/Stmt/DeclareDeclare.php',
+        'Node/Stmt/PropertyProperty.php',
+        'Node/Stmt/StaticVar.php',
+    ];
+
     public function buildPreloadScript(string $buildDirectory, string $preloadFile): void
     {
         $this->buildPreloadScriptPhpParser($buildDirectory, $preloadFile);
@@ -315,6 +331,17 @@ private function findPhpParserFilesAndSortThem(string $vendorDir): array
         // 2. put first-class usages first
         $fileInfos = $this->sortFileInfos($fileInfos);
 
+        foreach ($fileInfos as $key => $fileInfo) {
+            foreach (self::IN_USE_CLASS_FILES as $inUseClassFile) {
+                if (str_ends_with($fileInfo->getPathname(), $inUseClassFile)) {
+                    unset($fileInfos[$key]);
+                    continue 2;
+                }
+            }
+        }
+
+        $fileInfos = array_values($fileInfos);
+
         $stmtsAwareInterface = new SplFileInfo(__DIR__ . '/../src/Contract/PhpParser/Node/StmtsAwareInterface.php');
         array_splice($fileInfos, 1, 0, [$stmtsAwareInterface]);
 

build/config/config-downgrade.php

@@ -9,7 +9,7 @@
 return RectorConfig::configure()
     ->withSkip(DowngradeRectorConfig::DEPENDENCY_EXCLUDE_PATHS)
     ->withPHPStanConfigs([__DIR__ . '/phpstan-for-downgrade.neon'])
-    ->withDowngradeSets(php72: true);
+    ->withDowngradeSets(php74: true);
 
 /**
  * Configuration consts for the different rector.php config files

build/config/phpstan-for-downgrade.neon

@@ -5,7 +5,3 @@ parameters:
     scanDirectories:
         # this is needed for symfony/dependendency-injection as it has hidden dependency on symfony/expression-language that we don't use here
         - ../../stubs
-
-    # see https://github.com/rectorphp/rector/issues/3490#issue-634342324
-    featureToggles:
-        disableRuntimeReflectionProvider: true

build/target-repository/.github/workflows/along_other_packages.yaml

@@ -13,7 +13,7 @@ jobs:
         strategy:
             fail-fast: false
             matrix:
-                php_version: ['7.2', '7.3', '7.4', '8.0', '8.1']
+                php_version: ['7.4', '8.0', '8.1']
                 commands:
                     -
                         name: 'Composer Dependency'

build/target-repository/.github/workflows/e2e.yaml

@@ -14,7 +14,7 @@ jobs:
         strategy:
             fail-fast: false
             matrix:
-                php_version: ['7.2', '7.3', '7.4', '8.0', '8.1']
+                php_version: ['7.4', '8.0', '8.1']
                 directory:
                     - 'e2e/define-constant'
                     - 'e2e/dont-execute-code'

build/target-repository/.github/workflows/e2e_diff.yaml

@@ -14,7 +14,7 @@ jobs:
         strategy:
             fail-fast: false
             matrix:
-                php_version: ['7.2', '7.3', '7.4', '8.0', '8.1']
+                php_version: ['7.4', '8.0', '8.1']
                 directory:
                     - 'e2e/attributes'