Skip virtual nodes

UPGRADING.md

@@ -1,3 +1,63 @@
+# Upgrading from Psalm 6 to Psalm 7
+
+## Changed
+
+- [BC] Taints are now *internally* represented by a bitmap (an integer), instead of an array of strings. Users can still use the usual string taint identifiers (including custom ones, which will be automatically registered by Psalm), but internally, the type of `Psalm\Type\TaintKind` taint types is now an integer.
+
+- [BC] The maximum number of usable taint *types* (including both native taints and custom taints) is now equal to 32 on 32-bit systems and 64 on 64-bit systems: this should be enough for the vast majority of usecases, if more taint types are needed, consider merging some taint types or using some native taint types.  
+
+- [BC] `Psalm\Plugin\EventHandler\AddTaintsInterface::addTaints` and `Psalm\Plugin\EventHandler\RemoveTaintsInterface::removeTaints` now must return an integer taint instead of an array of strings (see the new [taint documentation](https://psalm.dev/docs/security_analysis/custom_taint_sources/) for more info).  
+
+- [BC] The type of the `$taints` parameter of `Psalm\Codebase::addTaintSource` and  `Psalm\Codebase::addTaintSink` was changed to an integer
+
+- [BC] Type of property `Psalm\Storage\FunctionLikeParameter::$sinks` changed from `array|null` to `int`
+
+- [BC] Type of property `Psalm\Storage\FunctionLikeStorage::$taint_source_types` changed from `array` to `int`
+
+- [BC] Type of property `Psalm\Storage\FunctionLikeStorage::$added_taints` changed from `array` to `int`
+
+- [BC] Type of property `Psalm\Storage\FunctionLikeStorage::$removed_taints` changed from `array` to `int`
+
+- [BC] The `startScanningFiles`, `startAnalyzingFiles`, `startAlteringFiles` of `Psalm\Progress\Progress` and subclasses were removed and replaced with a new `startPhase` method, taking a `Psalm\Progress\Phase` enum case.
+
+- [BC] The `start` method was removed, use `expand`, instead; the progress is reset to 0 when changing the current phase.  
+
+- [BC] Method `doesTerminalSupportUtf8` of class `Psalm\Progress\Progress` became final
+
+- [BC] Method debug() of class Psalm\Progress\Progress changed from concrete to abstract
+
+- [BC] Method alterFileDone() of class Psalm\Progress\Progress changed from concrete to abstract
+
+- [BC] Method expand() of class Psalm\Progress\Progress changed from concrete to abstract
+
+- [BC] Method taskDone() of class Psalm\Progress\Progress changed from concrete to abstract
+
+- [BC] Method finish() of class Psalm\Progress\Progress changed from concrete to abstract
+
+- [BC] The return type of Psalm\Type::getListAtomic() changed from Psalm\Type\Atomic\TKeyedArray to the non-covariant Psalm\Type\Atomic\TKeyedArray|Psalm\Type\Atomic\TArray
+
+- [BC] The return type of Psalm\Type::getListAtomic() changed from Psalm\Type\Atomic\TKeyedArray to Psalm\Type\Atomic\TKeyedArray|Psalm\Type\Atomic\TArray
+
+- [BC] The return type of Psalm\Type::getNonEmptyListAtomic() changed from Psalm\Type\Atomic\TKeyedArray to the non-covariant Psalm\Type\Atomic\TKeyedArray|Psalm\Type\Atomic\TArray
+
+- [BC] The return type of Psalm\Type::getNonEmptyListAtomic() changed from Psalm\Type\Atomic\TKeyedArray to Psalm\Type\Atomic\TKeyedArray|Psalm\Type\Atomic\TArray
+
+- [BC] Class Psalm\Type\Atomic\TKeyedArray became final
+
+- [BC] Class Psalm\Type\Atomic\TKeyedArray can only be created using the new `make` or `makeCallable` factory methods, the constructor was rendered private.  
+
+- [BC] Class Psalm\Type\Atomic\TCallableKeyedArray has been deleted, and replaced with a new `is_callable` flag in Psalm\Type\Atomic\TKeyedArray
+
+- [BC] Class Psalm\Type\Atomic\TCallableInterface has been deleted, use `\Psalm\Type\Atomic::isCallableType()` instead
+
+## Removed
+
+- [BC] Constant Psalm\Type\Atomic\TKeyedArray::NAME_ARRAY was removed
+
+- [BC] Constant Psalm\Type\Atomic\TKeyedArray::NAME_LIST was removed
+
+- [BC] Psalm\Type\Atomic\TKeyedArray#__construct() was made private
+
 # Upgrading from Psalm 5 to Psalm 6
 ## Changed
 

bin/docker/Dockerfile

@@ -57,7 +57,7 @@ RUN set -eux; \
 # Enable linker optimization (this sorts the hash buckets to improve cache locality, and is non-default)
 # https://github.com/docker-library/php/issues/272
 # -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64 (https://www.php.net/manual/en/intro.filesystem.php)
-ENV PHP_CFLAGS="-fstack-protector-strong -fpic -fpie -O3 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
+ENV PHP_CFLAGS="-fstack-protector-strong -fpic -fpie -O3 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g"
 ENV PHP_CPPFLAGS="$PHP_CFLAGS"
 ENV PHP_LDFLAGS="-Wl,-O1 -pie"
 
@@ -185,6 +185,7 @@ RUN set -eux; \
 	php --version
 
 COPY bin/docker/docker-php-* /usr/local/bin/
+ADD https://raw.githubusercontent.com/php/php-src/refs/heads/PHP-8.4/.gdbinit /root/.gdbinit
 
 # This line invalidates cache when master branch changes
 ADD https://github.com/vimeo/psalm/commits/master.atom /dev/null

composer.json

@@ -118,7 +118,7 @@
         "lint": "@php parallel-lint ./src ./tests",
         "phpunit": [
             "Composer\\Config::disableProcessTimeout",
-            "paratest -f --runner=WrapperRunner"
+            "@php paratest -f --runner=WrapperRunner"
         ],
         "phpunit-std": [
             "Composer\\Config::disableProcessTimeout",

config.xsd

@@ -81,7 +81,7 @@
         <xs:attribute name="skipChecksOnUnresolvableIncludes" type="xs:boolean" default="false" />
         <xs:attribute name="sealAllMethods" type="xs:boolean" default="true" />
         <xs:attribute name="sealAllProperties" type="xs:boolean" default="true" />
-        <xs:attribute name="runTaintAnalysis" type="xs:boolean" default="false" />
+        <xs:attribute name="runTaintAnalysis" type="xs:boolean" default="true" />
         <xs:attribute name="usePhpStormMetaPath" type="xs:boolean" default="true" />
         <xs:attribute name="allowInternalNamedArgumentCalls" type="xs:boolean" default="true" />
         <xs:attribute name="allowNamedArgumentCalls" type="xs:boolean" default="true" />

dictionaries/InternalTaintSinkMap.php

@@ -5,59 +5,59 @@
 // This maps internal function names to sink types that we don’t want to end up there
 
 /**
- * @var non-empty-array<string, non-empty-list<list<TaintKind::*>>>
+ * @var non-empty-array<string, non-empty-list<int-mask-of<TaintKind::*>>>
  */
 return [
-'exec' => [['shell']],
-'create_function' => [[], ['eval']],
-'file_get_contents' => [['file']],
-'file_put_contents' => [['file']],
-'fopen' => [['file']],
-'unlink' => [['file']],
-'copy' => [['file'], ['file']],
-'file' => [['file']],
-'link' => [['file'], ['file']],
-'mkdir' => [['file']],
-'move_uploaded_file' => [['file'], ['file']],
-'parse_ini_file' => [['file']],
-'chown' => [['file']],
-'lchown' => [['file']],
-'readfile' => [['file']],
-'rename' => [['file'], ['file']],
-'rmdir' => [['file']],
-'header' => [['header']],
-'symlink' => [['file']],
-'tempnam' => [['file']],
-'igbinary_unserialize' => [['unserialize']],
-'ldap_search' => [[], ['ldap'], ['ldap']],
-'mysqli_query' => [[], ['sql']],
-'mysqli::query' => [['sql']],
-'mysqli_real_query' => [[], ['sql']],
-'mysqli::real_query' => [['sql']],
-'mysqli_multi_query' => [[], ['sql']],
-'mysqli::multi_query' => [['sql']],
-'mysqli_prepare' => [[], ['sql']],
-'mysqli::prepare' => [['sql']],
-'mysqli_stmt::__construct' => [[], ['sql']],
-'mysqli_stmt_prepare' => [[], ['sql']],
-'mysqli_stmt::prepare' => [['sql']],
-'passthru' => [['shell']],
-'pcntl_exec' => [['shell']],
-'pg_exec' => [[], ['sql']],
-'pg_prepare' => [[], [], ['sql']],
-'pg_put_line' => [[], ['sql']],
-'pg_query' => [[], ['sql']],
-'pg_query_params' => [[], ['sql']],
-'pg_send_prepare' => [[], [], ['sql']],
-'pg_send_query' => [[], ['sql']],
-'pg_send_query_params' => [[], ['sql'], []],
-'setcookie' => [['cookie'], ['cookie']],
-'shell_exec' => [['shell']],
-'system' => [['shell']],
-'unserialize' => [['unserialize']],
-'popen' => [['shell']],
-'proc_open' => [['shell']],
-'curl_init' => [['ssrf']],
-'curl_setopt' => [[], [], ['ssrf']],
-'getimagesize' => [['ssrf']],
+'exec' => [TaintKind::INPUT_SHELL],
+'create_function' => [0, TaintKind::INPUT_EVAL],
+'file_get_contents' => [TaintKind::INPUT_FILE|TaintKind::INPUT_SSRF],
+'file_put_contents' => [TaintKind::INPUT_FILE],
+'fopen' => [TaintKind::INPUT_FILE],
+'unlink' => [TaintKind::INPUT_FILE],
+'copy' => [TaintKind::INPUT_FILE|TaintKind::INPUT_SSRF, TaintKind::INPUT_FILE],
+'file' => [TaintKind::INPUT_FILE],
+'link' => [TaintKind::INPUT_FILE, TaintKind::INPUT_FILE],
+'mkdir' => [TaintKind::INPUT_FILE],
+'move_uploaded_file' => [TaintKind::INPUT_FILE, TaintKind::INPUT_FILE],
+'parse_ini_file' => [TaintKind::INPUT_FILE],
+'chown' => [TaintKind::INPUT_FILE],
+'lchown' => [TaintKind::INPUT_FILE],
+'readfile' => [TaintKind::INPUT_FILE],
+'rename' => [TaintKind::INPUT_FILE, TaintKind::INPUT_FILE],
+'rmdir' => [TaintKind::INPUT_FILE],
+'header' => [TaintKind::INPUT_HEADER],
+'symlink' => [TaintKind::INPUT_FILE],
+'tempnam' => [TaintKind::INPUT_FILE],
+'igbinary_unserialize' => [TaintKind::INPUT_UNSERIALIZE],
+'ldap_search' => [0, TaintKind::INPUT_LDAP, TaintKind::INPUT_LDAP],
+'mysqli_query' => [0, TaintKind::INPUT_SQL],
+'mysqli::query' => [TaintKind::INPUT_SQL],
+'mysqli_real_query' => [0, TaintKind::INPUT_SQL],
+'mysqli::real_query' => [TaintKind::INPUT_SQL],
+'mysqli_multi_query' => [0, TaintKind::INPUT_SQL],
+'mysqli::multi_query' => [TaintKind::INPUT_SQL],
+'mysqli_prepare' => [0, TaintKind::INPUT_SQL],
+'mysqli::prepare' => [TaintKind::INPUT_SQL],
+'mysqli_stmt::__construct' => [0, TaintKind::INPUT_SQL],
+'mysqli_stmt_prepare' => [0, TaintKind::INPUT_SQL],
+'mysqli_stmt::prepare' => [TaintKind::INPUT_SQL],
+'passthru' => [TaintKind::INPUT_SHELL],
+'pcntl_exec' => [TaintKind::INPUT_SHELL],
+'pg_exec' => [0, TaintKind::INPUT_SQL],
+'pg_prepare' => [0, 0, TaintKind::INPUT_SQL],
+'pg_put_line' => [0, TaintKind::INPUT_SQL],
+'pg_query' => [0, TaintKind::INPUT_SQL],
+'pg_query_params' => [0, TaintKind::INPUT_SQL],
+'pg_send_prepare' => [0, 0, TaintKind::INPUT_SQL],
+'pg_send_query' => [0, TaintKind::INPUT_SQL],
+'pg_send_query_params' => [0, TaintKind::INPUT_SQL, 0],
+'setcookie' => [TaintKind::INPUT_COOKIE, TaintKind::INPUT_COOKIE],
+'shell_exec' => [TaintKind::INPUT_SHELL],
+'system' => [TaintKind::INPUT_SHELL],
+'unserialize' => [TaintKind::INPUT_UNSERIALIZE],
+'popen' => [TaintKind::INPUT_SHELL],
+'proc_open' => [TaintKind::INPUT_SHELL],
+'curl_init' => [TaintKind::INPUT_SSRF],
+'curl_setopt' => [0, 0, TaintKind::INPUT_SSRF],
+'getimagesize' => [TaintKind::INPUT_SSRF],
 ];

docs/running_psalm/command_line_usage.md

@@ -33,7 +33,9 @@ Currently, Shepherd tracks type coverage (the percentage of types Psalm can infe
 
 ## Running Psalm faster
 
-Psalm has a couple of command-line options that will result in faster builds:
+To run Psalm up to 50% faster, use the [official docker image](https://psalm.dev/docs/running_psalm/installation/#docker-image).  
+
+Psalm also has a couple of command-line options that will result in faster builds:
 
 - `--threads=[n]` to run Psalm’s analysis in a number of threads
 - `--diff` which only checks files you’ve updated since the last run (and their dependents).

docs/running_psalm/configuration.md

@@ -33,6 +33,12 @@ Configuration file may be split into several files using [XInclude](https://www.
 </projectFiles>
 ```
 
+## Different configuration file
+
+You can also create a different configuration file, then run Psalm with:
+```bash
+vendor/bin/psalm --config=other-psalm-config.xml
+```
 
 ## Optional &lt;psalm /&gt; attributes
 
@@ -393,7 +399,7 @@ When `true`, Psalm will treat all classes as if they had sealed properties, mean
 >
 ```
 
-When `true`, Psalm will run [Taint Analysis](../security_analysis/index.md) on your codebase. This config is the same as if you were running Psalm with `--taint-analysis`.
+When `true` (the default), Psalm will run [Taint Analysis](../security_analysis/index.md) on your codebase. This config is the same as if you were running Psalm with `--taint-analysis`.
 
 #### reportInfo
 

docs/running_psalm/issues/TaintedFile.md

@@ -26,7 +26,7 @@ It could range from:
 ```php
 <?php
 
-$content = file_get_contents($_GET['header']);
+$content = fopen($_GET['header'], "r");
 echo $content;
 ```
 

docs/running_psalm/plugins/plugins_type_system.md

@@ -252,7 +252,7 @@ More complex types can be constructed as follows. The following represents an as
 
 ``` php
         new Union([
-            new TKeyedArray([
+            TKeyedArray::make([
                 'key_1' => new Union([new TString()]),
                 'key_2' => new Union([new TInt()]),
                 'key_3' => new Union([new TBool()])])]);

docs/security_analysis/custom_taint_sources.md

@@ -6,7 +6,7 @@ You can define your own taint sources with an annotation or a plugin.
 
 You can use the annotation `@psalm-taint-source <taint-type>` to indicate a function or method that provides user input.
 
-In the below example the `input` taint type is specified as a standin for input taints as defined in [Psalm\Type\TaintKindGroup](https://github.com/vimeo/psalm/blob/master/src/Psalm/Type/TaintKindGroup.php).
+In the below example the `input` taint type is specified as a standin for input taints as defined in [Psalm\Type\TaintKind](https://github.com/vimeo/psalm/blob/master/src/Psalm/Type/TaintKind.php).
 
 ```php
 <?php
@@ -26,29 +26,70 @@ For example this plugin treats all variables named `$bad_data` as taint sources.
 namespace Psalm\Example\Plugin;
 
 use PhpParser\Node\Expr\Variable;
+use Psalm\Codebase;
 use Psalm\Plugin\EventHandler\AddTaintsInterface;
 use Psalm\Plugin\EventHandler\Event\AddRemoveTaintsEvent;
-use Psalm\Type\TaintKindGroup;
+use Psalm\Type\TaintKind;
 
 /**
- * Add input taints to all variables named 'bad_data'
+ * Add input taints to all variables named 'bad_data' or 'even_badder_data'.
+ * 
+ * RemoveTaintsInterface is also available to remove taints.
  */
 class TaintBadDataPlugin implements AddTaintsInterface
 {
+    private static int $myCustomTaint;
+    private static int $myCustomTaintAlias;
+    /**
+     * Must be called by the PluginEntryPointInterface (__invoke) of your plugin.
+     */
+    public static function init(Codebase $codebase): void
+    {
+        // Register a new custom taint
+        // The taint name may be used in @psalm-taint-* annotations in the code.
+        self::$myCustomTaint = $codebase->getOrRegisterTaint("my_custom_taint");
+
+        // Register a taint alias that combines multiple pre-registered taint types
+        // Taint alias names may be used in @psalm-taint-* annotations in the code.
+        self::$myCustomTaintAlias = $codebase->registerTaintAlias(
+            "my_custom_taint_alias",
+            self::$myCustomTaint | TaintKind::ALL_INPUT
+        );
+    }
+
     /**
      * Called to see what taints should be added
      *
-     * @return list<string>
+     * @return int A bitmap of taint from the IDs
      */
-    public static function addTaints(AddRemoveTaintsEvent $event): array
+    public static function addTaints(AddRemoveTaintsEvent $event): int
     {
         $expr = $event->getExpr();
 
         if ($expr instanceof Variable && $expr->name === 'bad_data') {
-            return TaintKindGroup::ALL_INPUT;
+            return TaintKind::ALL_INPUT;
+        }
+
+        if ($expr instanceof Variable && $expr->name === 'even_badder_data') {
+            return self::$myCustomTaint;
+        }
+
+        if ($expr instanceof Variable && $expr->name === 'even_badder_data_2') {
+            return self::$myCustomTaintAlias;
+        }
+
+        if ($expr instanceof Variable && $expr->name === 'secret_even_badder_data_3') {
+            // Combine taints using |
+            return self::$myCustomTaintAlias | USER_SECRET;
+        }
+
+        if ($expr instanceof Variable && $expr->name === 'bad_data_but_ok_cookie') {
+            // Remove taints using & and ~ to negate a taint (group)
+            return self::$myCustomTaintAlias & ~TaintKind::INPUT_COOKIE;
         }
 
-        return [];
+        // No taints
+        return 0;
     }
 }
 ```

docs/security_analysis/index.md

@@ -2,7 +2,7 @@
 
 Psalm can attempt to find connections between user-controlled input (like `$_GET['name']`) and places that we don’t want unescaped user-controlled input to end up (like `echo "<h1>$name</h1>"` by looking at the ways that data flows through your application (via assignments, function/method calls and array/property access).
 
-You can enable this mode with the `--taint-analysis` command line flag. When taint analysis is enabled, no other analysis is performed.  To [ensure comprehensive results](https://github.com/vimeo/psalm/issues/6156), Psalm should be run normally prior to taint analysis, and any errors should be fixed.
+You can enable this mode with the `--taint-analysis` command line flag.  
 
 Tainted input is anything that can be controlled, wholly or in part, by a user of your application. In taint analysis, tainted input is called a _taint source_.
 

examples/TemplateChecker.php

@@ -178,6 +178,7 @@ private function checkWithViewClass(Context $context, array $stmts): void
         $statements_source = new StatementsAnalyzer(
             $view_method_analyzer,
             new NodeDataProvider(),
+            false,
         );
 
         $statements_source->analyze($pseudo_method_stmts, $context);