Merge branch 'master' into ir-this-parameter

Bring in new tests so their output can be fixed

Author: rdmarsh2

change-notes/1.25/analysis-cpp.md

@@ -16,6 +16,7 @@ The following changes in version 1.25 affect C/C++ analysis in all applications.
 
 ## Changes to libraries
 
+* The library `VCS.qll` and all queries that imported it have been removed.
 * The data-flow library has been improved, which affects most security queries by potentially
   adding more results. Flow through functions now takes nested field reads/writes into account.
   For example, the library is able to track flow from `taint()` to `sink()` via the method
@@ -39,3 +40,5 @@ The following changes in version 1.25 affect C/C++ analysis in all applications.
       }
   };
   ```
+* The security pack taint tracking library (`semmle.code.cpp.security.TaintTracking`) now considers that equality checks may block the flow of taint.  This results in fewer false positive results from queries that use this library.
+

change-notes/1.25/analysis-javascript.md

@@ -3,8 +3,10 @@
 ## General improvements
 
 * Support for the following frameworks and libraries has been improved:
+  - [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise)
   - [bluebird](http://bluebirdjs.com/)
   - [express](https://www.npmjs.com/package/express)
+  - [fastify](https://www.npmjs.com/package/fastify)
   - [fstream](https://www.npmjs.com/package/fstream)
   - [jGrowl](https://github.com/stanlemon/jGrowl)
   - [jQuery](https://jquery.com/)
@@ -13,12 +15,11 @@
   - [mssql](https://www.npmjs.com/package/mssql)
   - [mysql](https://www.npmjs.com/package/mysql)
   - [pg](https://www.npmjs.com/package/pg)
-  - [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise)
   - [sequelize](https://www.npmjs.com/package/sequelize)
   - [spanner](https://www.npmjs.com/package/spanner)
   - [sqlite](https://www.npmjs.com/package/sqlite)
-  - [ssh2](https://www.npmjs.com/package/ssh2)
   - [ssh2-streams](https://www.npmjs.com/package/ssh2-streams)
+  - [ssh2](https://www.npmjs.com/package/ssh2)
 
 * TypeScript 3.9 is now supported.
 
@@ -35,41 +36,43 @@
 
 | **Query**                      | **Expected impact**          | **Change**                                                                |
 |--------------------------------|------------------------------|---------------------------------------------------------------------------|
-| Misspelled variable name (`js/misspelled-variable-name`) | Message changed | The message for this query now correctly identifies the misspelled variable in additional cases. |
-| Uncontrolled data used in path expression (`js/path-injection`) | More results | This query now recognizes additional file system calls. |
-| Uncontrolled command line (`js/command-line-injection`) | More results | This query now recognizes additional command execution calls. |
-| Client-side URL redirect (`js/client-side-unvalidated-url-redirection`) | Less results | This query now recognizes additional safe patterns of doing URL redirects. |
-| Client-side cross-site scripting (`js/xss`) | Less results | This query now recognizes additional safe strings based on URLs. |
+| Client-side cross-site scripting (`js/xss`) | Fewer results | This query no longer flags optionally sanitized values. |
+| Client-side URL redirect (`js/client-side-unvalidated-url-redirection`) | Fewer results | This query now recognizes additional safe patterns of doing URL redirects. |
+| Client-side cross-site scripting (`js/xss`) | Fewer results | This query now recognizes additional safe patterns of constructing HTML. |
+| Code injection (`js/code-injection`) | More results | More potential vulnerabilities involving NoSQL code operators are now recognized. |
+| Expression has no effect (`js/useless-expression`) | Fewer results | This query no longer flags an expression when that expression is the only content of the containing file. |
 | Incomplete URL scheme check (`js/incomplete-url-scheme-check`) | More results | This query now recognizes additional url scheme checks. |
+| Misspelled variable name (`js/misspelled-variable-name`) | Message changed | The message for this query now correctly identifies the misspelled variable in additional cases. |
 | Prototype pollution in utility function (`js/prototype-pollution-utility`) | More results | This query now recognizes additional utility functions as vulnerable to prototype polution. |
-| Expression has no effect (`js/useless-expression`) | Less results | This query no longer flags an expression when that expression is the only content of the containing file. |
-| Unknown directive (`js/unknown-directive`) | Less results | This query no longer flags directives generated by the Babel compiler. |
-| Code injection (`js/code-injection`) | More results | More potential vulnerabilities involving NoSQL code operators are now recognized. |
+| Prototype pollution in utility function (`js/prototype-pollution-utility`) | More results | This query now recognizes more coding patterns that are vulnerable to prototype pollution. |
+| Uncontrolled command line (`js/command-line-injection`) | More results | This query now recognizes additional command execution calls. |
+| Uncontrolled data used in path expression (`js/path-injection`) | More results | This query now recognizes additional file system calls. |
+| Unknown directive (`js/unknown-directive`) | Fewer results | This query no longer flags directives generated by the Babel compiler. |
+| Unused property (`js/unused-property`) | Fewer results | This query no longer flags properties of objects that are operands of `yield` expressions. |
 | Zip Slip (`js/zipslip`) | More results | This query now recognizes additional vulnerabilities. |
-| Unused property (`js/unused-property`) | Less results | This query no longer flags properties of objects that are operands of `yield` expressions. |
 
 The following low-precision queries are no longer run by default on LGTM (their results already were not displayed):
 
   - `js/angular/dead-event-listener`
   - `js/angular/unused-dependency`
-  - `js/conflicting-html-attribute`
-  - `js/useless-assignment-to-global`
-  - `js/too-many-parameters`
-  - `js/unused-property`
   - `js/bitwise-sign-check`
   - `js/comparison-of-identical-expressions`
-  - `js/misspelled-identifier`
+  - `js/conflicting-html-attribute`
+  - `js/ignored-setter-parameter`
   - `js/jsdoc/malformed-param-tag`
-  - `js/jsdoc/unknown-parameter`
   - `js/jsdoc/missing-parameter`
-  - `js/omitted-array-element`
-  - `js/ignored-setter-parameter`
+  - `js/jsdoc/unknown-parameter`
   - `js/json-in-javascript-file`
+  - `js/misspelled-identifier`
+  - `js/nested-loops-with-same-variable`
   - `js/node/cyclic-import`
   - `js/node/unused-npm-dependency`
-  - `js/single-run-loop`
-  - `js/nested-loops-with-same-variable`
+  - `js/omitted-array-element`
   - `js/return-outside-function`
+  - `js/single-run-loop`
+  - `js/too-many-parameters`
+  - `js/unused-property`
+  - `js/useless-assignment-to-global`
 
 ## Changes to libraries
 
@@ -79,3 +82,4 @@ The following low-precision queries are no longer run by default on LGTM (their
   - `Parameter.flow()` now gets the correct data flow node for a parameter. Previously this had a result, but the node was disconnected from the data flow graph.
   - `ParameterNode.asExpr()` and `.getAstNode()` now gets the parameter's AST node, whereas previously it had no result.
   - `Expr.flow()` now has a more meaningful result for destructuring patterns. Previously this node was disconnected from the data flow graph. Now it represents the values being destructured by the pattern.
+* The global data-flow and taint-tracking libraries now model indirect parameter accesses through the `arguments` object in some cases, which may lead to additional results from some of the security queries, particularly "Prototype pollution in utility function".

cpp/ql/src/Likely Bugs/Underspecified Functions/TooFewArguments.qll

@@ -6,29 +6,50 @@
 
 import cpp
 
-// True if function was ()-declared, but not (void)-declared or K&R-defined
+/**
+ * Holds if `fde` has a parameter declaration that's clear on the minimum
+ * number of parameters. This is essentially true for everything except
+ * `()`-declarations.
+ */
+private predicate hasDefiniteNumberOfParameters(FunctionDeclarationEntry fde) {
+  fde.hasVoidParamList()
+  or
+  fde.getNumberOfParameters() > 0
+  or
+  fde.isDefinition()
+}
+
+/* Holds if function was ()-declared, but not (void)-declared or K&R-defined. */
 private predicate hasZeroParamDecl(Function f) {
   exists(FunctionDeclarationEntry fde | fde = f.getADeclarationEntry() |
-    not fde.hasVoidParamList() and fde.getNumberOfParameters() = 0 and not fde.isDefinition()
+    not hasDefiniteNumberOfParameters(fde)
   )
 }
 
-// True if this file (or header) was compiled as a C file
+/* Holds if this file (or header) was compiled as a C file. */
 private predicate isCompiledAsC(File f) {
   f.compiledAsC()
   or
   exists(File src | isCompiledAsC(src) | src.getAnIncludedFile() = f)
 }
 
+/** Holds if `fc` is a call to `f` with too few arguments. */
 predicate tooFewArguments(FunctionCall fc, Function f) {
   f = fc.getTarget() and
   not f.isVarargs() and
   not f instanceof BuiltInFunction and
+  // This query should only have results on C (not C++) functions that have a
+  // `()` parameter list somewhere. If it has results on other functions, then
+  // it's probably because the extractor only saw a partial compilation.
   hasZeroParamDecl(f) and
   isCompiledAsC(f.getFile()) and
-  // There is an explicit declaration of the function whose parameter count is larger
-  // than the number of call arguments
-  exists(FunctionDeclarationEntry fde | fde = f.getADeclarationEntry() |
+  // Produce an alert when all declarations that are authoritative on the
+  // parameter count specify a parameter count larger than the number of call
+  // arguments.
+  forex(FunctionDeclarationEntry fde |
+    fde = f.getADeclarationEntry() and
+    hasDefiniteNumberOfParameters(fde)
+  |
     fde.getNumberOfParameters() > fc.getNumberOfArguments()
   )
 }