Merge pull request github#3622 from aschackmull/mergeback-124

p0 · web-flow · commit 7c0b8f5587d4 · 2020-06-08T08:38:12.000+01:00
Mergeback rc/1.24 -&gt; master
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -2,7 +2,7 @@
 
 We welcome contributions to our CodeQL libraries and queries. Got an idea for a new check, or how to improve an existing query? Then please go ahead and open a pull request! Contributions to this project are [released](https://help.github.com/articles/github-terms-of-service/#6-contributions-under-repository-license) to the public under the [project's open source license](LICENSE).
 
-There is lots of useful documentation to help you write queries, ranging from information about query file structure to tutorials for specific target languages. For more information on the documentation available, see [Writing CodeQL queries](https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html) on [help.semmle.com](https://help.semmle.com).
+There is lots of useful documentation to help you write queries, ranging from information about query file structure to tutorials for specific target languages. For more information on the documentation available, see [CodeQL queries](https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html) on [help.semmle.com](https://help.semmle.com).
 
 
 ## Submitting a new experimental query
@@ -32,7 +32,7 @@ If you have an idea for a query that you would like to share with other CodeQL u
 
     For details, see the [guide on query metadata](docs/query-metadata-style-guide.md).
 
-    Make sure the `select` statement is compatible with the query `@kind`. See [Introduction to query files](https://help.semmle.com/QL/learn-ql/writing-queries/introduction-to-queries.html#select-clause) on help.semmle.com.
+    Make sure the `select` statement is compatible with the query `@kind`. See [About CodeQL queries](https://help.semmle.com/QL/learn-ql/writing-queries/introduction-to-queries.html#select-clause) on help.semmle.com.
 
 3. **Formatting**
 
diff --git a/docs/language/learn-ql/java/ast-class-reference.rst b/docs/language/learn-ql/java/ast-class-reference.rst
@@ -1,7 +1,9 @@
-Classes for working with Java code
-==================================
+Abstract syntax tree classes for working with Java programs
+===========================================================
 
-CodeQL has a large selection of classes for working with Java statements and expressions.
+CodeQL has a large selection of classes for representing the abstract syntax tree of Java programs.
+
+.. include:: ../../reusables/abstract-syntax-tree.rst
 
 .. _Expr: https://help.semmle.com/qldoc/java/semmle/code/java/Expr.qll/type.Expr$Expr.html
 .. _Stmt: https://help.semmle.com/qldoc/java/semmle/code/java/Statement.qll/type.Statement$Stmt.html
diff --git a/docs/language/learn-ql/java/expressions-statements.rst b/docs/language/learn-ql/java/expressions-statements.rst
@@ -26,7 +26,7 @@ If ``l`` is bigger than 2\ :sup:`31`\ - 1 (the largest positive value of type ``
 
    All primitive numeric types have a maximum value, beyond which they will wrap around to their lowest possible value (called an "overflow"). For ``int``, this maximum value is 2\ :sup:`31`\ - 1. Type ``long`` can accommodate larger values up to a maximum of 2\ :sup:`63`\ - 1. In this example, this means that ``l`` can take on a value that is higher than the maximum for type ``int``; ``i`` will never be able to reach this value, instead overflowing and returning to a low value.
 
-We're going to develop a query that finds code that looks like it might exhibit this kind of behavior. We'll be using several of the standard library classes for representing statements and functions. For a full list, see :doc:`Classes for working with Java code <ast-class-reference>`.
+We're going to develop a query that finds code that looks like it might exhibit this kind of behavior. We'll be using several of the standard library classes for representing statements and functions. For a full list, see :doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`.
 
 Initial query
 -------------
diff --git a/docs/language/learn-ql/java/introduce-libraries-java.rst b/docs/language/learn-ql/java/introduce-libraries-java.rst
@@ -210,7 +210,7 @@ Class ``Variable`` represents a variable `in the Java sense <http://docs.oracle.
 Abstract syntax tree
 --------------------
 
-Classes in this category represent abstract syntax tree (AST) nodes, that is, statements (class ``Stmt``) and expressions (class ``Expr``). For a full list of expression and statement types available in the standard QL library, see :doc:`Classes for working with Java code <ast-class-reference>`.
+Classes in this category represent abstract syntax tree (AST) nodes, that is, statements (class ``Stmt``) and expressions (class ``Expr``). For a full list of expression and statement types available in the standard QL library, see :doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`.
 
 Both ``Expr`` and ``Stmt`` provide member predicates for exploring the abstract syntax tree of a program:
 
diff --git a/docs/language/learn-ql/java/ql-for-java.rst b/docs/language/learn-ql/java/ql-for-java.rst
@@ -34,5 +34,5 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
 
 -  :doc:`Working with source locations <source-locations>`: You can use the location of entities within Java code to look for potential errors. Locations allow you to deduce the presence, or absence, of white space which, in some cases, may indicate a problem.
 
--  :doc:`Classes for working with Java code <ast-class-reference>`: CodeQL has a large selection of classes for working with Java statements and expressions.
+-  :doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`: CodeQL has a large selection of classes for representing the abstract syntax tree of Java programs.
 
diff --git a/docs/language/learn-ql/java/types-class-hierarchy.rst b/docs/language/learn-ql/java/types-class-hierarchy.rst
@@ -114,7 +114,7 @@ To identify these cases, we can create two CodeQL classes that represent, respec
    class CollectionToArrayCall extends MethodAccess {
        CollectionToArrayCall() {
            exists(CollectionToArray m |
-               this.getMethod().getSourceDeclaration().overrides*(m)
+               this.getMethod().getSourceDeclaration().overridesOrInstantiates*(m)
            )
        }
 
@@ -124,7 +124,7 @@ To identify these cases, we can create two CodeQL classes that represent, respec
        }
    }
 
-Notice the use of ``getSourceDeclaration`` and ``overrides`` in the constructor of ``CollectionToArrayCall``: we want to find calls to ``Collection.toArray`` and to any method that overrides it, as well as any parameterized instances of these methods. In our example above, for instance, the call ``l.toArray`` resolves to method ``toArray`` in the raw class ``ArrayList``. Its source declaration is method\ ``toArray`` in the generic class ``ArrayList``, which overrides ``AbstractCollection.toArray``, which in turn overrides ``Collection.toArray``.
+Notice the use of ``getSourceDeclaration`` and ``overridesOrInstantiates`` in the constructor of ``CollectionToArrayCall``: we want to find calls to ``Collection.toArray`` and to any method that overrides it, as well as any parameterized instances of these methods. In our example above, for instance, the call ``l.toArray`` resolves to method ``toArray`` in the raw class ``ArrayList``. Its source declaration is ``toArray`` in the generic class ``ArrayList<T>``, which overrides ``AbstractCollection<T>.toArray``, which in turn overrides ``Collection<T>.toArray``, which is an instantiation of ``Collection.toArray`` (since the type parameter ``T`` in the overridden method belongs to ``ArrayList`` and is an instantiation of the type parameter belonging to ``Collection``).
 
 Using these new classes we can extend our query to exclude calls to ``toArray`` on an argument of type ``A[]`` which are then cast to ``A[]``:
 
diff --git a/docs/language/learn-ql/javascript/ast-class-reference.rst b/docs/language/learn-ql/javascript/ast-class-reference.rst
@@ -1,7 +1,9 @@
-Abstract syntax tree classes for JavaScript and TypeScript
-==========================================================
+Abstract syntax tree classes for working with JavaScript and TypeScript programs
+================================================================================
 
-CodeQL has a large selection of classes for working with JavaScript and TypeScript statements and expressions.
+CodeQL has a large selection of classes for representing the abstract syntax tree of JavaScript and TypeScript programs.
+
+.. include:: ../../reusables/abstract-syntax-tree.rst
 
 Statement classes
 -----------------
diff --git a/docs/language/learn-ql/javascript/dataflow-cheat-sheet.rst b/docs/language/learn-ql/javascript/dataflow-cheat-sheet.rst
@@ -142,7 +142,7 @@ Files
 AST nodes
 ---------
 
-See also: :doc:`Abstract syntax tree classes for JavaScript and TypeScript <ast-class-reference>`.
+See also: :doc:`Abstract syntax tree classes for working with JavaScript and TypeScript programs <ast-class-reference>`.
 
 Conversion between DataFlow and AST nodes:
 
diff --git a/docs/language/learn-ql/javascript/ql-for-javascript.rst b/docs/language/learn-ql/javascript/ql-for-javascript.rst
@@ -26,6 +26,6 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
 
 -  :doc:`Using type tracking for API modeling <type-tracking>`: You can track data through an API by creating a model using the CodeQL type-tracking library for JavaScript.
 
--  :doc:`Abstract syntax tree classes for JavaScript and TypeScript <ast-class-reference>`: CodeQL has a large selection of classes for working with JavaScript and TypeScript statements and expressions.
+-  :doc:`Abstract syntax tree classes for working with JavaScript and TypeScript programs <ast-class-reference>`: CodeQL has a large selection of classes for representing the abstract syntax tree of JavaScript and TypeScript programs.
 
 -  :doc:`Data flow cheat sheet for JavaScript <dataflow-cheat-sheet>`: This article describes parts of the JavaScript libraries commonly used for variant analysis and in data flow queries.
diff --git a/docs/language/ql-handbook/modules.rst b/docs/language/ql-handbook/modules.rst
@@ -80,9 +80,12 @@ Query modules
 A query module is defined by a ``.ql`` file. It can contain any of the elements listed 
 in :ref:`module-bodies` below. 
 
-The difference is that a query module must have at least one query in its 
-:ref:`namespace <namespaces>`. This is usually a :ref:`select clause <select-clauses>`, 
-but can also be a :ref:`query predicate <query-predicates>`.
+Query modules are slightly different from other modules:
+
+- A query module can't be imported.
+- A query module must have at least one query in its 
+  :ref:`namespace <namespaces>`. This is usually a :ref:`select clause <select-clauses>`, 
+  but can also be a :ref:`query predicate <query-predicates>`.
 
 For example:
 
diff --git a/docs/language/reusables/abstract-syntax-tree.rst b/docs/language/reusables/abstract-syntax-tree.rst
@@ -0,0 +1 @@
+The `abstract syntax tree (AST) <https://en.wikipedia.org/wiki/Abstract_syntax_tree>`__ represents the syntactic structure of a program. Nodes on the AST represent elements such as statements and expressions.
diff --git a/docs/ql-style-guide.md b/docs/ql-style-guide.md
@@ -216,7 +216,7 @@ class Type extends ... {
 
 General requirements:
 
-1. Documentation *must* adhere to the [QLDoc specification](https://help.semmle.com/QL/QLDocSpecification.html).
+1. Documentation *must* adhere to the [QLDoc specification](https://help.semmle.com/QL/ql-handbook/qldoc.html).
 1. Use `/** ... */` for documentation, even for single line comments.
 1. For single-line documentation, the `/**` and `*/` are written on the same line as the comment.
 1. For multi-line documentation, the `/**` and `*/` are written on separate lines. There is a `*` preceding each comment line, aligned on the first `*`.
@@ -417,16 +417,16 @@ deprecated Expr getInitializer()
 
 | Phrase      | Meaning  |
 |-------------|----------|
-| *[annotation](https://help.semmle.com/QL/QLLanguageSpecification.html#annotations)* | An additional specifier used to modify a declaration, such as `private`, `override`, `deprecated`, `pragma`, `bindingset`, or `cached`. |
+| *[annotation](https://help.semmle.com/QL/ql-handbook/language.html#annotations)* | An additional specifier used to modify a declaration, such as `private`, `override`, `deprecated`, `pragma`, `bindingset`, or `cached`. |
 | *body* | The text inside `{ }`, `( )`, or each section of an `if`-`then`-`else` or `from`-`where`-`select`. |
 | *binary operator* | An operator with two operands, such as comparison operators, `and`, `or`, `implies`, or arithmetic operators. |
 | *call* | A *formula* that invokes a predicate, e.g. `this.isStatic()` or `calls(a,b)`. |
-| *[conjunct](https://help.semmle.com/QL/QLLanguageSpecification.html#conjunctions)* | A formula that is an operand to an `and`. |
+| *[conjunct](https://help.semmle.com/QL/ql-handbook/language.html#conjunctions)* | A formula that is an operand to an `and`. |
 | *declaration* | A class, module, predicate, field or newtype. |
-| *[disjunct](https://help.semmle.com/QL/QLLanguageSpecification.html#disjunctions)* | A formula that is an operand to an `or`. |
-| *[formula](https://help.semmle.com/QL/QLLanguageSpecification.html#formulas)* | A logical expression, such as `A = B`, a *call*, a *quantifier*, `and`, `or`, `not`, `in` or `instanceof`. |
+| *[disjunct](https://help.semmle.com/QL/ql-handbook/language.html#disjunctions)* | A formula that is an operand to an `or`. |
+| *[formula](https://help.semmle.com/QL/ql-handbook/language.html#formulas)* | A logical expression, such as `A = B`, a *call*, a *quantifier*, `and`, `or`, `not`, `in` or `instanceof`. |
 | *should/should not/avoid/prefer* | Adhere to this rule wherever possible, where it makes sense. |
 | *may/can* | This is a reasonable alternative, to be used with discretion. |
 | *must/always/do not* | Always adhere to this rule. |
-| *[quantifier/aggregation](https://help.semmle.com/QL/QLLanguageSpecification.html#aggregations)* | `exists`, `count`, `strictcount`, `any`, `forall`, `forex` and so on. |
+| *[quantifier/aggregation](https://help.semmle.com/QL/ql-handbook/language.html#aggregations)* | `exists`, `count`, `strictcount`, `any`, `forall`, `forex` and so on. |
 | *variable* | A parameter to a predicate, a field, a from variable, or a variable introduced by a *quantifier* or *aggregation*. |
diff --git a/docs/query-help-style-guide.md b/docs/query-help-style-guide.md
@@ -36,7 +36,7 @@ Section-level elements are used to group the information within the query help f
 3.  `example`—an example of code showing the problem. Where possible, this section should also include a solution to the issue.
 4.  `references`—relevant references, such as authoritative sources on language semantics and best practice. 
 
-For further information about the other section-level, block, list and table elements supported by query help files, see the [Query help reference](https://help.semmle.com/QL/learn-ql/ql/writing-queries/query-help.html) on help.semmle.com.
+For further information about the other section-level, block, list and table elements supported by query help files, see [Query help files](https://help.semmle.com/QL/learn-ql/ql/writing-queries/query-help.html) on help.semmle.com.
 
 
 ## English style
diff --git a/docs/query-metadata-style-guide.md b/docs/query-metadata-style-guide.md
@@ -11,22 +11,22 @@ Query files have the extension `.ql`. Each file has two distinct areas:
 *   Metadata area–displayed at the top of the file, contains the metadata that defines how results for the query are interpreted and gives a brief description of the purpose of the query.
 *   Query definition–defined using QL. The query includes a select statement, which defines the content and format of the results. For further information about writing QL, see the following topics:
     *   [Learning CodeQL](https://help.semmle.com/QL/learn-ql/index.html)
-    *   [QL language handbook](https://help.semmle.com/QL/ql-handbook/index.html)
-    *   [QL language specification](https://help.semmle.com/QL/ql-spec/language.html)
+    *   [QL language reference](https://help.semmle.com/QL/ql-handbook/index.html)
     *   [CodeQL style guide](https://github.com/github/codeql/blob/master/docs/ql-style-guide.md) 
 
 
 For examples of query files for the languages supported by CodeQL, visit the following links: 
 
 *   [C/C++ queries](https://help.semmle.com/wiki/display/CCPPOBJ/)
 *   [C# queries](https://help.semmle.com/wiki/display/CSHARP/)
+*   [Go queries](https://help.semmle.com/wiki/display/GO/)
 *   [Java queries](https://help.semmle.com/wiki/display/JAVA/)
 *   [JavaScript queries](https://help.semmle.com/wiki/display/JS/)
 *   [Python queries](https://help.semmle.com/wiki/display/PYTHON/)
 
 ## Metadata area
 
-Query file metadata contains important information that defines the identifier and purpose of the query. The metadata is included as the content of a valid [QLDoc](https://help.semmle.com/QL/ql-spec/qldoc.html) comment, on lines with leading whitespace followed by `*`, between an initial `/**` and a trailing `*/`. For example:
+Query file metadata contains important information that defines the identifier and purpose of the query. The metadata is included as the content of a valid [QLDoc](https://help.semmle.com/QL/ql-handbook/qldoc.html) comment, on lines with leading whitespace followed by `*`, between an initial `/**` and a trailing `*/`. For example:
 
 ```
 /**
@@ -134,6 +134,7 @@ There are also more specific `@tags` that can be added. See, the following pages
 
 *   [C/C++ queries](https://help.semmle.com/wiki/display/CCPPOBJ/)
 *   [C# queries](https://help.semmle.com/wiki/display/CSHARP/)
+*   [Go queries](https://help.semmle.com/wiki/display/GO/)
 *   [Java queries](https://help.semmle.com/wiki/display/JAVA/)
 *   [JavaScript queries](https://help.semmle.com/wiki/display/JS/)
 *   [Python queries](https://help.semmle.com/wiki/display/PYTHON/)
@@ -158,7 +159,7 @@ When you tag a query like this, the associated CWE pages from [MITRE.org](http:/
 
 ## QL area
 
-### Alert  messages
+### Alert messages
 
 The select clause of each alert query defines the alert message that is displayed for each result found by the query. Alert messages are strings that concisely describe the problem that the alert is highlighting and, if possible, also provide some context. For consistency, alert messages should adhere to the following guidelines:
 
@@ -167,14 +168,15 @@ The select clause of each alert query defines the alert message that is displaye
 *   Program element references should be in 'single quotes' to distinguish them from ordinary words. Quotes are not needed around substitutions ($@).
 *   Avoid constant alert message strings and include some context, if possible. For example, `The class 'Foo' is duplicated as 'Bar'.` is preferable to `This class is duplicated here.`
 *   Where you reference another program element, link to it if possible using a substitution (`$@`). Links should be used inline in the sentence, rather than as parenthesised lists or appositions. 
-*   When a message contains multiple links, construct a sentence that has the most variable link (that is, the link with most targets) last. For further information, see [Defining select statements](https://help.semmle.com/QL/learn-ql/ql/writing-queries/select-statement.html).
+*   When a message contains multiple links, construct a sentence that has the most variable link (that is, the link with most targets) last. For further information, see [Defining the results of a query](https://help.semmle.com/QL/learn-ql/ql/writing-queries/select-statement.html).
 
 For examples of select clauses and alert messages, see the query source files at the following pages:
 
 *   [C/C++ queries](https://help.semmle.com/wiki/display/CCPPOBJ/)
 *   [C# queries](https://help.semmle.com/wiki/display/CSHARP/)
+*   [Go queries](https://help.semmle.com/wiki/display/GO/)
 *   [Java queries](https://help.semmle.com/wiki/display/JAVA/)
 *   [JavaScript queries](https://help.semmle.com/wiki/display/JS/)
 *   [Python queries](https://help.semmle.com/wiki/display/PYTHON/)
 
-For further information on query writing, see  [Writing CodeQL queries](https://help.semmle.com/QL/learn-ql/ql/writing-queries/writing-queries.html). For more information on learning CodeQL, see [Learning CodeQL](https://help.semmle.com/QL/learn-ql/index.html).
+For further information on query writing, see [CodeQL queries](https://help.semmle.com/QL/learn-ql/ql/writing-queries/writing-queries.html). For more information on learning CodeQL, see [Learning CodeQL](https://help.semmle.com/QL/learn-ql/index.html).

Original file line number	Diff line number	Diff line change
`@@ -34,5 +34,5 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat`
`34`	`34`
`35`	`35`	- :doc:`Working with source locations <source-locations>`: You can use the location of entities within Java code to look for potential errors. Locations allow you to deduce the presence, or absence, of white space which, in some cases, may indicate a problem.
`36`	`36`
`37`		-- :doc:`Classes for working with Java code <ast-class-reference>`: CodeQL has a large selection of classes for working with Java statements and expressions.
	`37`	+- :doc:`Abstract syntax tree classes for working with Java programs <ast-class-reference>`: CodeQL has a large selection of classes for representing the abstract syntax tree of Java programs.
`38`	`38`
Original file line number	Diff line number	Diff line change
`@@ -114,7 +114,7 @@ To identify these cases, we can create two CodeQL classes that represent, respec`
`114`	`114`	`class CollectionToArrayCall extends MethodAccess {`
`115`	`115`	`CollectionToArrayCall() {`
`116`	`116`	`exists(CollectionToArray m \|`
`117`		`- this.getMethod().getSourceDeclaration().overrides*(m)`
	`117`	`+ this.getMethod().getSourceDeclaration().overridesOrInstantiates*(m)`
`118`	`118`	`)`
`119`	`119`	`}`
`120`	`120`
`@@ -124,7 +124,7 @@ To identify these cases, we can create two CodeQL classes that represent, respec`
`124`	`124`	`}`
`125`	`125`	`}`
`126`	`126`
`127`		-Notice the use of ``getSourceDeclaration`` and ``overrides`` in the constructor of ``CollectionToArrayCall``: we want to find calls to ``Collection.toArray`` and to any method that overrides it, as well as any parameterized instances of these methods. In our example above, for instance, the call ``l.toArray`` resolves to method ``toArray`` in the raw class ``ArrayList``. Its source declaration is method\ ``toArray`` in the generic class ``ArrayList``, which overrides ``AbstractCollection.toArray``, which in turn overrides ``Collection.toArray``.
	`127`	+Notice the use of ``getSourceDeclaration`` and ``overridesOrInstantiates`` in the constructor of ``CollectionToArrayCall``: we want to find calls to ``Collection.toArray`` and to any method that overrides it, as well as any parameterized instances of these methods. In our example above, for instance, the call ``l.toArray`` resolves to method ``toArray`` in the raw class ``ArrayList``. Its source declaration is ``toArray`` in the generic class ``ArrayList<T>``, which overrides ``AbstractCollection<T>.toArray``, which in turn overrides ``Collection<T>.toArray``, which is an instantiation of ``Collection.toArray`` (since the type parameter ``T`` in the overridden method belongs to ``ArrayList`` and is an instantiation of the type parameter belonging to ``Collection``).
`128`	`128`
`129`	`129`	Using these new classes we can extend our query to exclude calls to ``toArray`` on an argument of type ``A[]`` which are then cast to ``A[]``:
`130`	`130`