Merge pull request github#11420 from github/felicitymay-8441-query-guides-java

LGTM deprecation: updates to CodeQL for Java articles

Author: felicitymay

docs/codeql/codeql-language-guides/annotations-in-java.rst

@@ -51,7 +51,7 @@ We could then write this query to find all ``@SuppressWarnings`` annotations att
        anntp.hasQualifiedName("java.lang", "SuppressWarnings")
    select ann, ann.getValue("value")
 
-➤ `See the full query in the query console on LGTM.com <https://lgtm.com/query/1775658606775222283/>`__. Several of the LGTM.com demo projects use the ``@SuppressWarnings`` annotation. Looking at the ``value``\ s of the annotation element returned by the query, we can see that the *apache/activemq* project uses the ``"rawtypes"`` value described above.
+If the codebase you are analyzing uses the ``@SuppressWarnings`` annotation, you can check the ``value``\ s of the annotation element returned by the query. They should use the ``"rawtypes"`` value described above.
 
 As another example, this query finds all annotation types that only have a single annotation element, which has name ``value``:
 
@@ -66,8 +66,6 @@ As another example, this query finds all annotation types that only have a singl
    )
    select anntp
 
-➤ `See the full query in the query console on LGTM.com <https://lgtm.com/query/2145264152490258283/>`__.
-
 Example: Finding missing ``@Override`` annotations
 --------------------------------------------------
 
@@ -124,7 +122,7 @@ This makes it very easy to write our query for finding methods that override ano
        not overriding.getAnAnnotation() instanceof OverrideAnnotation
    select overriding, "Method overrides another method, but does not have an @Override annotation."
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7419756266089837339/>`__. In practice, this query may yield many results from compiled library code, which aren't very interesting. It's therefore a good idea to add another conjunct ``overriding.fromSource()`` to restrict the result to only report methods for which source code is available.
+In practice, this query may yield many results from compiled library code, which aren't very interesting. It's therefore a good idea to add another conjunct ``overriding.fromSource()`` to restrict the result to only report methods for which source code is available.
 
 Example: Finding calls to deprecated methods
 --------------------------------------------
@@ -237,7 +235,7 @@ Now we can extend our query to filter out calls in methods carrying a ``Suppress
        and not call.getCaller().getAnAnnotation() instanceof SuppressDeprecationWarningAnnotation
    select call, "This call invokes a deprecated method."
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8706367340403790260/>`__. It's fairly common for projects to contain calls to methods that appear to be deprecated.
+It's fairly common for projects to contain calls to methods that appear to be deprecated.
 
 Further reading
 ---------------

docs/codeql/codeql-language-guides/codeql-for-java.rst

@@ -26,7 +26,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
    working-with-source-locations
    abstract-syntax-tree-classes-for-working-with-java-programs
 
--  :doc:`Basic query for Java code <basic-query-for-java-code>`: Learn to write and run a simple CodeQL query using LGTM.
+-  :doc:`Basic query for Java code <basic-query-for-java-code>`: Learn to write and run a simple CodeQL query.
 
 -  :doc:`CodeQL library for Java <codeql-library-for-java>`: When analyzing Java code, you can use the large collection of classes in the CodeQL library for Java.
 

docs/codeql/codeql-language-guides/codeql-library-for-java.rst

@@ -70,7 +70,7 @@ For example, the following query finds all variables of type ``int`` in the prog
        pt.hasName("int")
    select v
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/860076406167044435/>`__. You're likely to get many results when you run this query because most projects contain many variables of type ``int``.
+You're likely to get many results when you run this query because most projects contain many variables of type ``int``.
 
 Reference types are also categorized according to their declaration scope:
 
@@ -87,7 +87,7 @@ For instance, this query finds all top-level types whose name is not the same as
    where tl.getName() != tl.getCompilationUnit().getName()
    select tl
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/4340983612585284460/>`__. This pattern is seen in many projects. When we ran it on the LGTM.com demo projects, most of the projects had at least one instance of this problem in the source code. There were many more instances in the files referenced by the source code.
+You will typically see this pattern in the source code of a repository, with many more instances in the files referenced by the source code.
 
 Several more specialized classes are available as well:
 
@@ -109,7 +109,7 @@ As an example, we can write a query that finds all nested classes that directly
    where nc.getASupertype() instanceof TypeObject
    select nc
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8482509736206423238/>`__. You're likely to get many results when you run this query because many projects include nested classes that extend ``Object`` directly.
+You're likely to get many results when you run this query because many projects include nested classes that extend ``Object`` directly.
 
 Generics
 ~~~~~~~~
@@ -143,8 +143,6 @@ For instance, we could use the following query to find all parameterized instanc
        pt.getSourceDeclaration() = map
    select pt
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7863873821043873550/>`__. None of the LGTM.com demo projects contain parameterized instances of ``java.util.Map`` in their source code, but they all have results in reference files.
-
 In general, generic types may restrict which types a type parameter can be bound to. For instance, a type of maps from strings to numbers could be declared as follows:
 
 .. code-block:: java
@@ -166,8 +164,6 @@ As an example, the following query finds all type variables with type bound ``Nu
        tb.getType().hasQualifiedName("java.lang", "Number")
    select tv
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6740696080876162817/>`__. When we ran it on the LGTM.com demo projects, the *neo4j/neo4j*, *hibernate/hibernate-orm* and *apache/hadoop* projects all contained examples of this pattern.
-
 For dealing with legacy code that is unaware of generics, every generic type has a "raw" version without any type parameters. In the CodeQL libraries, raw types are represented using class ``RawType``, which has the expected subclasses ``RawClass`` and ``RawInterface``. Again, there is a predicate ``getSourceDeclaration`` for obtaining the corresponding generic type. As an example, we can find variables of (raw) type ``Map``:
 
 .. code-block:: ql
@@ -179,8 +175,6 @@ For dealing with legacy code that is unaware of generics, every generic type has
        rt.getSourceDeclaration().hasQualifiedName("java.util", "Map")
    select v
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/4032913402499547882/>`__. Many projects have variables of raw type ``Map``.
-
 For example, in the following code snippet this query would find ``m1``, but not ``m2``:
 
 .. code-block:: java
@@ -230,7 +224,7 @@ For example, the following query finds all expressions whose parents are ``retur
    where e.getParent() instanceof ReturnStmt
    select e
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1947757851560375919/>`__. Many projects have examples of ``return`` statements with child expressions.
+Many projects have examples of ``return`` statements with child expressions.
 
 Therefore, if the program contains a return statement ``return x + y;``, this query will return ``x + y``.
 
@@ -244,7 +238,7 @@ As another example, the following query finds statements whose parent is an ``if
    where s.getParent() instanceof IfStmt
    select s
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1989464153689219612/>`__. Many projects have examples of ``if`` statements with child statements.
+Many projects have examples of ``if`` statements with child statements.
 
 This query will find both ``then`` branches and ``else`` branches of all ``if`` statements in the program.
 
@@ -258,8 +252,6 @@ Finally, here is a query that finds method bodies:
    where s.getParent() instanceof Method
    select s
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1016821702972128245/>`__. Most projects have many method bodies.
-
 As these examples show, the parent node of an expression is not always an expression: it may also be a statement, for example, an ``IfStmt``. Similarly, the parent node of a statement is not always a statement: it may also be a method or a constructor. To capture this, the QL Java library provides two abstract class ``ExprParent`` and ``StmtParent``, the former representing any node that may be the parent node of an expression, and the latter any node that may be the parent node of a statement.
 
 For more information on working with AST classes, see the :doc:`article on overflow-prone comparisons in Java <overflow-prone-comparisons-in-java>`.
@@ -278,7 +270,7 @@ For annotations, class ``Annotatable`` is a superclass of all program elements t
    from Constructor c
    select c.getAnAnnotation()
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/3206112561297137365/>`__. The LGTM.com demo projects all use annotations, you can see examples where they are used to suppress warnings and mark code as deprecated.
+You may see examples where annotations are used to suppress warnings or to mark code as deprecated.
 
 These annotations are represented by class ``Annotation``. An annotation is simply an expression whose type is an ``AnnotationType``. For example, you can amend this query so that it only reports deprecated constructors:
 
@@ -292,7 +284,7 @@ These annotations are represented by class ``Annotation``. An annotation is simp
        anntp.hasQualifiedName("java.lang", "Deprecated")
    select ann
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/5393027107459215059/>`__. Only constructors with the ``@Deprecated`` annotation are reported this time.
+Only constructors with the ``@Deprecated`` annotation are reported this time.
 
 For more information on working with annotations, see the :doc:`article on annotations <annotations-in-java>`.
 
@@ -307,7 +299,7 @@ For Javadoc, class ``Element`` has a member predicate ``getDoc`` that returns a
        jdoc = f.getDoc().getJavadoc()
    select jdoc
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6022769142134600659/>`__. You can see this pattern in many projects.
+You can see this pattern in many projects.
 
 Class ``Javadoc`` represents an entire Javadoc comment as a tree of ``JavadocElement`` nodes, which can be traversed using member predicates ``getAChild`` and ``getParent``. For instance, you could edit the query so that it finds all ``@author`` tags in Javadoc comments on private fields:
 
@@ -321,8 +313,6 @@ Class ``Javadoc`` represents an entire Javadoc comment as a tree of ``JavadocEle
        at.getParent+() = jdoc
    select at
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/2510220694395289111/>`__. None of the LGTM.com demo projects uses the ``@author`` tag on private fields.
-
 .. pull-quote::
 
    Note
@@ -349,7 +339,7 @@ For example, the following query finds methods with a `cyclomatic complexity <ht
        mc.getCyclomaticComplexity() > 40
    select m
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/6566950741051181919/>`__. Most large projects include some methods with a very high cyclomatic complexity. These methods are likely to be difficult to understand and test.
+Most large projects include some methods with a very high cyclomatic complexity. These methods are likely to be difficult to understand and test.
 
 Call graph
 ----------
@@ -369,8 +359,6 @@ We can use predicate ``Call.getCallee`` to find out which method or constructor
        m.hasName("println")
    select c
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/5861255162551917595/>`__. The LGTM.com demo projects all include many calls to methods of this name.
-
 Conversely, ``Callable.getAReference`` returns a ``Call`` that refers to it. So we can find methods and constructors that are never called using this query:
 
 .. code-block:: ql
@@ -381,7 +369,7 @@ Conversely, ``Callable.getAReference`` returns a ``Call`` that refers to it. So
    where not exists(c.getAReference())
    select c
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/7261739919657747703/>`__. The LGTM.com demo projects all appear to have many methods that are not called directly, but this is unlikely to be the whole story. To explore this area further, see ":doc:`Navigating the call graph <navigating-the-call-graph>`."
+Codebases often have many methods that are not called directly, but this is unlikely to be the whole story. To explore this area further, see ":doc:`Navigating the call graph <navigating-the-call-graph>`."
 
 For more information about callables and calls, see the :doc:`article on the call graph <navigating-the-call-graph>`.
 

docs/codeql/codeql-language-guides/javadoc.rst

@@ -149,8 +149,6 @@ Now we can write a query for finding all callables ``c`` and ``@throws`` tags ``
        not mayThrow(c, exn)
    select tt, "Spurious @throws tag."
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/1258570917227966396/>`__. This finds several results in the LGTM.com demo projects.
-
 Improvements
 ~~~~~~~~~~~~
 
@@ -216,7 +214,7 @@ The first case can be covered by changing ``getDocumentedException`` to use the
        (result.hasName(tt.getExceptionName()) and visibleIn(tt.getFile(), result))
    }
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/8016848987103345329/>`__. This finds many fewer, more interesting results in the LGTM.com demo projects.
+This query should find many fewer, more interesting results.
 
 Currently, ``visibleIn`` only considers single-type imports, but you could extend it with support for other kinds of imports.
 

docs/codeql/codeql-language-guides/overflow-prone-comparisons-in-java.rst

@@ -44,7 +44,7 @@ We'll start by writing a query that finds less-than expressions (CodeQL class ``
        expr.getRightOperand().getType().hasName("long")
    select expr
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/490866529746563234/>`__. This query usually finds results on most projects.
+This query usually finds results on most codebases.
 
 Notice that we use the predicate ``getType`` (available on all subclasses of ``Expr``) to determine the type of the operands. Types, in turn, define the ``hasName`` predicate, which allows us to identify the primitive types ``int`` and ``long``. As it stands, this query finds *all* less-than expressions comparing ``int`` and ``long``, but in fact we are only interested in comparisons that are part of a loop condition. Also, we want to filter out comparisons where either operand is constant, since these are less likely to be real bugs. The revised query looks like this:
 
@@ -59,7 +59,7 @@ Notice that we use the predicate ``getType`` (available on all subclasses of ``E
        not expr.getAnOperand().isCompileTimeConstant()
    select expr
 
-➤ `See this in the query console on LGTM.com <https://lgtm.com/query/4315986481180063825/>`__. Notice that fewer results are found.
+Notice that fewer results are found.
 
 The class ``LoopStmt`` is a common superclass of all loops, including, in particular, ``for`` loops as in our example above. While different kinds of loops have different syntax, they all have a loop condition, which can be accessed through predicate ``getCondition``. We use the reflexive transitive closure operator ``*`` applied to the ``getAChildExpr`` predicate to express the requirement that ``expr`` should be nested inside the loop condition. In particular, it can be the loop condition itself.
 
@@ -113,16 +113,44 @@ Now we rewrite our query to make use of these new classes:
 
 .. code-block:: ql
 
-   import Java
-
-   // Insert the class definitions from above
-
-   from OverflowProneComparison expr
-   where exists(LoopStmt l | l.getCondition().getAChildExpr*() = expr) and
-   not expr.getAnOperand().isCompileTimeConstant()
-   select expr
-
-➤ `See the full query in the query console on LGTM.com <https://lgtm.com/query/506868054626167462/>`__.
+    import java
+
+    // Return the width (in bits) of a given integral type 
+    int width(PrimitiveType pt) {
+    (pt.hasName("byte") and result=8) or
+    (pt.hasName("short") and result=16) or
+    (pt.hasName("char") and result=16) or
+    (pt.hasName("int") and result=32) or
+    (pt.hasName("long") and result=64)
+    }
+
+    // Find any comparison where the width of the type on the smaller end of 
+    // the comparison is less than the width of the type on the greater end
+    abstract class OverflowProneComparison extends ComparisonExpr {
+    Expr getLesserOperand() { none() }
+    Expr getGreaterOperand() { none() }
+    }
+
+    // Return `<=` and `<` comparisons
+    class LTOverflowProneComparison extends OverflowProneComparison {
+    LTOverflowProneComparison() {
+        (this instanceof LEExpr or this instanceof LTExpr) and
+        width(this.getLeftOperand().getType()) < width(this.getRightOperand().getType())
+    }
+    }
+
+    // Return `>=` and `>` comparisons
+    class GTOverflowProneComparison extends OverflowProneComparison {
+    GTOverflowProneComparison() {
+        (this instanceof GEExpr or this instanceof GTExpr) and
+        width(this.getRightOperand().getType()) < width(this.getLeftOperand().getType())
+    }
+    }
+
+    from OverflowProneComparison expr
+    where exists(LoopStmt l | l.getCondition().getAChildExpr*() = expr) and
+        not expr.getAnOperand().isCompileTimeConstant()
+    select expr
 
 Further reading
 ---------------