[Type Inference] Instrument javac for trace/log/debug dump generation during type inference

[srikanth-sankaran]

In the area of type inference, there is a recurrent stream of non-trivial number of defect reports in the nature of this compiles with ECJ but not with javac and vice versa.

While both compilers attempt to implement the specification as closely as possible in letter and spirit, there are subtle differences that continue to cause grief. It is known that javac's inference engine predates the specification at JDK 8 time and differs in subtle ways in how it does some extra constitutional things. (@stephan-herrmann - Insert a link to some defect comment link that summarizes such)

In the absence of clarity, it is hard to bring about alignment between the three parties.

This ticket is for work on gathering insight into javac's inference engine's internal state by logging/tracing/dumping relevant internal state of the compiler.

To facilitate easy comparison with the trace/log/dump of ECJ's own internal state, we need to ascertain what information should be gathered and in what format it should be expressed.

This ticket is for discussions related to that.

It is conceivable that at some point, we may even submit a patch to OpenJDK so this tracing facility becomes a feature of "stock" OpenJDK javac enabled by a command line option.

[stephan-herrmann]

I'm maintaining a list of relevant topics in https://github.com/eclipse-jdt/eclipse.jdt.core/wiki/ECJ-Relevant-JDK-Issues

As further justification for this approach I can mention that in some of the JDK issues on that list we learned from Oracle, that partially javac may be closer to the intended semantics, while it is not 100% clear how this should be finally captured in JLS. Learning how javac comes to its conclusion in such cases is of paramount interest.

One of the difficulties with type inference is the fact, that this is a multi-phase algorithm, where in cases of deviation it may be arbitrarily hard to figure out which part of the algorithm diverged. Hopefully the planned tracing will narrow this down in a straight forward way.

[stephan-herrmann]

First on my wish list is a capability to print the current bound set at specific points during the algorithm.

ECJ has useful toString() methods in these classes:

• InferenceContext18
• BoundSet
• TypeBound
  • here ecj distinguishes TypeBound vs "Dependency", which, however, is not essential for tracing
• Subclasses of ConstraintFormula.
  • I seem to remember a hint that javac does not have a direct representation of constraints, but goes directly to TypeBounds. If that's so, this shouldn't be too much of a problem

For easy comparison the printout could perhaps be normalized

• can we define a normal direction for bounds (e.g. T1 <: T2 vs. T2 :> T1)?
  • if there's no natural direction we might even just print the thing in both directions for stable output
• we may want to sort bounds in a set (e.g., by plain text order)
• show types with simple name or FQN?

In a first go, it would be good to print the above before/after each of the major phases

• constructing constraints
• reduction
• incorporation
• resolution

At each point it is also relevant to distinguish applicability inference (strict, loose, varargs) vs invocation type inference etc.

Once we get to nested inference, things will get more tricky 😄

[srikanth-sankaran]

Once #4840 (looks ready) is integrated As #4840 is integrated, this is my top priority. Can we settle on a small sample program to begin with and can you share what information you are able to get from ECJ today ?

[stephan-herrmann]

Can we settle on a small sample program to begin with and can you share what information you are able to get from ECJ today ?

I checked a few open issues as candidates, but if only one inference variable is involved, the output is kind of meager.

How about this one from #3588:

import java.util.List;
import java.util.ArrayList;
import java.util.Arrays;

public class HelloWorld {
	public static void main(String... args) {
		Class[] classes = new Class[] { String.class, List.class };
		List<Class<?>> asdf = new ArrayList<>(Arrays.asList(classes));
	}
}

With InferenceContext18.DEBUG set to true, it spits out several pages showing how the bound set evolves, the final state and result being:

Infer invocation type for new ArrayList<>(Arrays.asList(classes)) with target List<Class<?>>
Result=
Type Bounds:
        TypeBound  T#0 :> java.lang.Class
        TypeBound  T#0 = java.lang.Class
        Dependency T#0 <: E'#2
        TypeBound  T#0 <: java.lang.Class<?>
        TypeBound  E'#2 :> java.lang.Class
        TypeBound  E'#2 = java.lang.Class<?>
Local Capture Bounds: <empty>
Other Capture Bounds: <empty>

Method substitute: public static ArrayList<Class<?>> <factory>(Collection<? extends Class<?>>) 

Does this give an idea?

Please ignore the difference of Local vs Other Capture bounds.

You might recall that e prime is an invention of ecj to represent a type variable during diamond inference, where E'#2 is then the ivar representing E', which happens to be the second ivar in this context.

[stephan-herrmann]

The above example is not extremely relevant due to the use of raw types.

What about #3351?

import java.util.List;
public class PassThroughGenerics {
    private class MyComp implements Comparable<MyComp> {
        @Override public int compareTo(MyComp other) { return 0; }
    }
    static <E extends Comparable<E>> List<E> sort(List<E> list) {
        return list;
    }
    static <T> List<T> genericList() {
        return null;
    }    
    public static void main(String[] args) {
        List<MyComp> sorted = sort(genericList());
        System.out.println(sorted);
    }
}

Here's the corresponding extract from ecj debug output:

Infer invocation type for sort(genericList()) with target List<PassThroughGenerics.MyComp>
Result=
Type Bounds:
        TypeBound  E#1 = PassThroughGenerics.MyComp
        Dependency E#1 <: java.lang.Comparable<E#1>
        TypeBound  E#1 <: java.lang.Comparable<PassThroughGenerics.MyComp>
        Dependency T#0 = E#1
        TypeBound  T#0 = PassThroughGenerics.MyComp
        Dependency T#0 <: java.lang.Comparable<E#1>
        TypeBound  T#0 <: java.lang.Comparable<PassThroughGenerics.MyComp>
Local Capture Bounds: <empty>
Other Capture Bounds: <empty>

Method substitute: static List<PassThroughGenerics.MyComp> sort(List<PassThroughGenerics.MyComp>)