AFU runs out of Java heap space when processing a large number of unrelated Java source files

[jthaine]

Given an input set of 1858 small (< 6 KB each) unrelated Java source files, on a Fedora VM with 8 GB of RAM, the AFU runs out of heap space when inserting annotations into source.

Repro steps:

1. Copy /www/secs-jenkins/bug_repros/afu/lotsofjavafiles.zip from tern.cs.washington.edu to your local machine.
2. Unzip the file.
3. Run:
   insert-annotations-to-source -v -i default.jaif @javafiles

Actual behavior: after processing ~1600 input files, the AFU slows down tremendously on the processing of each file, and ultimately fails with:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2367)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
at java.lang.AbstractStringBuilder.insert(AbstractStringBuilder.java:973)
at java.lang.StringBuilder.insert(StringBuilder.java:291)
at annotator.Source.insert(Source.java:151)
at annotator.Main.main(Main.java:971)

Expected: the AFU should not leak memory and thus should be able to process an unlimited number of unrelated Java source files without slowing down.

[dbrosoft]

It appears that the shared name table in the system Java compiler, which gets retrieved from the javac API and invoked (parse task and analyze task) once for each source file, is not getting reset in between files. Since invocation with thousands of files may be an uncommon mode of use, and since the problem can be worked around simply by invoking the program multiple times with subsets of the input, I have marked this issue as low priority.

[dbrosoft]

(Oops, wrong button!)