Very slow update/refresh with Gradle multi project including non Java project with large number of files

[jsaulou]

Hi,

I'm using a multi project build with a Spring Boot backend and an Angular frontend.
The Gradle build is used to create a jar for the frontend that serves as a dependency for the Spring Boot project.
Each project has its own build.gradle file. No issues there.

The problem is that the Java Language Server spends a large of amount of time (~15 minutes) updating and refreshing the root project. It seems that this is due to the large amount of files in the frontend project (node_modules) because removing this directory considerably improves the refresh time. Also, completely removing the frontend project makes everything work quickly as expected.

The refresh is slow even when the frontend project is not referenced at all in the Gradle build (not declared as a subproject and not referenced as a dependency).

Using the Gradle CLI, everything works normally. Configuration is fast.

I tried to remove the build.gradle files from the frontend project and configure it from the root project's build.gradle but same result.

Is the Language Server indexing or browsing all the files from the frontend project even though it is not a Java project?
Is there any way to make it ignore files or directories or non Java projects?

Thanks for your pointers.

Environment

• Operating System: Windows 10 Pro
• JDK version: openjdk11
• Visual Studio Code version: 1.43.2
• Java extension version: 0.59.0

[a-st]

We recently added a frontend component to one of our submodules and I can confirm this behaviour.

[snjeza]

@jsaulou @a-st could you try to set the following property:

"java.import.exclusions": [
        "**/node_modules/**",
        "**/.metadata/**",
        "**/archetype-resources/**",
        "**/META-INF/maven/**",
    ],

[fbricon]

@snjeza I think the problem is invoking IResource.refresh on projects containing large subdirectories. I'm not aware of ways to exclude directories from resource.refresh in Eclipse API

[a-st]

@snjeza Thanks a lot, I've added the settings to my settings.json and restarted vscode. Actually it takes quite long to finish this step

[jsaulou]

Thanks @snjeza for the tip but I had already tried that. I think I tried all combinations of exclusions like java.import.exclusions, files.watcherExclude, files.exclude. This is definitely better but still quite slow compared to a "standard" project (I think files.watcherExclude, files.exclude make the biggest difference). I'm pretty sure all files are refreshed in a way.

Also, I make use of Gradle's includeBuild in other projects referencing this project with large folders, and I have no idea how this would be treated by the extension. The settings defined in the large project cannot be applied then I guess.

---

Finally, I noticed that when using includeBuild, the extension will sometimes refresh a project that it shouldn't. For instance, let's say I have projects A and B.

A uses B with includeBuild. B has no dependency on A.

Well, sometimes, when opening or synchronizing B Gradle configuration, the extension will refresh project A. Not every time. I wonder if it's only when A is also opened and there is some kind of Gradle daemon mixup or something. I'm just speculating. Anyway, this might be another discussion.

[snjeza]

@a-st @jsaulou You can try to add the following code:

eclipse {
    project { 
        resourceFilter {
        matcher {
            id = 'org.eclipse.core.resources.regexFilterMatcher'
            arguments = "node_modules|.git"
        }
    }
  }
}

to your build.gradle file.

buildship doesn't support Eclipse resource filters. See eclipse-buildship/buildship#271
You can add them to your .project file using the following command:

./gradlew eclipse

[jsaulou]

Thank you so much @snjeza!

I've tried it and it makes a world of difference. It now only takes ~30s to update/sync the Gradle configuration.

[a-st]

I manually added the regexFilterMatcher to .project, which also solved the perfomance issues. Thank you very much @snjeza and @fbricon!

<projectDescription>
  <filteredResources>
    <filter>
      <matcher>
        <id>org.eclipse.core.resources.regexFilterMatcher</id>
        <arguments>node_modules</arguments>
      </matcher>
    </filter>
  </filteredResources>
</projectDescription>

[Slay3r]

@snjeza I have a similar issue.
I have a root project.
One sub-project which is using 'java' and 'application' in the build.gradle file (Spring Boot Server).
11 sub-projects which are 'java-library's.
Every sub-project have its own build.gradle file.

The Spring Boot Server is referencing to the 11 sub-project(libraries).

Every time I open vscode the extension starts to synchronize with gradle like in a earlier post.
This takes up to 5 minutes which is really annoying.

"node_modules" are not included in my project.

Any Idea how to fix?
Thanks in advance.

.project file which got generated:

<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
	<name>projetcxxxxxx.emf</name>
	<comment>Project projetcxxxxxx.emf created by Buildship.</comment>
	<projects>
	</projects>
	<buildSpec>
		<buildCommand>
			<name>org.eclipse.jdt.core.javabuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
		<buildCommand>
			<name>org.eclipse.buildship.core.gradleprojectbuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
	</buildSpec>
	<natures>
		<nature>org.eclipse.jdt.core.javanature</nature>
		<nature>org.eclipse.buildship.core.gradleprojectnature</nature>
	</natures>
</projectDescription>

.classpath file which got generated:

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
	<classpathentry kind="src" output="bin/main" path="src/main/java">
		<attributes>
			<attribute name="gradle_scope" value="main"/>
			<attribute name="gradle_used_by_scope" value="main,test"/>
		</attributes>
	</classpathentry>
	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-11/"/>
	<classpathentry kind="con" path="org.eclipse.buildship.core.gradleclasspathcontainer"/>
	<classpathentry kind="output" path="bin/default"/>
</classpath>

[jsaulou]

Hi @Slay3r,

If this can help, for me, the issue was not specifically a node_modules folder but rather a large number of files in some projects. If you have projects with large number of files, try what @snjeza said in the his previous comment.

[Slay3r]

Hi @jsaulou,
thanks for answering.

After adding the eclipse plugin and the command provided by @snjeza I could improve the Language Server startup time.

I am building emf plugins and in the emf edit package were folders with a lot of icons.
I excluded those folders.

Its now from 3 minutes down to 1 minute.
I think its still too long because my project is not that big but it's a step in the right direction.
Thanks.

[andywarren86]

I also recently noticed a large increase in the time required to synchronise the project. I too have a project with a nested node_modules folder. While ignoring node_modules as per above comments helped, it was still taking a while. I reverted the plugin back to v61 and sync speed dropped back to only a few seconds.

Has anything changed in v62 that could have caused this?

[Slay3r]

Like @andywarren86 I switched back to version 0.61.0.
The startup time is a lot better.