Gradle configuration time too slow for development

[anuraaga]

I've noticed lately that during development when I have to retry tests, it's slowed down a lot since configuration time in our project seems unusually slow. The output in IDE will include something like this

> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
> Configure project :benchmark
Inferred project: opentelemetry-java-instrumentation, version: 1.1.0-SNAPSHOT
> Task :testing-common:processResources NO-SOURCE
> Task :instrumentation-api:sqlSanitizerJflex UP-TO-DATE

And it takes about 7s on my Macbook Pro 16 to move from

> Configure project :benchmark
Inferred project: opentelemetry-java-instrumentation, version: 1.1.0-SNAPSHOT

to

> Task :testing-common:processResources NO-SOURCE

I don't think it's supposed to take this long, and dominates the entire time for small test runs (or even many agent tests).

[ryandens]

I'd like to attempt some improvements here! I have two ideas

• Breaking the conventions included build into a few separate modules to reduce the impact of changes to plugins on every module. For example, the otel.jmh-conventions is only used in a few subprojects, but right now if any changes is made to jmh-conventions then every project needs to be reconfigured/rebuilt. This will have a relatively minor impact on day to day for most developers, but should have a significant impact when working on build changes

• Working towards enabling the Gradle Configuration Cache. This will take significant effort, but I'm excited by the potential savings here. There are a couple of issues I'm working on fixing, but mainly the problems are in upstream plugins.

  • I've already made changes to the jflex plugin to make it compatible with the configuration cache
  • I have a working prototype of changes to the Gradle XJC plugin used by this project. I'm less certain the XJC plugin will get merged though, as the project hasn't had maintenance in some time.
  • There's a small Gradle bug with the Scala plugin with old versions of scala with the configuration cache enabled. I have a reproducer project here and am determining what if any workarounds are available. It's possible the build tool will not change to support this older version of Scala.

  if anyone has other ideas for speeding up configuration time, let me know, and I'd be happy to try them out!

[trask]

hi @ryandens!

should have a significant impact when working on build changes

this part is not particularly a priority for us (but we wouldn't turn away a PR related to it)

Working towards enabling the Gradle Configuration Cache

awesome👍

It's possible the build tool will not change to support this older version of Scala

what's the oldest version of Scala that works? depending on how old, we might be able to consider changing our Scala version baseline

[ryandens]

this part is not particularly a priority for us (but we wouldn't turn away a PR related to it)

Makes sense! 👍

what's the oldest version of Scala that works? depending on how old, we might be able to consider changing our Scala version baseline

2.10.0 is the earliest that I've found that works! This was when Scala started supporting incremental compilation, which Gradle expects to be available, causing the failure. Not sure whether or not a Gradle fix/workaround is possible, but this isn't the only thing preventing configuration cache adoption, so there is no need to rush on a decision about compatibility. Good to know there's some flexibility there!

[ryandens]

I've found a potential workaround for the Scala issue! The scala-fork-join-2.8 project applies the otel.scala-conventions plugin which in turn applies the Scala Gradle plugin. However, this plugin only applies the Scala plugin to compile Scala in test source sets. The scala-fork-join-2.8 does not define any Scala code that needs to be compiled, so we could remove the otel.scala-conventions plugin from the scala-fork-join-2.8 project to simplify the build of that project. Other Scala instrumentation libraries like akka could still apply the otel.scala-conventions plugin and define custom Scala code in their test source set since they're testing with a supported version of Scala

[trask]

Closing, can revisit later if anyone has renewed interest.