What's necessary to make Mill good for building Java ecosystem projects?

[lihaoyi]

One of my mid-term goals is to try and make Mill an attractive environment for building Java projects. Most Java projects are on Maven or Gradle (IntelliJ Developer Survey), so people are already writing Groovy or XML to configure their builds, and getting them to learn Scala doesn't seem like such a stretch (despite Scala's reputation for confusingness)

Scala and Java are close enough that a bunch of the stuff already translates over: dependencies, classpaths, jars, assemblies, etc..
We already have decent support for Java as a language, including a dedicated section in the documentation (https://mill-build.org/mill/0.11.11/Java_Intro_to_Mill.html). If we like Mill and think it's intuitive, then it would be great if other JVM language folks could benefit from using Mill to build their projects

But Java as an ecosystem does things differently than projects using Scala, and so there is probably more work to be done to make using Mill an attractive option. The question is, what?

Some ideas off the top of my head:

1. First class support for big Java frameworks:

   • Testing frameworks (JUnit, Mockito, Powermock, TestNG)
   • Web frameworks (Spring Boot, Spring MVC, Quarkus, Micronaut)
   • Linters (Checkstyle, Errorprone, Findbugs)
   • Autoformatters (Google Java Format, Spotless)
   • Code Coverage (Jacoco)

2. Support BOMs and dependency management (Support BOMs/dependencyManagement coursier/coursier#1390)

3. Work to get rid of weird DSLs and Scala-isms (Make T an alias for Task instead of Target, move T.* operations to Task.* #3356)

4. Support Kotlin? https://github.com/lefou/mill-kotlin already exists, but we could in-source it and provide the same fleshed out documentation we now provide for Java and Scala

5. Port SBT-Pom-Reader to Mill, to simplify the migration story https://github.com/sbt/sbt-pom-reader

6. General quality-of-life improvements: better IDE support, better CLI, better performance, etc.. Not necessarily specific to Java, but would benefit anyone using Mill

What else do we need to make Java folks feel at home using Mill? Please let me know what you think below!

[Krever]

I’m not sure how popular that is, but at some point when using Java ecosystem, I was almost forced to use "annotation processors". It was the first and only time I've seen this, and quick research indicated that there is no easy way to use this in sbt. If it's a popular mechanism in java, it might be something to add to mill. It would also benefit scala builds.

https://picocli.info/#_annotation_processor

[lefou]

We have an example for using an annotation processor in the documentation:

• Annotation Processors with Lombok

[Krever]

Ah, didn't know, thanks and sorry for noise.

[lefou]

Ah, didn't know, thanks and sorry for noise.

It's not noise. Any feedback helps. Thank you!

[nafg]

I'm curious what is the impetus behind trying to target Java devs?

[roman-mibex-2]

• A bigger audience
• There are certainly tons of non-standard (aka don't fit well into Maven defaults) suffering from clunky builds.

[nafg]

My question wasn't why might Java developers want Mill, it was why does Mill want Java developers

[lihaoyi]

For the same reason we want Scala developers: because its cool, and we would like to share its coolness to benefit as many people as possible

[lefou]

I wonder whether there are any Eclispe IDE users? If not for Scala or Kotlin, Eclipse IDE used to be a nice performer, so the question is, whether we might want to add a generator for some rudimentary Eclipse IDE project files (.classpath and .project and some files under .settings/).

[thahnen]

I was just about to open an issue (asking for this feature). We are heavily using Eclipse currently with a mixture of Ant and other scripts as our main build system for our huge legacy Java + GWT project, and I was thinking about at least prototyping the move to Mill because it sounds very promising. We tried Gradle, but the performance improvements were only minimal.

Since the Scala and Kotlin plug-ins for Eclipse are strictly speaking dead or ill-maintained, the only real purpose of such a generator would be to generate Eclipse JDT projects.
I've been circling Mill now for quite some time and try to devour every talk or other resource about it I can get my hands on, thinking about building that generator myself. So far, only two things are preventing me from doing so:

• never used Scala, I'm familiar with both Java and Kotlin very well, but this would take a lot more to learn, I fear (I have the gut feeling the learning curve for Scala is quite unforgiving when not familiar with its concepts)
• looking through the codebase for how the IntelliJ IDEA XML Support is implemented, the in-code documentation is lacking a bit IMO, but this may only be since I'm neither familiar with Scala nor with the codebase

Eclipse project files to generate should be straightforward for the "simple" cases, and only the edge cases might be a bit trickier.

• Mill modules will be separate, independent Eclipse projects linked together, I guess.
• Classpath dependencies must be somehow converted to absolute paths for Eclipse .classpath files (in theory, that can maybe be solved easier as well).

The resources I gathered so far, to plug in a Eclipse IDE Project File Support (name pending 😆) are as follows:

• Documentation
• https://github.com/com-lihaoyi/mill/blob/1.0.1/runner/daemon/src/mill/daemon/MillMain0.scala#L409-L420
• https://github.com/com-lihaoyi/mill/blob/1.0.1/libs/scalalib/src/mill/scalalib/exports.scala#L15 (not sure about that tho since it should be only for supporting Java projects at first)
• https://github.com/com-lihaoyi/mill/tree/1.0.1/libs/javalib/src/mill/javalib/idea
• https://github.com/com-lihaoyi/mill/tree/1.0.1/runner/idea
• https://github.com/com-lihaoyi/mill/tree/1.0.1/core/api/daemon/src/mill/api/daemon/internal/idea
• https://github.com/com-lihaoyi/mill/tree/1.0.1/integration/ide/gen-idea (only tests I know, but good inspiration)
• https://github.com/com-lihaoyi/mill/blob/1.0.1/libs/javalib/src/mill/javalib/JavaModule.scala#L41
• https://github.com/com-lihaoyi/mill/blob/1.0.1/core/api/daemon/src/mill/api/daemon/internal/JavaModuleApi.scala
  I think I might have missed a couple of them.

@lihaoyi, @lefou, may I ask for your opinion on this matter, whether or not this seems to be a feasible thing to do? Feedback of any kind (and maybe further links I might have missed) is more than welcome.
I think contributing this would be very beneficial not only to me (or us, speaking from our company's perspective) but maybe also for other Eclipse-based users?

Best,
Tobias

[lihaoyi]

@thahnen thanks for reaching out! What you proposed is definitely doable. I assume that Eclipse has some file-based project config that we can generate similar to IntelliJ, and I assume Eclipse will want to know all the same pieces of information that IntelliJ does: source folders, module dependencies, third-party dependencies, etc.

In that case, it should be straightforward to copy-paste the mill.idea/ implementation we use for IntelliJ, and change it from generating IntelliJ's XML files to generating Eclipse's project config files somehow. I'm not familiar with the Eclipse configuration format, but if you already are configuring Eclipse by generating project files, it should be easy enough to look at the format of those files and make Mill generate a similar format, similar to how we reverse-engineered the IntelliJ .idea/ folder contents just by looking at the .idea/ folder generated by tools like Maven/Gradle/SBT and making sure we generated the same files with the same contents

[thahnen]

Hey @lihaoyi,

thanks for your reply and your recommendations. I took the weekend to create a rudimentary version of an Eclipse file generator (see #5614).
I will finalize in the coming days with some possible improvements and optimizations that can be done separately.

The challenge was the in fact that for Eclipse there will be multiple "projects" for a whole Mill build compared to IntelliJ and that these projects link to each other in the workspace. Testing mostly against the Java projects inside the example folder it works already for a good chunk of cases. Most of them not working are actually corner cases or require some more logic to work in the Eclipse project / workspace context.

[lefou]

Maybe, if we write a nice tutorial how to publish to Maven Central, from account creation at Sonatype, over generating a token up to setting it up in Mill and maybe automation via GitHub, we may cover a very pressing topic.

These questions were asked in all JVM communities and having a single link to up-to-date documentation could be a game changer. It's not easy in any build tool, but besides the typical network issues and unavailability of Sonatype services (which we should mention with example how to identify) it works rather nice in Mill. The workflows are much better that e.g. in Maven, where you can find whole plugins just to manage editing pom.xml files and git-commits to get a release out.

[yawaramin]

Yes. What this discussion shows is that Mill is trying to do DevRel and reach a wider audience. Something I heard recently is that the essence of DevRel is that you ask what problems developers have (eg 'I need to figure out how to publish to Sonatype') and try to solve them.

So the key is to ask what problems Java developers have (that can be solved by Mill).

[lostiniceland]

As a Java, Kotlin developer who just found out about Mill I am quite interested in this topic.

In order to validate, I am currently migrating a moderately complex Kotlin Spring-Boot application (build as a modulith) from Gradle to Mill. I've currently spent around a day to learn the concepts and setup basic modules. So far, compilation works (also thanks to the two weeks old OpenApi plugin).
Now the tricky parts need to be investigated:

• how to migrate test-fixture
• how modify Kotlin classes to be used with Hibernate (Gradles jpa-plugin, which opens up the classes and creates a no-arg constructor)
• how to easily generate the SpringBoot app and run SpringBoot in dev mode (this plugin seems not to work anymore)
• etc pp

In general for enterprisy Java projects, at least from clients I'worked with, the following stack would be great

• Groovy support (mainly due to the awesome Spock test-framework)
• Spock support (Groovy should be enough, but maybe having a TestModule.Spock which automatically configures Groovy...)
• Annotations-Processors, mostly due to Mapstruct (Lombok is also often used though I am not a fan personally). This seems to be already working.
• SpringBoot
• Quarkus - probably tricky due to a lot of compiler magic by extensions (maybe best if done by Quarkus team).
• OpenApi. Seems to work, need to investigate more.
• SBOM
• Update dependencies by Renovate (most companies use it so far)

I would be interested in contributing (in order to code some Scala again), but I have to get more familiar with Mill first.

[lihaoyi]

If you could open separate issues explaining what is missing for kotlin-hibernate/gradle-jpa, and with spring boot dev mode, that would help us figure out what we need to support in Mill

[lefou]

I think for the kotlin-hibernate issue, all you need is to use the kotlin-allopen-compiler-plugin and configure it. It has even a preset for spring. Read more at https://kotlinlang.org/docs/all-open-plugin.html#command-line-compiler

  override def kotlincPluginMvnDeps = super.kotlincPluginMvnDeps() ++ Seq(
    mvn"org.jetbrains.kotlin:kotlin-allopen-compiler-plugin:${kotlinVersion()}"
  )
  override def kotlincOptions = super.kotlincOptions() ++ Seq(
    "-P",
    "plugin:org.jetbrains.kotlin.allopen:preset=spring"
  )

Regarding the Spring Boot support. Mill comes with a RepackageModule which uses the spring-boot-tools to create a JAR assembly (essentially what mvn spring-boot:repackage does). Read more at https://mill-build.org/mill/javalib/publishing.html#_building_executable_assemblies_with_repackagemodule

What do you mean by running spring boot apps in dev-mode exactly? Do you need more debug output or do you mean adding the spring-boot-devtools dependency to the project?

[lostiniceland]

Thanks for your responses. As said, I am still in active evaluation and haven't been able to migrate everything yet. Your suggestions already help me a great deal.

If you are interested and in case I succeed, I could provide a documentation PR which focuses a bit more on an actual framework stack to get started. Currently I have to hunt down quite some pieces.

Regarding the Groovy module: since I would need it for another project where all tests are written with Spock, I will have a look and try to come up with a PR.

[lostiniceland]

I have been able to create mill.groovylib PoC which is able to compile the Groovy (Spock) sources. Though this is just a rip of the Kotlin implementation it was still quite easy to pick up (which speaks for Mills design). Whats currently not working is that the compiled tests get executed. I will open another discussion for this after I cleanup up and commited it to my fork.

[lefou]

Fell free to open a draft PR to get early feedback.