ClojureCLR build tooling

[dmiller]

With the advent of cljr, ClojureCLR now has some of the goodness of the CLI/deps.edn tooling available to Clojure(JVM).

Some functionality is still missing, centered around tool definition and invocation and project preparation. In the Prep libs: section of the CLI guide, we find:

Source libs with Clojure source can immediately be added to the classpath of a project using it. However, some source libs require some preparation before they can be added, for example due to needing Java compilation, or copying / replacing resource files, etc.

The library tools.build provides functionality for building Clojure projects. It is very JVM-specific, so I have not bothered to port it. We need our own.

Questions to contemplate:

• What would be in a tools.build for ClojureCLR?
• How do we add :nuget coordinates to cljr? Being able to retrieve packages, dealing with versioning, etc., is straightforward enough, but there are other issues:
  • How to deal with multiple frameworks/runtimes?
  • How best to package pure Clojure code projects? (The current method is clunky. Is there a better way?)
• How do we specify the version of ClojureCLR in a deps.edn file? (This can be done for Clojure. However, the differences between Java classpaths and what we have to do with assemblies in ClojureCLR prevents a similar approach -- unless perhaps we solve all the other problems here.)
• Are there better ways for ClojureCLR to reference assemblies?

Let's go!

(Note: this discussion is hosted on the clojure-clr-next repo because I own it and can do things such as set up discussions. However, this is really a discussion about ClojureCLR as it is now.)

[brandoncorrea]

What would be in a tools.build for ClojureCLR?

tools.build seems to capture discrete steps of the Java build process. It would be neat if the CLR version did the same.

Maybe it would include some of these steps:

• Create .sln and .csproj files (probably .fsproj and .vbproj as well)
• Compile .NET code to something like /bin or /out (or any user-specified output directory)
• Packaging...
  • The JVM has jars. Does .NET have such a thing? In my experience, it has always been a hundred files in a directory with a main.exe nested somewhere within.

It would be nice to have some sort of handling for users with mixed sources (clj + fs + cs). How does the JVM version handle mixed languages?

[dmiller]

A lot of functionality can be offloaded to the dotnet command.
Re packaging: Of course, assemblies serve part of the purpose of packaging. But beyond, the closest thing I know of is nuget packages.
Re mixed languages: Everything compiles to bytecode in .class files and ends up in jars, uberjars, or just the files themselves sitting on the classpath. It's very uniform. And relatively transparent, if you don't mind perusing through jar files to find .class files. In .NET-land, of course we can compile to IL, but identity and access is assembly-based, and that lumpiness is what we need to work around.

[brandoncorrea]

Are there better ways for ClojureCLR to reference assemblies?

How does .NET do this? Does it not make sense to add it to :deps and maybe extend the :mvn/repos section?

[dmiller]

The two requirements are to find ways to specify in deps.edn and then to make sure ClojureCLR itself has the mechanisms to handle it.
One could reference an assembly via a :local for something on the filesystem, or a :nuget coordinate (when implemented). But ClojureCLR does not have an automatic way to pull in assemblies based on that, so that would have to be added.

[brandoncorrea]

I think that makes sense then, :nuget for deployed packages and :local for assemblies & projects. That's how .*proj references seem to work (to my knowledge).

[brandoncorrea]

How do we specify the version of ClojureCLR in a deps.edn file?

Java has {:jvm-opts ["--release 11"]}. Why not something like {:clr-opts ["..."]}? Unless these are too different to be as closely related as they look.

[dmiller]

Rather than the version of the CLR, I was thinking of the version of ClojureCLR itself. On the Clojure(JVM) side, you can find examples such as:

{:paths ["src/main/clojure"]
 :deps {org.clojure/clojure {:mvn/version "1.9.0"}}}

That works to bring down the 1.9 version of Clojure. Once that is on the classpath, that's what you are going to pick up when you run.
You run java, specify that clojure.main or whatever is to be started, and it is found on the classpath.

A bit more problematic on the CLR side. Currently the ClojureCLR runtime installs as a tool. Likely we could install as a local tool to get versioning, but that definitely would not be part of the standard :deps processing. Or we bring down the assembly using a :nuget coordinate, and use dotnet run as our startup rather than invoking the tool. Or something else. We don't have a direct equivalent of java clojure.main (or whatever the proper invocation is.

[brandoncorrea]

It seems as though the JVM side brings down the correct Clojure version from Maven. I think it would make sense that ClojureCLR do the same with Nuget.

If I'm not mistaken, the Clojure JVM clj command is a tool, but different from the Clojure version specified in deps.edn, allowing the project to run a different version of Clojure from that of the clj command.

This would probably affect how the cljr command works though, and maybe some other existing tools? Any reason not to take the same approach as JVM?

In this scenario, I wonder if cljr could be so decoupled that it could be written on the JVM, if so desired (though I imagine .NET is our friend here).

[brandoncorrea]

How do we add :nuget coordinates to cljr? Being able to retrieve packages, dealing with versioning, etc., is straightforward enough, but there are other issues:

I believe :mvn/version fetches packages from either Maven or Clojars (and anywhere else you specify in :mvn/repos). Would CLR's version (:nuget/version & :nuget/repos maybe?) do something similar by fetching from either NuGet or Clojars?

How to deal with multiple frameworks/runtimes?

Whose responsibility is this? Clojure JVM has supported all versions of Java from 1.8 up until recently. Clojure CLR I think should certainly do something similar, but as far as packages go, wouldn't that be up to the maintainer who they want to support?

How best to package pure Clojure code projects? (The current method is clunky. Is there a better way?)

Clojars? Or do we want to deploy these to nuget as well?

[brandoncorrea]

On :nuget/repos, I'm pretty sure private nuget repos are a thing. So it may be worth considering. Maybe very little need though, assuming packages can be pulled from private git repos (with proper SSH keys).

[dmiller]

I'm thinking NuGet packages to bring down non-Clojure dependencies -- the library written in C# that you need for your project.
The problem is how to do loading of those assemblies. I found code to do pulls, manage version numbering, etc., from Nuget. I could port that for use in the CLI / deps.edn. Bringing down the bytes is the easy part. I'm not sure how to deal with platform versioning, other files in the download, etc.