Offline build

[fpesari]

Hello,

I am a packager from the GeekosDAW project on openSUSE GNU/Linux and I'd like to make a package for mugene-ng, so that openSUSE users can easily discover and install it from their package manager. To build and publish packages on openSUSE we use the Open Build Server (OBS), which offers offline virtualized environments. For this reason, it's very important that programs can be built without an Internet connection and, if there are any dependencies, they are either vendored with the programs or packaged separately by us (ktmidi is a useful library so I can probably package it separately).

Premise: I am no gradle expert even if I've built Java programs in the past (also with Maven). I tried to follow the official build instructions in the README but the build process required an Internet connection. I tried to use gradle -offline build with the tasks you indicated in the README but that did not work. Could you please help me build this project without an Internet connection, so that I can make a package for it on openSUSE? We really need a well-written, still-maintained MML-to-MIDI translator in our repository. Thank you! 🙇

[atsushieno]

Hi, thanks for your interest in the project and offering packaging, that's music to my ears! Let me make it clear first, mugene-ng and ktmidi are typical (well, to some extent) Kotlin application and library, and I believe it does depend on remote packages. TBH I am not familiar with offline builds and if it has been viable for a Kotlin project like this.

The first and biggest thing that comes to my mind is, there should be Kotlin compiler toolchains, Gradle, and so on that should also be available offline (I guess?), as well as its standard libraries (kotlin-stdlib, coroutines, and so on), which would need a thorough set of the framework ecosystem. I was originally in the mono team, which you would be able to find that it consists of a lot of library packages. Providing the entire packages had (has) been a massive task. And the same would apply to Kotlin. So unless they are already packaged, I assume you are going to have to work on a far beyond packaging just one app.

All those dependencies in the project are listed in dependencies section of build.gradle, which are Maven packages. It would be still possible if you have sources for those maven dependency packages then you can build them and "publish" to mavenLocal repository (which is under $HOME/.m2/repository. Then ./gradlew -offline build would work (I haven't actually verified, but it should be in theory).

On how to deal with Maven package equivalents, some existing knowledge base on those existing Java packages would help. For Kotlin toolchains, maybe you need some help from distros (I'm assuming it is rather a distro-specific issue).

[atsushieno]

I was just curious and tried on Ubuntu 20.04, and found that it seems kotlin is provided as a snap package. Maybe that does not count for your requirement I guess.

[fpesari]

Hi, thanks for your interest in the project and offering packaging, that's music to my ears!

It's me who thanks you, because you finally released a MML toolkit comprehensible by people who can't read kana and kanji which has clearly some robustness to it and a proper FLOSS license! 😃 I think it's time that the Japanese audio ecosystem is better represented on mainstream GNU/Linux distributions.

All those dependencies in the project are listed in dependencies section of build.gradle, which are Maven packages. It would be still possible if you have sources for those maven dependency packages then you can build them and "publish" to mavenLocal repository (which is under $HOME/.m2/repository. Then ./gradlew -offline build would work (I haven't actually verified, but it should be in theory).

This is actually the info I was looking for, thanks. I think there is a lot of work to do in order to finish the build on openSUSE. I ask you to kindly keep this ticket open so that I can share any obstacles to a fully offline build.

Meanwhile, are the NPM and Android dependencies mandatory on GNU/Linux or will mugene-ng build without them?

[atsushieno]

Well, yeah I agree that their FLOSS incompatibility has been annoying for me too, which was one of the reasons why I ended up to create one :-)

Meanwhile, are the NPM and Android dependencies mandatory on GNU/Linux or will mugene-ng build without them?

I think you can avoid JS/NPM and Android (I'm not sure how Kotlin Gradle plugin works and thus unsure about the build consequences) - I usually just run build when building everything (unless I directly debug app on IntelliJ IDEA), but that can be publishJvmPublicationToMavenLocal, publishLinuxX64PublicationToMavenLocal in mugene-project directory, and then mugene-console:build and optionally mugene-console-jvm:build in mugene-console-project directory.

I haven't created anything further beyond just mugene-console/build/bin/native/{debug|release}Executable/mugene-console.kexe (standalone executable) or mugene-console-jvm/build/libs/mugene-console-jvm-0.1.0-SNAPSHOT.jar (jvm jar). If you need some shell script then they would have to be created manually (Kotlin Gradle plugin might have some easing features, but I have no right idea). If you want to add some files in this repo, please create a PR.

[atsushieno]

One thing I noticed w.r.t. package resolution is that build.gradle in any library that depends on packages in mavenLocal will have to add explicit declaration like repositories { mavenLocal() }, which usually does not happen in the sources (i.e. local patches would be needed). My libraries, including ktmidi, has mavenLocal() specification, but dependencies like antlr-kotlin would not. Just FYI.