Kotlin Multiplatform

[serpro69]

Discussion around adding mutiplatform support to kotlin-faker, which is done in #226

[tamimattafi]

@serpro69 Have you ever tried building the project for macOS aarch64?
I'm getting this error while syncing the project with JDK 11 or 17

Build file '/Users/tamimattafi/Projects/kmm/kotlin-faker/cli-bot/build.gradle.kts' line: 64

Failed to calculate the value of task ':cli-bot:compileJava' property 'javaCompiler'.
> No matching toolchains found for requested specification: {languageVersion=8, vendor=matching('Temurin'), implementation=vendor-specific} for MAC_OS on aarch64.
   > No locally installed toolchains match and toolchain download repositories have not been configured.

Line 64 of cli-bot/build.gradle.kts is

"Class-Path" to project.configurations.compileClasspath.get().joinToString(" ") { it.name },

Commenting this line doesn't hide the issue, it's still present but with line 143 (The end of the file)

Changing IDE's JDK to 1.8 throws another error

A problem occurred configuring project ':cli-bot'.
> Could not resolve all files for configuration ':cli-bot:classpath'.
   > Could not resolve org.graalvm.buildtools:native-gradle-plugin:0.10.1.
     Required by:
         project :cli-bot > org.graalvm.buildtools.native:org.graalvm.buildtools.native.gradle.plugin:0.10.1
      > No matching variant of org.graalvm.buildtools:native-gradle-plugin:0.10.1 was found. The consumer was configured to find a library for use during runtime, compatible with Java 8, packaged as a jar, and its dependencies declared externally, as well as attribute 'org.gradle.plugin.api-version' with value '8.4' but:
          - Variant 'apiElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a component for use during compile-time, compatible with Java 11 and the consumer needed a component for use during runtime, compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')
          - Variant 'javadocElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a component for use during runtime, and its dependencies declared externally:
              - Incompatible because this component declares documentation and the consumer needed a library
              - Other compatible attributes:
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')
          - Variant 'runtimeElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a library for use during runtime, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a component, compatible with Java 11 and the consumer needed a component, compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')
          - Variant 'sourcesElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a component for use during runtime, and its dependencies declared externally:
              - Incompatible because this component declares documentation and the consumer needed a library
              - Other compatible attributes:
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')

Searching for a solution.

[serpro69]

Here's a git diff for excluding the cli module altogether. I've just tested it myself and I can build everything else with just jdk8.

diff --git a/bom/build.gradle.kts b/bom/build.gradle.kts
index cde7bffe..086d5c12 100644
--- a/bom/build.gradle.kts
+++ b/bom/build.gradle.kts
@@ -26,7 +26,7 @@ val isRelease = provider {
 
 // Exclude subprojects that will never be published so that when configuring this project
 // we don't force their configuration and do unnecessary work
-val excludeFromBom = listOf(":cli-bot", ":docs", ":extension", ":faker", ":test", ":extension:kotest-property-test")
+val excludeFromBom = listOf(/*":cli-bot", */":docs", ":extension", ":faker", ":test", ":extension:kotest-property-test")
 fun projectsFilter(candidateProject: Project) =
     excludeFromBom.none { candidateProject.path == it }
         && candidateProject.name != bom.name
diff --git a/build.gradle.kts b/build.gradle.kts
index 07e444ba..52e20c3d 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -172,5 +172,5 @@ tasks.withType<TagTask>().configureEach {
 }
 
 apiValidation {
-    ignoredProjects += listOf("bom", "cli-bot", "docs", "test")
+    ignoredProjects += listOf("bom", /*"cli-bot", */"docs", "test")
 }
diff --git a/settings.gradle.kts b/settings.gradle.kts
index e8b4fb2b..0b8c676d 100644
--- a/settings.gradle.kts
+++ b/settings.gradle.kts
@@ -15,7 +15,7 @@ rootProject.name = "kotlin-faker"
 include(
     ":bom",
     ":core",
-    ":cli-bot",
+    // ":cli-bot",
     ":docs",
 )
 

Try if this works for you, and if so, then I think we can just keep the cli ignored for now.

[tamimattafi]

@serpro69
Yes, I tried to compile from master without any multiplatform setup. I tried to exclude the cli-bot from settings.gradle.kts but it seems like that wasn't enough, I will try it with your instruction.

As for JDK versions, I will take a look on the prerequisites.
I will keep this thread updated.

[serpro69]

In short, the problem with cli module is that it uses a graal gradle plugin that seems to require jdk version >= 11 (there's no mention of that in the docs that I could see, but I couldn't get it to work with java8)
Graal-CE distribution itself starts from version 17. But the gradle process needs to be running with java 11 because that's maximum supported version for a plugin in the docs module. So it's a small mess because of these hard dependencies on different java versions....

I'm now starting to look into refactoring docs and getting rid of the old unsupported framework that is currently used to build documentation. But that will only solve half of the problem and CLI would still need to be tackled sooner or later. I'm not sure yet what to do with it, so I'll have to take a look into it once I have some time. But since it's not part of the library itself, and isn't that critical, I think it should be OK to just ignore it for now so it doesn't become a blocker. If need be, I can just keep compiling it locally for awhile, in worst case.

What version of java do you have locally btw? And what vendor? I think this could also be causing some issues if you have a different vendor locally:

kotlin-faker/cli-bot/build.gradle.kts

Line 54 in d9cd70a

vendor.set(JvmVendorSpec.matching("Temurin"))

(I've just removed this part, so it should't be tied to a specific vendor anymore. Have no idea why I added that... and seems like that's the only place where vendor was specified explicitly)

But anyways, just try to exclude cli completely, and if that works - let's keep it excluded for now.

[tamimattafi]

@serpro69 Having JDK installed as mentioned in the prerequisites fixed the problem, and even tho there's an issue on my machine with graalvm-17, the sync completed successfully. Thank you for the tips.

I will study the build config and start adding the multiplatform plugin this week, hopefully on the weekend I will have something to show.

[serpro69]

Awesome! Feel free to post more questions here if/when you need further help.

[tamimattafi]

@serpro69 Hello!
Unfortunately, I didn't make any significant progress this week due to tight time at work. However, I studied the build config and made some observations, which might be important for further steps.

1. The current config uses maven-publish to upload artifacts to mavenCentral, however, this might not work seamlessly for Kotlin Multiplatform projects. I suggest using com.vanniktech.maven.publish plugin since it handles all artifact types. Here's an example of how to set it up: here
2. It might be worth it to move from buildSrc to includeBuild and break plugins into separate modules, for example: here. This will make build logic cleaner and more flexible
3. Not sure about the use-case differences between faker-lib and faker-ext conventions, what is each of them used for?
4. What is faker-provider conventions used for?

The first step will require implementing kotlin multiplatform plugin with basic targets, and move all the current logic from the main source set to jvmMain, and test to jvmTest (Basically rename the directories). This will lay the foundation for creating new source sets such as commonMain and move shared logic eventually. However, this should be done and merged quickly to avoid potential conflicts.

[tamimattafi]

a bit busy at work at the moment :D

I couldn't relate more 🥲

[serpro69]

Hey @tamimattafi 👋 ,
...I've been away from this project for a little 🙈 while and only had time for maintenance and small improvements the past year or so, unfortunately. But I spent some time today on this and I think I managed to convert the build configuration to use kotlin multiplatform plugin (as in, tests pass fine via ./gradlew clean jvmTest; need to check other stuff also like publishing and so on).
I had to make some semi-drastic changes, particularly removing 'shadow' plugin completely (I can't get it to work yet, and atm can't be bothered to do so either 😁 ), and I've also merged integration tests with unit tests to simplify the whole gradle code.
There's a draft PR here if you'd like to take a look: #278

[tamimattafi]

Hey @serpro69!
Great news! I took a quick look, everything looks good to me.

As for publishing, that's normal that a separate component will be created for each sourceSet/platform, for example, module for common, module-jvm for JVM, module-mingwx64 and so on. I believe the Kotlin team is making (Or already made) good progress with klib to generate fewer artifacts for publication, but I haven't checked it in a long time.
I suggest using a publishing plugin, such as com.vanniktech.maven.publish, to avoid some headaches. Here's an example of a convention plugin.

As for the Gradle scripts, everything looks neat. I can suggest doing a minor change to make code more concise, but it's more of a taste than a rule, and it works for some source sets only, not all:

kotlin {
     sourceSets {
          // instead of this
          val androidMain by getting {
              dependencies {
                   // add something here
              }
          }
          
          // you can do this
          androidMain.dependencies {
              // add something here
          }
     }
}

Great work!
I hope you will soon be able to support more platforms in addition to JVM!

[serpro69]

Just a small update: had a bit more time to spend on this, and I think I mostly fixed the publishing logic. Want to test it a bit more and clean up, but at least all artifacts pass validation in central now.

[serpro69]

If you want to test, I've now published the snapshot versions of the artifacts.

repositories {
    mavenCentral()
    maven {
        url = URI("https://central.sonatype.com/repository/maven-snapshots/")
    }
}

dependencies {
    testImplementation(kotlin("test"))
    testImplementation(platform("io.github.serpro69:kotlin-faker-bom:2.1.0-SNAPSHOT"))
    testImplementation("io.github.serpro69:kotlin-faker-jvm") // can also use dependency w/o -jvm qualifier
    testImplementation("io.github.serpro69:kotlin-faker-ext-blns-jvm")
    testImplementation("io.github.serpro69:kotlin-faker-books-jvm")
}