From 1c84aed172ba46bce81cc5cce2bc15995f0fb934 Mon Sep 17 00:00:00 2001 From: Alexander Sysoev Date: Wed, 23 Jul 2025 17:18:17 +0200 Subject: [PATCH 1/3] Add a doc for KMP source sets with gRPC --- .../topics/grpc-configuration.topic | 491 ++++++++++++++---- 1 file changed, 392 insertions(+), 99 deletions(-) diff --git a/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic b/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic index aba2cfd52..a8f000560 100644 --- a/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic +++ b/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic @@ -1,4 +1,8 @@ + +

- Artifacts for gRPC integration are published separately + Artifacts for gRPC integration are published separately and updated frequently, independent of the main releases.

@@ -39,137 +44,425 @@ build.gradle.kts:

- plugins { - kotlin("jvm") version "%kotlin-version%" - kotlin("plugin.serialization") version "%kotlin-version%" - id("org.jetbrains.kotlinx.rpc.plugin") version "<version>" - id("com.google.protobuf") version "0.9.4" - } - - repositories { - mavenCentral() - maven("https://maven.pkg.jetbrains.space/public/p/krpc/grpc") - } - - dependencies { - implementation("org.jetbrains.kotlinx:kotlinx-rpc-grpc-core:<version>") - implementation("ch.qos.logback:logback-classic:1.5.16") - implementation("io.grpc:grpc-netty:1.69.0") - } + plugins { + kotlin("jvm") version "%kotlin-version%" + id("org.jetbrains.kotlinx.rpc.plugin") version "<version>" + } + + repositories { + mavenCentral() + maven("https://maven.pkg.jetbrains.space/public/p/krpc/grpc") + } + + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-rpc-grpc-core:<version>") + implementation("ch.qos.logback:logback-classic:1.5.16") + implementation("io.grpc:grpc-netty:1.73.0") + }

Here <version> comes from the badge above.

- - The setup has only been tested on Kotlin/JVM projects. - + + We prohibit using com.google.protobuf Gradle Plugin + in projects with our plugin to prevent conflicts. + - -

- gRPC requires additional code generation from the protoc - compiler. - It is set up automatically for you when the com.google.protobuf - plugin is present in the project. -

+ + + Working with <code>.proto</code> files +

- We provide additional options for configuration: + The minimum required configuration looks like this:

rpc { - grpc { - // Enforce additional checks on the project configuration - enabled = true - - // Quick access to a `Locator` and `Options` - // for the kotlinx-rpc Protobuf plugin - plugin { - options { - // Add or modify options - option("debugOutput=myFile.txt") - } - - locator { - // Override artifact coordinates - artifact = "some-other:artifact:version" - } - } - - // same as `plugin`, but for gRPC Java generation - grpcJavaPlugin { ... } - // same as `plugin`, but for gRPC Kotlin generation - grpcKotlinPlugin { ... } - - // access `generateProto` tasks - tasks { - plugins { - create("python") - } - } - - // access `generateProto` tasks with a filter - tasksMatching { it.isTest }.all { - plugins { - create("cpp") - } - } - } + grpc() }

- You can still use protobuf extension to access the configuration. - The following is the equivalent for the above code using the protobuf extension: + This will enable the code generation for your .proto files. + We create special source sets for .proto files: +

+ +
  • + main and test - source sets for Kotlin/JVM projects. + Default source directories are src/main/proto and src/test/proto. +
    • +
  • + jvmMain and jvmTest - source sets for Kotlin/Multiplatform projects. + Default source directories are src/jsmMain/proto and src/jvmTest/proto. + + This will change to src/commonMain/proto and src/commonTest/proto in the + future. + + + Source set hierarchy is not supported, and we have no plans to support it. + That means, for example, + that you can't have jsMain and jsTest source sets for proto files. + Same for native and WASM. + Proto files can only be in the jvmMain and jvmTest source sets. + (Later to be replaced with commonMain and commonTest) +
    +
    + If you have a use case for other source sets and a hierarchy, + please report it. +     +
    • +
  • + Android source sets are not supported yet. WIP +
    • +     +

    + All source sets are generated automatically and available + via the protoSourceSets Gradle Project extension:

    - protobuf { - plugins { - named(GrpcExtension.LOCATOR_NAME) { - artifact = "some-other:artifact:version" + protoSourceSets { + // for Kotlin/JVM projects + main { + proto { + // SourceDirectorySet for .proto files + include("**/main.proto") // optional filters + exclude("**/excluded.proto") } - - named(GrpcExtension.GRPC_JAVA_LOCATOR_NAME) { ... } - named(GrpcExtension.GRPC_KOTLIN_LOCATOR_NAME) { ... } } + test { } - generateProtoTasks { - all().all { - plugins { - named(GrpcExtension.LOCATOR_NAME) { - option("debugOutput=myFile.txt") - } - - create("python") + // for Kotlin/Multiplatform projects + jvmMain { } + jvmTest { } + } + +

    + By default, four source directories will be generated: +

    + +
  • + java - protobuf Java declarations, attached to java sources +
    • +
  • + grpc-java - gRPC Java declarations, attached to java sources +
    • +
  • + grpc-kotlin - gRPC Kotlin wrappers for Java, attached to kotlin sources +
    • +
  • + kotlinx-multiplatform - our wrappers for all of the above, attached to kotlin sources +
    • +     + + Only the declarations from the kotlinx-multiplatform source set are intended to be used. + The other ones are for internal use only and will be removed in the future. + +     + +

    + To generate code, we need to use protoc plugins. + By default, we use the following plugins: +

    + +
  • + protobuf-java - Buf's version of the official Java + plugin. +
    • +
  • + grpc-java - Buf's version of the official gRPC Java plugin. +
    • +
  • + grpc-kotlin - Buf's version of the official gRPC Kotlin + plugin. +
    • +
  • + kotlinx-multiplatform - out own protoc plugin. +
    • +     +

    + You can configure the plugins in the rpc block: +

    + + rpc { + grpc { + // configure plugins here + protocPlugins { + kotlinMultiplatform { } + protobufJava { } + grpcJava { } + grpcKotlin { } - if (isTest) { - create("cpp") - } - } + create("myPlugin") { } } } }

    - The minimum recommended configuration looks like this: + protocPlugins is a NamedDomainObjectContainer<ProtocPlugin>, + which allows to add your own plugins. +

    +

    + Every plugin is a ProtocPlugin class, that can be configured like this:

    - rpc { - grpc { - enabled = true + // or like protocPlugins.kotlinMultiplatform for default plugins + protocPlugins.create("myPlugin") { + isJava = true // add generated code to the java source set + + // add plugin options + options.put("myOption") + options.put("myOption1", "myValue") + + // Plugin can be local or remote + + // For a local plugin + local { + // Path to the plugin executable + executor("java", "-jar", "myPlugin.jar") + // or + javaJar("myPlugin.jar") // uses java.home + // or + javaJar("myPlugin.jar", "path/to/my/java") + // or + executor("myCustomExe.exe") + } + + // For a remote plugin + remote { + // Buf's BSR locator + locator = "buf.build/grpc/go" } + + // other Buf configuration options: + strategy = ProtocPlugin.Strategy.All + includeImports = false + includeWkt = false + types = listOf("my.package.MyType") + excludeTypes = listOf("my.package.NotMyType") }

    - By default, four source sets will be generated: + Buf's plugin configuration + options:

    -
  • java - protobuf Java declarations
    • -
  • grpc - gRPC Java declarations
    • -
  • grpckt - gRPC Kotlin wrappers for Java
    • -
  • kotlinx-rpc - our wrappers for all of the above
    • +
  • + strategy + — All or Directory +
    • +
  • + includeImports + — true or false +
    • +
  • + includeWkt + — true or false +
    • +
  • + types + — a list of types to generate. +
    • +
  • + excludeTypes + — a list of types to exclude from generation. +

    • - Only the declarations from the kotlinx-rpc source set are intended to be used. + A plugin, once added to the protocPlugins container, can be used in a source set:

    + + protoSourceSets { + jvmMain { + protocPlugin(rpc.grpc.protocPlugins.named("myPlugin")) + } + } + +     +

    - Source sets are generated into the $BUILD_DIR/generated/source/proto/main directory - unless specified otherwise. + As you may already notice, + we use Buf — + a tool for managing and building Protobuf schemas. + We use its CLI to execute tasks like code generation.

    + + We don't use Buf's build.buf Gradle Plugin + and prohibit using it in projects with our plugin to prevent conflicts. + + +

    + To make UX smooth, we introduce a concept of a generated workspace. + It's a directory that contains all the necessary files for Buf to work: +

    + +
  • + Filtered .proto files +
    • +
  • + Generated buf.yaml file +
    • +
  • + Generated buf.gen.yaml file +
    • +
  • + Automatically managed imports from a main source set to a test source set, + making imports intuitive. +
    • +     +

    + This workspace is created automatically for each source set, + and can be found in the build/protoBuild directory. + You don't need to do anything with it, unless you want to customise it. +

    +     + +

    + We create the following tasks if the project is configured to use grpc: +

    + +
  • + generateBufYaml<sourceSet> + — generates buf.yaml +
    +
    + Type: kotlinx.rpc.buf.tasks.GenerateBufYaml +
    • +
  • + generateBufGenYaml<sourceSet> + — generates buf.gen.yaml +
    +
    + Type: kotlinx.rpc.buf.tasks.GenerateBufGenYaml +
    • +
  • + processProtoFiles<sourceSet> + — copies proto files to the workspace directory +
    +
    + Type: kotlinx.rpc.proto.ProcessProtoFiles +
    • +
  • + processProtoFilesImport<sourceSet> + — copies import proto files to the workspace directory +
    +
    + Type: kotlinx.rpc.proto.ProcessProtoFiles +
    • +
  • + bufGenerate<sourceSet> + — runs buf generate command. +
    +
    + Type: kotlinx.rpc.buf.tasks.BufGenerateTask +
    • +     +

    + To configure Buf, use the buf block: +

    + + rpc { + grpc { + buf { + // configure Buf here + logFormat = BufExtenstion.LogFormat.Text + timeout = 60.seconds + + generate { + // configure Buf generate here + + includeImports = false + includeWkt = false + errorFormat = BufGenerateExtenstion.ErrorFormat.Text + } + } + } + } + +

    + General configuration options: +

    + +
  • + logFormat + — either Text, Color, Json or Default +
    • +
  • + timeout + — timeout for the buf commands, always converted to whole seconds +
    • +
  • + Gradle's --debug enables Buf's --debug option. +
    • +     +

    + buf generate configuration options: +

    + +
  • + includeImports + — true or false +
    • +
  • + includeWkt + — true or false +
    • +
  • + errorFormat + — either Text, Json, Msvs, + Junit, GithubActions or Default +
    • +     +     + +

    + We only support generate tasks for now. + But Buf has great capabilities for managing .proto files, + like linting, detection of breaking changes, etc. + So we made it possible to create custom buf tasks. +

    +

    + To create a custom task, it must be a subclass of the kotlinx.rpc.buf.tasks.BufExecTask. + It doesn't need to define a @TaskAction, just a set of arguments and a command to execute. +

    +

    + Registering the task can be done in two ways: +

    + +
  • +

    A workspace task (recommended):

    + + rpc.grpc.buf.tasks { + val provider = registerWorkspaceTask<MyBufLintTask>("lint") { + // configure task here + } + + // bufLintMain, or bufLintJvmMain for KMP projects + provider.mainTask + + // bufLintTest, or bufLintJvmTest for KMP projects + provider.testTask + } + +

    + These tasks can be executed on a generated workspace for all proto source sets. +

    + + Note that the tasks are not plugged into the build cycle, + so they won't be executed automatically. + +
    • +
  • +

    A regular task:

    + + project.registerBufExecTask<MyBufLintTask>( + name = "bufLint", + workingDir = workingDirProvider, + ) { + // configure task here + } + +

    + It is just a regular Gradle task that has access to the Buf's executable + and some predefined properties. +

    +
    • +     +     From cd9fe94da0e8206ec79d83dec10d57e28c01829c Mon Sep 17 00:00:00 2001 From: Alexander Sysoev Date: Thu, 31 Jul 2025 15:19:36 +0200 Subject: [PATCH 2/3] GH comments --- .../topics/grpc-configuration.topic | 172 +++++++++--------- 1 file changed, 87 insertions(+), 85 deletions(-) diff --git a/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic b/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic index a8f000560..f58fd2f64 100644 --- a/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic +++ b/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic @@ -68,7 +68,7 @@ - Working with <code>.proto</code> files + Working with proto-files

    The minimum required configuration looks like this: @@ -79,36 +79,24 @@ }

    - This will enable the code generation for your .proto files. - We create special source sets for .proto files: + This enables code generation for your .proto files. + Special source sets are created for them:

  • main and test - source sets for Kotlin/JVM projects. - Default source directories are src/main/proto and src/test/proto. + Default source directories are src/main/proto and src/test/proto.
  • jvmMain and jvmTest - source sets for Kotlin/Multiplatform projects. - Default source directories are src/jsmMain/proto and src/jvmTest/proto. + Default source directories are src/jsmMain/proto and src/jvmTest/proto. - This will change to src/commonMain/proto and src/commonTest/proto in the + These will change to src/commonMain/proto and src/commonTest/proto in the future. - - Source set hierarchy is not supported, and we have no plans to support it. - That means, for example, - that you can't have jsMain and jsTest source sets for proto files. - Same for native and WASM. - Proto files can only be in the jvmMain and jvmTest source sets. - (Later to be replaced with commonMain and commonTest) -
    -
    - If you have a use case for other source sets and a hierarchy, - please report it. -
  • - Android source sets are not supported yet. WIP + Android source set support is not available yet but will be included in a future release.

    • @@ -133,30 +121,44 @@ }

    - By default, four source directories will be generated: + By default, the following source directories are generated:

  • - java - protobuf Java declarations, attached to java sources + java - protobuf Java declarations, attached to java sources.
  • - grpc-java - gRPC Java declarations, attached to java sources + grpc-java - gRPC Java declarations, attached to java sources.
  • - grpc-kotlin - gRPC Kotlin wrappers for Java, attached to kotlin sources + grpc-kotlin - gRPC Kotlin wrappers for Java, attached to kotlin sources.
  • - kotlinx-multiplatform - our wrappers for all of the above, attached to kotlin sources + kotlin-multiplatform - wrappers for all of the above, attached to kotlin sources.
    • - Only the declarations from the kotlinx-multiplatform source set are intended to be used. - The other ones are for internal use only and will be removed in the future. + Only the declarations from the kotlin-multiplatform source set are intended to be used. + The others are for internal use only and will be removed in a future release.     + +

    + Source set hierarchy is not supported, and we have no plans to support it. + That means, for example, + that you can't have jsMain and jsTest source sets for proto files. + The same applies for native and Wasm targets. + Currently, .proto files are only supported in the jvmMain and jvmTest source sets. + (Later to be replaced with commonMain and commonTest) +
    +
    + If you have a use case for other source sets and a hierarchy, + please report it. +

    +

    - To generate code, we need to use protoc plugins. + To generate code, we use protoc plugins. By default, we use the following plugins:

    @@ -172,7 +174,7 @@ plugin.
  • - kotlinx-multiplatform - out own protoc plugin. + kotlin-multiplatform - out own protoc plugin.

    • @@ -195,17 +197,17 @@

    protocPlugins is a NamedDomainObjectContainer<ProtocPlugin>, - which allows to add your own plugins. + which allows you to add your own plugins.

    - Every plugin is a ProtocPlugin class, that can be configured like this: + Each plugin is represented by a ProtocPlugin and can be configured like this:

    - // or like protocPlugins.kotlinMultiplatform for default plugins + // Or use protocPlugins.kotlinMultiplatform for default plugins protocPlugins.create("myPlugin") { isJava = true // add generated code to the java source set - // add plugin options + // Add plugin options options.put("myOption") options.put("myOption1", "myValue") @@ -229,7 +231,7 @@ locator = "buf.build/grpc/go" } - // other Buf configuration options: + // Additional Buf configuration options: strategy = ProtocPlugin.Strategy.All includeImports = false includeWkt = false @@ -241,30 +243,30 @@ Buf's plugin configuration options:

    - -
  • + + strategy — All or Directory -
    • -
  • + + includeImports — true or false -
    • -
  • + + includeWkt — true or false -
    • -
  • + + types — a list of types to generate. -
    • -
  • + + excludeTypes — a list of types to exclude from generation. -
    • -     + +

    - A plugin, once added to the protocPlugins container, can be used in a source set: + Once a plugin is added to the protocPlugins container, it can be used in a source set:

    protoSourceSets { @@ -283,22 +285,22 @@

    We don't use Buf's build.buf Gradle Plugin - and prohibit using it in projects with our plugin to prevent conflicts. + and disallow using it in projects that apply our plugin, in order to avoid conflicts.

    - To make UX smooth, we introduce a concept of a generated workspace. + To improve the developer experience, we introduce the concept of a generated workspace. It's a directory that contains all the necessary files for Buf to work:

  • - Filtered .proto files + Filtered .proto files
  • - Generated buf.yaml file + Generated buf.yaml file
  • - Generated buf.gen.yaml file + Generated buf.gen.yaml file
  • Automatically managed imports from a main source set to a test source set, @@ -306,26 +308,26 @@

    • - This workspace is created automatically for each source set, - and can be found in the build/protoBuild directory. + This workspace is created automatically for each source set + and can be found in the build/protoBuild directory. You don't need to do anything with it, unless you want to customise it.

    - We create the following tasks if the project is configured to use grpc: + If your project is configured to use gRPC, the following tasks will be generated:

  • generateBufYaml<sourceSet> - — generates buf.yaml + — generates buf.yaml

    Type: kotlinx.rpc.buf.tasks.GenerateBufYaml
  • generateBufGenYaml<sourceSet> - — generates buf.gen.yaml + — generates buf.gen.yaml

    Type: kotlinx.rpc.buf.tasks.GenerateBufGenYaml @@ -359,12 +361,12 @@ rpc { grpc { buf { - // configure Buf here + // Configure Buf logFormat = BufExtenstion.LogFormat.Text timeout = 60.seconds generate { - // configure Buf generate here + // Configure Buf generate includeImports = false includeWkt = false @@ -377,51 +379,51 @@

    General configuration options:

    - -
  • + + logFormat — either Text, Color, Json or Default -
    • -
  • + + timeout — timeout for the buf commands, always converted to whole seconds -
    • -
  • - Gradle's --debug enables Buf's --debug option. -
    • -     + + + Running Gradle with --debug enables Buf's --debug option as well. + +

    buf generate configuration options:

    - -
  • + + includeImports — true or false -
    • -
  • + + includeWkt — true or false -
    • -
  • + + errorFormat — either Text, Json, Msvs, Junit, GithubActions or Default -
    • -     + +

    - We only support generate tasks for now. - But Buf has great capabilities for managing .proto files, - like linting, detection of breaking changes, etc. - So we made it possible to create custom buf tasks. + Currently, we only support generate tasks. + However, because of Buf's capabilities for managing .proto files, + like linting and detection of breaking changes, + we provide a way to create custom Buf tasks.

    - To create a custom task, it must be a subclass of the kotlinx.rpc.buf.tasks.BufExecTask. - It doesn't need to define a @TaskAction, just a set of arguments and a command to execute. + To create a custom task, extend the kotlinx.rpc.buf.tasks.BufExecTask class. + You don't need to define a @TaskAction; just a set of arguments and a command to execute.

    - Registering the task can be done in two ways: + You can register the task in two ways:

  • @@ -443,8 +445,8 @@ These tasks can be executed on a generated workspace for all proto source sets.

    - Note that the tasks are not plugged into the build cycle, - so they won't be executed automatically. + These tasks are not plugged into the Gradle build cycle, + so you must run them manually
  • @@ -458,7 +460,7 @@ }

    - It is just a regular Gradle task that has access to the Buf's executable + It is a standard Gradle task that has access to the Buf's executable and some predefined properties.

    • From c9951053921b22ce71896eb6197cdd352b044304 Mon Sep 17 00:00:00 2001 From: Alexander Sysoev Date: Thu, 31 Jul 2025 15:40:25 +0200 Subject: [PATCH 3/3] GH comments --- docs/pages/kotlinx-rpc/topics/grpc-configuration.topic | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic b/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic index f58fd2f64..92783b636 100644 --- a/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic +++ b/docs/pages/kotlinx-rpc/topics/grpc-configuration.topic @@ -310,7 +310,7 @@

    This workspace is created automatically for each source set and can be found in the build/protoBuild directory. - You don't need to do anything with it, unless you want to customise it. + You don't need to do anything with it, unless you want to customize it.