Jackson used instead of kotlinx.serialization for more complex types

[Kwasow]

This is possibly an issue with Spring Boot and not the framework, so let me know if I should move the issue there.

Project setup

In a Spring Boot 4.0 project (haven't tested older versions) add the kotlinx.serialization plugin and json library. Add a type that is marked as @Serializable by the kotlinx.serialization library. Implement an endpoint that returns one of the types described in the next section.

@Serializable
data class DemoEntity(
    val id: Int = -1,
    @SerialName("sval_kotlinx")
    @JsonProperty("sval_jackson")
    val sval: String,
) {
    val customSval: String
        get() = "CUSTOM!$sval"
}

What happens

Depending on what type is returned, a different serialization library is used, even though kotlinx.serialization supports the serialization of all the types and could be used for all.

Type	Serializer used	Demo Endpoint
DemoEntity	kotlinx.serialization ✔️	/demo1
Map<Int, DemoEntity>	kotlinx.serialization ✔️	/demo2
List<DemoEntity>	kotlinx.serialization ✔️	/demo3
Map<Int, List<DemoEntity>>	Jackson 🚫	/demo4

This is an issue, since kotlinx.serialization ignores properties without a backing field (the customSval in the example) during serialization. Objects serialized by kotlinx.serialization don't include the customSval field (what I want), but when Spring falls back to Jackson, it is included (and later fails deserialization with kotlinx.serialization in a client).

In this case, this could be easily solved by adding @JsonIgnore to the field, so Jackson also ignores it, or using Json { ignoreUnknownKeys = true } as the deserializer in the client. However, this could cause issues in more complex examples and also requires the developer to maintain two libraries and have double annotations (one for each library) whenever something non-standard is used.

Minimal example

Removing the jackson-module-kotlin dependency from build.gradle doesn't change the results of the experiment. Field name annotations are only used to further demonstrate the issues.

https://github.com/Kwasow/spring-boot-issue-kotlinx-serialization

[bclozel]

Thanks for reaching out. I think that the behavior you are expecting is the opposite of what the community expressed so far, see #35761 for more background. We also recently fixed a few issues around message converters in Spring Boot.

I'm not sure we can really reason about complex types like Map<Int, List<DemoEntity>, but in your case it seems that you expect Kotlinx Serialization to handle all payloads for serialization? In Spring Boot, this should happen by default if you don't have any other Java library supporting JSON. You can still force this by configuring something like:

@Configuration(proxyBeanMethods = false)
public class KotlinSerializationConfiguration {

	@Bean
	ServerHttpMessageConvertersCustomizer kotlinSerializationCustomizer(Json kotlinJson) {
		return converters -> {
			// the predicate argument can be customized with your own function to select more precisely
			// which payloads should be selected for Kotlinx Serialization handling
			KotlinSerializationJsonHttpMessageConverter converter =
					new KotlinSerializationJsonHttpMessageConverter(kotlinJson, (type) -> true);
			converters.withKotlinSerializationJsonConverter(converter);
		};
	}
}

[Kwasow]

Hi, thank you for your swift response.

Adding the suggested configuration does indeed force Spring to always use Kotlinx Serialization, even for the "double-complex" types. Sorry, I was unaware I could configure a predicate in the converter constructor - maybe it's just me being a beginner in Spring. If the community preferred this serialization behavior, I'll gladly agree and use the proposed fix.

Maybe it would be worth documenting what the default behavior is? The only mention of Kotlinx Serialization in the docs is here. To me, it sounds a bit like "just add the dependency, and it will work", while there are some nuances that you mentioned and Jackson might still be used at times. If it's not worth documenting in your opinion, then this issue will serve that purpose.

[bclozel]

@Kwasow As you can see in the linked issue (and recent ones in Boot like spring-projects/spring-boot#48310), those are very recent changes that came in with feedback from the community in the release candidate stage.

So in your case, your application has Jackson on the classpath but you don't really need it? Or is it useful for something else (maybe Spring Boot Actuator)? We can definitely use this issue to improve the documentation.

Pinging @sdeleuze because we've been working on this together.

[Kwasow]

I think this is especially confusing, since it also happens, when Jackson is not a direct dependency. In my case (repo) the Jackson library is included in the classpath, because org.springframework.boot:spring-boot-starter-webmvc (and com.google.firebase:firebase-admin) depends on it. Since many projects will likely use the webmvc starter, this could become a common issue.

You can observe it in the demo project - Jackson is included even after the direct dependency (implementation("com.fasterxml.jackson.module:jackson-module-kotlin")) is removed.

If I explicitly exclude the jackson dependency like below, then /demo4 is correctly serialized using Kotlinx Serialization (after removing the Jackson annotation from DemoController).

dependencies {
    implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.9.0")

    implementation("org.springframework.boot:spring-boot-starter-webmvc") {
        exclude(group = "org.springframework.boot", module = "spring-boot-starter-jackson")
    }
    // implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    providedRuntime("org.springframework.boot:spring-boot-starter-tomcat-runtime")
    testImplementation("org.springframework.boot:spring-boot-starter-webmvc-test") {
        exclude(group = "org.springframework.boot", module = "spring-boot-starter-jackson-test")
    }
    testImplementation("org.jetbrains.kotlin:kotlin-test-junit5")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}

But I may not be able to do the same with the firebase-admin dependency and with other dependencies that I might introduce later in the project.

[sdeleuze]

I think we should probably refine KotlinDetector#hasSerializableAnnotation to detect the Map<Int, List<DemoEntity>> use case.