Json serialization fails or a specific case that contains generics & static methods with generic parameters

[lhotari]

Reproducing the issue

I wasn't able to analyze the exact reason, however I have created a repository that reproduces the issue.
The problem seem to occur only with a certain structure and I wasn't yet able to isolate it further.

The test case that reproduces the issue
References CloudEvent, CloudEventImpl and AttributesImpl classes from CloudEvents Java SDK 1.3.0 .

to reproduce:

git clone https://github.com/lhotari/jackson-bug-2020-08-18
cd jackson-bug-2020-08-18
./gradlew test

fails with exception.

com.github.lhotari.jacksonbug.JacksonBugTest > reproduceSerializerBug() FAILED
    com.fasterxml.jackson.databind.JsonMappingException: Strange Map type java.util.Map: cannot determine type parameters (through reference chain: com.github.lhotari.jacksonbug.JacksonBugTest$MyValue["events"]->java.util.Collections$SingletonList[0]->io.cloudevents.v1.CloudEventImpl["attributes"])
        at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:295)
        at com.fasterxml.jackson.databind.SerializerProvider.reportMappingProblem(SerializerProvider.java:1309)
        at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1447)
        at com.fasterxml.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:562)
        at com.fasterxml.jackson.databind.ser.impl.UnwrappingBeanPropertyWriter._findAndAddDynamic(UnwrappingBeanPropertyWriter.java:211)
        at com.fasterxml.jackson.databind.ser.impl.UnwrappingBeanPropertyWriter.serializeAsField(UnwrappingBeanPropertyWriter.java:102)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755)
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755)
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
        at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4407)
        at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3661)
        at com.github.lhotari.jacksonbug.JacksonBugTest.reproduceSerializerBug(JacksonBugTest.java:39)

        Caused by:
        java.lang.IllegalArgumentException: Strange Map type java.util.Map: cannot determine type parameters
            at com.fasterxml.jackson.databind.type.TypeFactory._mapType(TypeFactory.java:1178)
            at com.fasterxml.jackson.databind.type.TypeFactory._fromWellKnownClass(TypeFactory.java:1471)
            at com.fasterxml.jackson.databind.type.TypeFactory._fromClass(TypeFactory.java:1414)
            at com.fasterxml.jackson.databind.type.TypeFactory.constructType(TypeFactory.java:705)
            at com.fasterxml.jackson.databind.introspect.AnnotatedClass.resolveType(AnnotatedClass.java:229)
            at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.getParameterType(AnnotatedMethod.java:143)
            at com.fasterxml.jackson.databind.introspect.AnnotatedWithParams.getParameter(AnnotatedWithParams.java:86)
            at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addCreators(POJOPropertiesCollector.java:500)
            at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collectAll(POJOPropertiesCollector.java:327)
            at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getJsonValueAccessor(POJOPropertiesCollector.java:203)
            at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.findJsonValueAccessor(BasicBeanDescription.java:252)
            at com.fasterxml.jackson.databind.ser.BasicSerializerFactory.findSerializerByAnnotations(BasicSerializerFactory.java:396)
            at com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:216)
            at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:165)
            at com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1474)
            at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1442)
            ... 16 more

Comments

• Test passes with Jackson 2.11.1 version, but fails with 2.11.2 .
• Diff between Jackson 2.11.2 and 2.11.1

Changes in TypeFactory.constructType in 2.11.2 for #2796 might have caused the change in behavior.
Looks like the problem that 910edfb fixes could have been similar.

By debugging it can be seen that resolving parameter types with the bindings in AnnotedClass doesn't produce the correct result.

Another observation here is that the method that is been processed by Jackson is produced by a lambda. The method name in the repro case is private static void io.cloudevents.v1.AttributesImpl.lambda$marshal$3(java.util.Map,java.time.ZonedDateTime)

The failure seems to happen when Jackson tries to resolve the parameters for this method generated by the lambda defined at this location:

https://github.com/cloudevents/sdk-java/blob/361a34cc639ddaa75b2a5080f117fc282be7625b/api/src/main/java/io/cloudevents/v1/AttributesImpl.java#L172-L173

[dariuszkuc]

I just hit the same error. My simple test case

@JsonIgnoreProperties(ignoreUnknown = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
data class GenericResponse<T>(
    val data: T? = null,
    val extensions: Map<Any, Any>? = null
)

data class MyData(val foo: Int)

fun main() {
    val mapper = jacksonObjectMapper()
    val input =
        """{"data":{"foo":1}, "extensions":{"bar":2}}"""

    val response: GenericResponse<MyData> = mapper.readValue(input)
    println(response)
}

Above works fine in with Jackson 2.11.1 but fails in 2.11.2 with following exception

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Strange Map type java.util.Map: cannot determine type parameters
 at [Source: (String)"{"data":{"foo":1}, "extensions":{"bar":2}}"; line: 1, column: 1]
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:62)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:227)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:414)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:349)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
	at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:491)
	at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4711)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4520)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3466)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3449)
	at example.ApplicationKt.main(Application.kt:25)
	at example.ApplicationKt.main(Application.kt)

[MartinTeeVarga]

Having similar issues. When I change @dariuszkuc 's example this way, it works:

data class GenericResponse<T>(
    val data: T? = null,
    val extensions: Map<Any, Any>? = null
)

data class MyData(val foo: Int)

fun main() {
    val typeRef = GenericResponse(MyData(1), emptyMap())::class.java

    val mapper = jacksonObjectMapper()
    val input =
        """{"data":{"foo":1}, "extensions":{"bar":2}}"""

    val response: GenericResponse<MyData> = mapper.readValue(input, typeRef)
    println(response)
}

I believe the problem might be somewhere in the TypeFactory refactoring.

[cowtowncoder]

Quick note: unfortunately Kotlin-based examples won't help a lot when working on databind (although they are obviously fine for Kotlin module), as I can not use them for unit tests.
It would be super helpful if one of simpler examples could be "javafied": I can follow the example but am afraid my translation might lose something (f.ex not sure whether Map<Any, Any> would exactly translate to Map<?, ?> or not -- sometimes differences matter).

But maybe I can simplify the original test which is Java only I think.

[cowtowncoder]

I hope to look into this soon -- I suspect that @lhotari is correct about issue related to (that is, fix that may have caused regression). I am guessing that things might have worked as a side-effect.

Was able to test against 2.12.0-SNAPSHOT with following def in build.gradle:

repositories {
    mavenCentral()
    maven {
        url "https://oss.sonatype.org/content/repositories/snapshots"
    }
}

and looks like the problem still persists (which is not entirely unexpected).

What I need is then just self-contained version to add as unit test; however, should also be able to import existing project into IDE for some investigation once I have time.

[MartinTeeVarga]

Breaking Java code similar to the above Kotlin example would be:

public class ObjectMapperGenericClassTest extends BaseTest {

    static final String JSON = "{ \"field\": { \"number\": 1 }, \"map\": { \"key\": \"value\" } }";

    static class GenericEntity<T> {
        T field;

        Map map;

        public void setField(T field) {
            this.field = field;
        }

        public T getField() {
            return field;
        }

        public Map getMap() {
            return map;
        }

        public void setMap(Map map) {
            this.map = map;
        }
    }

    static class SimpleEntity {
        Integer number;

        public void setNumber(Integer number) {
            this.number = number;
        }

        public Integer getNumber() {
            return number;
        }
    }

    public void test() throws Exception {
        ObjectMapper m = new ObjectMapper();
        GenericEntity<SimpleEntity> genericEntity = m.readValue(JSON, new TypeReference<GenericEntity<SimpleEntity>>() {});
    }
}

It works in 2.11.2 if I add any type parameter to the Map and it also works if I remove the type parameter <T> and the whole SimpleEntity and keep the Map without type parameter:

    static final String JSON = "{ \"map\": { \"key\": \"value\" } }";

    static class GenericEntity {

        Map map;

        public Map getMap() {
            return map;
        }

        public void setMap(Map map) {
            this.map = map;
        }
    }

    public void test throws Exception {
        ObjectMapper m = new ObjectMapper();
        GenericEntity genericEntity = m.readValue(JSON, new TypeReference<GenericEntity>() {});
    }

[MartinTeeVarga]

In my real case I am actually seeing:

java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to MyClassAsTypeParameterOfAnotherClass

I am still working on a minimal example for that.

[lhotari]

Thanks for the repros @dariuszkuc and @MartinTeeVarga . It seems that the issue with Kotlin and the example that @MartinTeeVarga provided are somewhat different than what I have. In my case, the issue is in serialization. I assume that the problems are related.
@cowtowncoder I now had some time to isolate the issue from the repository that I originally provided. This is the simplest case I was able to strip it down to. I made some observations that if I change on of the 3 things: "Entity<?> -> Entity", "remove public static Attributes dummyMethod(Map attributes)" or "remove @JsonUnwrapped", the test will pass and JSON serialization won't fail. It seems that the JsonUnwrapped together with generics and the static "dummyMethod" (in the repro) trigger the issue that I have faced.

public class JacksonBugIsolatedTest {
    static final class Wrapper {
        // if Entity<?> -> Entity , the test passes
        private final List<Entity<?>> entities;

        @JsonCreator
        public Wrapper(List<Entity<?>> entities) {
            this.entities = entities;
        }

        public List<Entity<?>> getEntities() {
            return this.entities;
        }
    }

    public static class Entity<T> {
        @JsonIgnore
        private final Attributes attributes;

        private final T data;

        public Entity(Attributes attributes, T data) {
            this.attributes = attributes;
            this.data = data;
        }

        // if @JsonUnwrapped is removed, the test passes
        @JsonUnwrapped
        public Attributes getAttributes() {
            return attributes;
        }

        public T getData() {
            return data;
        }

        @JsonCreator
        public static <T> Entity<T> create(Attributes attributes, T data) {
            return new Entity<>(attributes, data);
        }
    }

    public static class Attributes {
        private final String id;

        public Attributes(String id) {
            this.id = id;
        }

        public String getId() {
            return id;
        }

        @JsonCreator
        public static Attributes create(String id) {
            return new Attributes(id);
        }

        // if this method is removed, the test passes
        public static Attributes dummyMethod(Map attributes) {
            return null;
        }
    }

    // this test passes with Jackson 2.11.1, but fails with Jackson 2.11.2
    @Test
    public void reproduceSerializerBug() throws JsonProcessingException {
        ObjectMapper objectMapper = new ObjectMapper().findAndRegisterModules();
        Entity<String> entity = new Entity<>(new Attributes("id"), "hello");
        Wrapper val = new Wrapper(Collections.singletonList(entity));
        // fails with com.fasterxml.jackson.databind.JsonMappingException: Strange Map type java.util.Map: cannot determine type parameters (through reference chain: com.github.lhotari.jacksonbug.JacksonBugIsolatedTest$Wrapper["entities"]->java.util.Collections$SingletonList[0]->com.github.lhotari.jacksonbug.JacksonBugIsolatedTest$Entity["attributes"])
        System.out.println(objectMapper.writeValueAsString(val));
    }
}

This test class is also available in the repro repo: https://github.com/lhotari/jackson-bug-2020-08-18/blob/master/src/test/java/com/github/lhotari/jacksonbug/JacksonBugIsolatedTest.java

Also tried to add the test to jackson-databind: lhotari@ecdfd6c

[cowtowncoder]

Thank you everyone for help here! Yes, it is possible there are multiple issues and they might need separate fixes (or at least test cases). I'll first focus on @lhotari 's case.

[cowtowncoder]

Hmmh. findAndRegisterModules() is bit problematic as it could load anything -- looking at code, I am guessing it probably loads jackson-module-parameter-names since otherwise parameter names are not visible. But I should be able to add @JsonProperty annotations to remove that dependency.