Deserialization of class with generic collection inside depends on how is was deserialized first time

[lunaticare]

Hi everyone!

After migration from Jackson 1.9.x to 2.x (the problem can be reproduced in 2.5.0) I've got a problem deserializing generic collections.
Here's the test that documents the behavour.
Am I doing something wrong?

package com.fasterxml.jackson.databind.test;

import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import org.json.JSONException;
import org.json.JSONObject;
import org.junit.Test;
import org.skyscreamer.jsonassert.JSONAssert;

import java.io.IOException;
import java.util.Date;
import java.util.Map;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotEquals;

/**
 * Deserialization of class with generic collection inside depends on how is was deserialized first time.
 */
public class JacksonPolymorphicDeSerFailTest {
    private static final int TIMESTAMP = 123456;
    private final MapContainer originMap =
            new MapContainer(ImmutableMap.<String, Object>of("DateValue", new Date(TIMESTAMP)));

    /**
     * If the class was first deserialized as polymorphic field,
     * deserialization will fail at complex type.
     */
    @Test
    public void testFactorsSerializationFail() throws IOException, JSONException {
        ObjectMapper mapper = new ObjectMapper();

        // incorrect
        MapContainer deserMapBad = createDeSerMapContainer(originMap, mapper);
        assertNotEquals(originMap, deserMapBad);
        assertEquals(ImmutableList.of("java.util.Date", TIMESTAMP),
                deserMapBad.getMap().get("DateValue"));

        // incorrect again
        assertNotEquals(originMap, mapper.readValue(mapper.writeValueAsString(originMap),
                MapContainer.class));
    }

    /**
     * If the class was first deserialized as is,
     * deserialization will work correctly.
     */
    @Test
    public void testFactorsSerializationMagicGood() throws IOException, JSONException {
        ObjectMapper mapper = new ObjectMapper();
        // commenting out the following statement will fail the test
        assertEquals(new MapContainer(ImmutableMap.<String, Object>of("1", 1)),
                mapper.readValue(
                        mapper.writeValueAsString(new MapContainer(ImmutableMap.<String, Object>of("1", 1))),
                        MapContainer.class));

        MapContainer deserMapGood = createDeSerMapContainer(originMap, mapper);
        // correct
        assertEquals(originMap, deserMapGood);
        assertEquals(new Date(TIMESTAMP), deserMapGood.getMap().get("DateValue"));

        // correct again
        assertEquals(originMap, mapper.readValue(mapper.writeValueAsString(originMap), MapContainer.class));
    }

    private static MapContainer createDeSerMapContainer(MapContainer originMap, ObjectMapper mapper)
            throws JSONException, IOException {
        PolymorphicValueWrapper result = new PolymorphicValueWrapper();
        result.setValue(originMap);
        String json = mapper.writeValueAsString(result);
        System.out.println("Original map json: " + json);
        JSONAssert.assertEquals("{\"value\":{\"@class\":"
                        + "\"com.fasterxml.jackson.databind.test.JacksonPolymorphicDeSerFailTest$MapContainer\","
                        + "\"map\":{\"DateValue\":[\"java.util.Date\",123456]}}}",
                new JSONObject(json), true);
        PolymorphicValueWrapper deserializedResult = mapper.readValue(json, PolymorphicValueWrapper.class);
        System.out.println("Deserialized map json: " + mapper.writeValueAsString(deserializedResult));
        return (MapContainer) deserializedResult.getValue();
    }

    @JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL)
    public static class MapContainer {
        @JsonTypeInfo(use = JsonTypeInfo.Id.CLASS,
                include = JsonTypeInfo.As.PROPERTY,
                property = "@class")
        private Map<String, Object> map;

        public MapContainer() { }

        public MapContainer(Map<String, Object> map) {
            this.map = map;
        }

        public Map<String, Object> getMap() {
            return map;
        }

        public void setMap(Map<String, Object> map) {
            this.map = map;
        }

        @Override
        public boolean equals(Object o) {
            if (this == o) {
                return true;
            }
            if (o == null || getClass() != o.getClass()) {
                return false;
            }

            MapContainer that = (MapContainer) o;

            if (map != null ? !map.equals(that.map) : that.map != null) {
                return false;
            }

            return true;
        }

        @Override
        public int hashCode() {
            return map != null ? map.hashCode() : 0;
        }

        @Override
        public String toString() {
            return "MapContainer{" +
                    "map=" + map +
                    '}';
        }
    }

    @JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL)
    public static class PolymorphicValueWrapper {
        @JsonTypeInfo(use = JsonTypeInfo.Id.CLASS,
                include = JsonTypeInfo.As.PROPERTY,
                property = "@class")
        private Object value;

        public Object getValue() {
            return value;
        }

        public void setValue(Object value) {
            this.value = value;
        }
    }
}

[cowtowncoder]

At first look, I suspect this may be the case where it is not possible to support polymorphic handling, because all that is known, statically, is that a java.lang.Object is being handled. While class information for that value can be handled, same is not true for typing within contained structured type.

That is, at least not currently possible to support. Theoretically perhaps it would be possible to include generic type signature instead of raw class; although in this case even that would not be help, when PolymorphicValueWrapper is passed as object, so type erasure removes generic parameters as well.

From user perspective, it may be possible to make this case work by using "default typing", in which case determination of whether type information is included is based on checking what kind of class we have. If applied to java.lang.Object (one of choices at least does include that), type information should be included.

[lunaticare]

Thank you, @cowtowncoder, for investigating the case.

In both cases serialization gives JSON with all necessary information, that is:

{"value":
  {"@class":"com.fasterxml.jackson.databind.test.JacksonPolymorphicDeSerFailTest$MapContainer",
  "map":
    {"DateValue":
      ["java.util.Date",123456]
    }
  }
}

However, if one tries to first call

ObjectMapper.readValue(jsonStr, MapContainer.class)

he'll get correct map values henceforth, no matter if he deserializes MapContainer or PolymorphicValueWrapper. On the other side, if one calls

ObjectMapper.readValue(jsonStr, PolymorphicValueWrapper.class)

MapContainer values will always be deserialized as ArrayList.

So Jackson can do its job pretty good. Looks like some internal cache is built differently in each case.

To workaround the problem I "initialize" ObjectMapper deserializing empty MapContainer.

[cowtowncoder]

That does sound like a bug. A unit test to reproduce this would be great, as behavior simply does not make sense. Your guess as to root cause sounds plausible.

[cowtowncoder]

This is still a big mystery. Disabling caching for both TypeFactory and DeserializerCache (or, alternatively, for deserializers being called) does not help prevent failure, so while some caching/reuse issue would fit the failure mode, it is not a straight-forward problem there.
Some problems were actually fixed wrt caching of Map deserializers, but that hasn't helped either.

[lunaticare]

Maybe we could think of more test cases with type variations, map size, etc?

[cowtowncoder]

@lunaticare I am not sure whether that would help figure out where the problem occurs, unfortunately. But it does seem like polymorphic handling is required, so provingdisproving that (failing without polymorphic type or not) could help. My best guess is that polymorphic type is indeed required.

[cowtowncoder]

Actually, yes, it was due to caching in cases where it shouldnt, due to type handler existence.
But the test provided was slightly off as it actually verified incorrect deser behavior (as list), so I did not first realize fix works... anyway, now fixed for 2.5(.4), master (2.6.0).

[lunaticare]

@cowtowncoder thank you very much!