@JsonProperty is ignored, sometimes

[brefsdal]

I've found a case where the JsonProperty annotation is ignored by the serializer. Sometimes. Unfortunately, I haven't found out why this occurs, seemly at random. Here is a example schema

case class MyBase(
  @JsonProperty("my_value"): myValue: Int
)
case class MyClass(
  @JsonProperty("my_name") myName: String,
  @JsonUnwrapped base: MyBase
)
case class App(
  name: String,
  @JsonUnwrapped class: MyClass
)

Most of the time I'll see 'my_name' in the output JSON string. Sometimes a machine will output myName. Once the machine starts outputting camelcase like this, it continues to do so until I restart the JVM. Here is the configuration of the ObjectMapper below

import com.fasterxml.jackson.annotation.JsonInclude
import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper, SerializationFeature}
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper

class ObjectMapperWithScalaMixin extends ObjectMapper with ScalaObjectMapper
object JacksonConfig {
  val apiObjectMapper = {
    val objectMapper = new ObjectMapperWithScalaMixin
    objectMapper.registerModule(DefaultScalaModule)
    objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL)
    objectMapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
    objectMapper
  }

JacksonConfig.objectMapper.writeValueAsString(instanceOfApp)

[brefsdal]

I am using version 2.4.4

[brefsdal]

java version "1.7.0_65"
OpenJDK Runtime Environment (IcedTea 2.5.3) (7u71-2.5.3-0ubuntu0.14.04.1)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)

[cowtowncoder]

While it is not guaranteed to change things, I would suggest latest patch release, 2.4.5, since jackson-databind has certain fixes to caches which could theoretically affect things here. But mostly just to rule out that possibility.

[brefsdal]

I've tried version 2.4.5 to no avail. I will have to remove these annotations in order to avoid this issue.

[christophercurrie]

Thanks for reporting this. I know it's terribly inconvenient. This might be a lot to ask, but are you able to reproduce this problem with an Oracle JDK? We've had other reports (#189) of issues when using OpenJDK, and I suspect this may be related.

[brefsdal]

@christophercurrie This is occurring on Oracle Hotspot JDK as well.

[christophercurrie]

Cool, thanks for checking. I'll see if I can reproduce this. You might also try Jackson 2.5.2, for which the Scala module has improved certain aspects of property detection.

[brefsdal]

Upon upgrading to Jackson 2.5.2 I find compilation errors that seem to require properties to be unwrapped require the @JsonProperty annotation as well. In referencing my example above.
@JsonUnwrapped class: MyClass
should be
@JsonUnwrapped @JsonProperty("myClass") class: MyClass

Is this expected? Even though there will never be a JSON field named "myClass".

[christophercurrie]

Not expected, and I especially wouldn't expect compilation errors. What errors is the compiler giving you?

[brefsdal]

@christophercurrie Sorry not a compilation error, but a runtime exception
com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'camelcaseVariable' (in class foo.bar.Response)

[brefsdal]

Upgrading to Jackson 2.5.2 has exacerbated the problem. Even more instances of ignored @JsonProperty annotations.

[jendap]

The description here is only one of the manifestations of the problem. The exact class hierarchy and their fields do not matter. We see it happening randomly on any class in our class hierarchy. It is happening during startup - first request and stays for the process lifetime. Jackson properly walk the class and find all the fields. However it sometimes does not process @JsonProperty annotations. Are those two separate steps? Can there be a race condition between them? Or can multiple ObjectMappers hurt?

@christophercurrie would you have any tip where to look? It looks like some race condition.

[christophercurrie]

I suppose a race condition is possible, but I am more likely to suspect the fact that Java reflection, upon which Jackson is based, stopped providing consistent ordering of methods, fields, and annotations, as of Java 7. A good test of this theory would be to run the relevant code under a Java 6 JVM which, despite Oracle's discontinuation of support, is still the only JVM version upon which any guarantees are made for Scala (and which the library must still support).

Which is not to say that this issue shouldn't be fixed, just that it is both hard to reproduce and diagnose. If the issue appears in Java 6, then it's not due to reflection ordering, and will be relatively simple to locate and fix. If not, then it's almost certainly an ordering issue, and diagnosing it will likely require repeated traces with a debugger. This issue is definitely on my list to address, but I have not had the time to investigate.

[cowtowncoder]

@jendap Multiple ObjectMappers should not hurt: no state is shared between mappers, at least if constructed from scratch. If using mapper.copy() there should not be coupling either, but there would be theoretical possibility of unintentional sharing.
So the problem should not be worse with multiple mappers; if anything, use of a single mapper could possibly lead to false caching. But even then there should be something else to trigger problems: one candidate would be use of mix-ins (which could be problematic in 2.4.4/2.4.5 and 2.5.0/2.5.1, when used with Maps with POJO values).

[jendap]

I will find it. It is top priority for us. We are loosing money on it. Unfortunately I can't even after two days replicate it locally. (which is also why I suspect race condition). But we have a production heap dump. We can see anything including the data from java reflection. Comparing the BeanSerializer...
The good one:
_props[]._name is correct from the annotation
_props[]._member._annotations has the annotation
The bad one:
_props[]._name is wrong from field name
_props[]._member._annotations is empty!
... I will try to dig deeper how can the annotations be sometimes empty.

Do you think adding the JsonProperty annotation on methods as well as on fields can make difference?

We can try java6 on a few nodes. The problem is that we need G1 garbage collector to keep our SLA. And if we put it on a few nodes and restart a couple of times it is hard to say if it is not happening for sure. Currently it is probably something like 1 in 20 restarts.