JsonpMappingException thrown when parsing explicitly null co.elastic.clients.elasticsearch._types.Script from Json

[Strongbeard]

Java API client version

8.13.2

Java version

17.0.11

Elasticsearch Version

8.3.2

Problem description

Executive Summary:

Attempting to parse an co.elastic.clients.elasticsearch._types.Script field in a response that is explicitly set to null (ie: the json payload specifically has "script": null in it) throws an JsonpMappingException.

Affected Versions:

At least 8.13.2 through the latest version (8.14.0) as of opening this ticket. I have not tested, nor intend to test, any prior versions to the one I am using.

Details:

A call to co.elastic.clients.elasticsearch.indices.ElasticsearchIndicesClient.getFieldMapping(GetFieldMappingRequest request) with the includeDefaults field of the GetFieldMappingRequest set to true produced the below redacted stacktrace. Setting includeDefaults in the request to its default value of false behaves as expected (aka: no exception thrown).

...
Caused by: co.elastic.clients.transport.TransportException: node: ██████████, status: 200, [es/indices.get_field_mapping] Failed to decode response
        at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:404)
        at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:363)
        at co.elastic.clients.transport.ElasticsearchTransportBase.performRequest(ElasticsearchTransportBase.java:147)
        at co.elastic.clients.elasticsearch.indices.ElasticsearchIndicesClient.getFieldMapping(ElasticsearchIndicesClient.java:1128)
        at ██████████(██████████.java:██████████)
        ... ██████████ common frames omitted
Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.mapping.FloatNumberProperty: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'VALUE_NULL' instead of '[START_OBJECT, KEY_NAME, VALUE_STRING]' (JSON path: ['██████████'].mappings['██████████'].mapping['██████████'].script) (line no=██████████, column no=██████████, offset=██████████)
        at co.elastic.clients.json.JsonpMappingException.from0(JsonpMappingException.java:134)
        at co.elastic.clients.json.JsonpMappingException.from(JsonpMappingException.java:121)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:218)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
        at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
        at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:209)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        at co.elastic.clients.json.BuildFunctionDeserializer.deserialize(BuildFunctionDeserializer.java:47)
        at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
        at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:349)
        at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:333)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
        at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
        at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:349)
        at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:333)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
        at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
        at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:349)
        at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:333)
        at co.elastic.clients.elasticsearch.indices.GetFieldMappingResponse.lambda$createGetFieldMappingResponseDeserializer$0(GetFieldMappingResponse.java:189)
        at co.elastic.clients.json.JsonpDeserializer$3.deserialize(JsonpDeserializer.java:136)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        ... ██████████ common frames omitted
        at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:399)
Caused by: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'VALUE_NULL' instead of '[START_OBJECT, KEY_NAME, VALUE_STRING]'
        at co.elastic.clients.json.JsonpUtils.ensureAccepts(JsonpUtils.java:117)
        at co.elastic.clients.json.UnionDeserializer.deserialize(UnionDeserializer.java:258)
        at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
        at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192)
        ... ██████████ common frames omitted

I wrote the following test to isolate the issue. When "script": null is explicitly present in a payload that would serialize to a FloatNumberProperty, the serialization throws the same JsonpMappingException. Removing the "script": null, entry in the below payload results in successful parsing with the script field in FloatNumberProperty (inherited from the NumberPropertyBase superclass) being defaulted to null, as expected.

{
    "type": "float",
    "index": true,
    "doc_values": true,
    "store": false,
    "ignore_malformed": false,
    "coerce": true,
    "null_value": null,
    "script": null,
    "on_script_error": "fail",
    "meta": {},
    "time_series_dimension": false,
    "time_series_metric": null
}

ObjectMapper objectMapper = new ObjectMapper();
JsonpMapper jsonpMapper = new JacksonJsonpMapper(objectMapper);

try(
    InputStream stream = this.getClass().getResourceAsStream("/FloatNumberProperty.json");
    JsonParser parser = jsonpMapper.jsonProvider().createParser(stream);
) {
    FloatNumberProperty property = mapper.deserialize(parser, FloatNumberProperty.class); // Exception thrown here
}

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.mapping.FloatNumberProperty: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'VALUE_NULL' instead of '[START_OBJECT, KEY_NAME, VALUE_STRING]' (JSON path: script) (line no=9, column no=19, offset=-1)
        at co.elastic.clients.json.JsonpMappingException.from0(JsonpMappingException.java:134)
        at co.elastic.clients.json.JsonpMappingException.from(JsonpMappingException.java:121)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:218)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148)
        at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
        at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
        at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
        at co.elastic.clients.json.JsonpMapperBase.deserialize(JsonpMapperBase.java:69)
        at co.elastic.clients.json.JsonpMapper.deserialize(JsonpMapper.java:49)
        at ██████████(██████████.java:██████████)
Caused by: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'VALUE_NULL' instead of '[START_OBJECT, KEY_NAME, VALUE_STRING]'
        at co.elastic.clients.json.JsonpUtils.ensureAccepts(JsonpUtils.java:117)
        at co.elastic.clients.json.UnionDeserializer.deserialize(UnionDeserializer.java:258)
        at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
        at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
        at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192)
        ... 10 more

From this I concluded that:

1. Script._DESERIALIZER and potentially any other deserializers built from UnionDeserializer.Builder might throw a JsonpMappingException when attempting to serialize an explicit null entry in a payload if none of the elements in the union allow the VALUE_NULL jsonp event.
2. Any request that allows the same behavior that setting includeDefaults in GetFieldMappingRequest triggers could cause the same exception to be thrown if the response contains a null value that is deserialized using a deserializer described in the 1st conclusion.

[l-trotta]

Hello, thank you for the detailed report. I have managed to reproduce the error, but I didn't understand the cause yet, since the same property is deserialized correctly in other union types (aggregations for example), we'll investigate this more.

[l-trotta]

testing with newer versions of the client I can see this was fixed, sorry for the delay!