Issue with HealthResponse After Upgrading Elasticsearch Java Client to 8.17

[rawnak35]

Java API client version

8.17

Java version

11

Elasticsearch Version

8.12

Problem description

After upgrading the Elasticsearch Java client from version 8.12 to 8.17 (server version still 8.12), the call to client.cluster().health() is failing with the following exception:

Caused by: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'HealthResponse.unassignedPrimaryShards'
    at co.elastic.clients.util.ApiTypeHelper.requireNonNull(ApiTypeHelper.java:76)

I attempted to use the workaround suggested in the documentation for handling missing required properties (ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(true);). However, it resulted in a NullPointerException:

Caused by: java.lang.NullPointerException
    at co.elastic.clients.elasticsearch.cluster.health.HealthResponseBody.<init>(HealthResponseBody.java:132)
    at co.elastic.clients.elasticsearch.cluster.HealthResponse.<init>(HealthResponse.java:59)

[floorleap]

I'm getting a very similar issue, so I'll tag along here.

Mine is on ES version and API version 8.17 as well, but for a PIT request. The missing field is OpenPointInTimeResponse.shards.

The first error was caused by: node: https://localhost:50388/, status: 200, [es/open_point_in_time] Failed to decode response
co.elastic.clients.transport.TransportException: node: https://localhost:50388/, status: 200, [es/open_point_in_time] Failed to decode response
	at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:409)
	at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:367)
	at co.elastic.clients.transport.ElasticsearchTransportBase.performRequest(ElasticsearchTransportBase.java:148)
	at co.elastic.clients.elasticsearch.ElasticsearchClient.openPointInTime(ElasticsearchClient.java:1809)
[...]

Caused by: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'OpenPointInTimeResponse.shards'

[l-trotta]

Hello!

@rawnak35 unassignedPrimaryShards was added in version 8.16.0 of the server as a mandatory parameter, so that's why versions of the client > 8.16.0 require it as well; our usual policy is that the client should fully support only versions of the server equal or greater (forward compatibility), but I can see that many users are experiencing the same issue with this specific property, so I'll see if we can make an exception just for this case and make it optional.

@floorleap this is a similar problem, but the issue is completely different, so it would be better to open a new one! In the meantime: can you confirm that you're using 8.17.0 for both the client and the server? We have recently fixed OpenPointInTime and the elasticsearch server should always send _shard in the response from version 8.16.0 forward.