TypedKeys property not working - always return true

[bsehrabt]

Elastic.Clients.Elasticsearch version: 8.15.8 / 8.16.3

Elasticsearch version: 8.13.4

.NET runtime version: 8.0

Operating system version: Win 11

Description of the problem including expected versus actual behavior:
A clear and concise description of what the bug is.

TypedKeys property is not honored and always returns true.

Steps to reproduce:

1. Write any simple query with property TypedKeys(false)
2. Elasticsearch client I'm using has property DisableDirectStreaming()
3. Search response always returned with type_keys=true which adds prefixes to the response. For example sterm#, filters#, cardinality# etc.
4. TypedKeys property is not honoured and always returns true. I can't switch it off by using TypedKeys(false)

Expected behavior
A clear and concise description of what you expected to happen.

SearchAsync should return search results with typed_keys false

Provide ConnectionSettings (if relevant):

Provide DebugInformation (if relevant):

[bsehrabt]

Example Query:
var searchResponse = _client.SearchAsync(s => s.Index("INDEXNAME"))
.Query(q => q.Bool(b => b.Filter("YOUR_QUERY")))
.Aggregations("YOUR_AGGS").Size(0)
.TrackTotalHits(new TrackHits(false)).FilterPath("aggregations").TypedKeys(false));

[flobernd]

Hi @bsehrabt,

typed-keys are forced to `true´ by the client. This is required to correctly deserialize the response (otherwise the client would not know what type of aggregation was returned by the server).

May I ask why this is a concern? The client as well removes the prefix internally so that you can still access the aggregations by the name you provided:

request.Aggregations.GetCardinality("my_agg_name")

[bsehrabt]

Hi @flobernd , Thanks for your reply.

Actually we have a custom method which processes the search response.
In v7 NEST we use TypedKeys(false).
After migration to v8, Search response always return with typed keys true i.e. appending prefixes to response properties
And it breaks our functionality when trying to access JSON response, as a workaround, we had to make changes to our method to look with prefixes. It would be great if we could get turn this off by passing false to TypedKeys property as it was supported in v7.

Example showing accessing json response both with TypedKeys(false):

In v7 - SelectTokensByPath(jsonDecisionObj, "aggregations.appkeys.buckets") -- works
In v8 -  SelectTokensByPath(jsonDecisionObj, "aggregations.sterms#appkeys.buckets")  -- see sterms#

Also we can no longer access filters aggs with keys in v8, there is an open bug for this -> #8242 and we are making use of index for now as a workaround.

[flobernd]

Hi @bsehrabt,

It would be great if we could get turn this off by passing false to TypedKeys property as it was supported in v7.

This would completely break deserialization of the response and cause an exception. I'm very sorry, but you are using the client in a very non-standard way. Accessing the raw JSON response by using DisableDirectStreaming was never an intended use-case. This property is only meant to be used for debugging/logging purposes.

The intended way of performing low-level requests, is using the transport layer directly. Read more about this here.

If you don't want to manually craft the request, I would suggest something like this:

var searchRequest = new SearchRequest("my-index-000001")
{
    From = 0,
    Size = 10,
    Aggregations = new Dictionary<string, Aggregation>
    {
        { "my_agg", Aggregation.Cardinality(new CardinalityAggregation{ Field = "my_field" })}
    }
};

var myJsonRequest = client.RequestResponseSerializer.SerializeToString(searchRequest);
var path = new EndpointPath(HttpMethod.GET, "/my-index-000001/_search?filter_path=aggregations&typed_keys=false"); // Or better: Completely omit 'typed_keys' from the query portion of the url.
var myJsonResponse = client.Transport.RequestAsync<StringResponse>(path, PostData.String(myJsonRequest), null, null, default);

Please note, that only the body properties are serialized this way. Some parameters must be configured in the path (e.g. the index name in this case) and others in the query portion of the url (like e.g. typed_keys, filter_path, pretty, etc.). Using the low-level transport moves responsibility for these parameters to the caller.

[bsehrabt]

@flobernd we only use DisableDirectStreaming for one specific use case where we need to access to full search response.
I understand your concerns that it would break deserialization of the response and cause an exception.
I can work on an alternative approach, thanks for your response