JsonSerializerOptions DefaultBufferSize for ASP.NET Core with source generator

[ladeak]

Source generated JSON serialization code is only invoked for response serialization in ASP.NET Core if the response size is less than half of the JsonSerializerOptions.DefaultBufferSize.

What is the best way to tune the setting, when most of the responses are within the default size, but there are a few very frequent responses that are close to MB (over the default limit)? Does it make sense to set 2 MB buffer size, even if it allocates the buffer pool on the LOH? If I keep the limit as default, it does not allocate the buffer pool on the LOH, but these large responses for every request would still allocate a temporary buffer on the heap? (seemingly making this even less attractive)

As I understand this is a "streaming" scenario with ASP.NET Core, meaning it is the async version of the serialize methods. Based on the remarks it is unclear if increasing the size to a MB is reasonable choice.

I wonder if @eiriktsarpalis has some thoughts about this too.

[eiriktsarpalis]

Source generated JSON serialization code is only invoked for response serialization in ASP.NET Core if the response size is less than half of the JsonSerializerOptions.DefaultBufferSize.

To clarify, the source generator is being used regardless of what the response size is. What changes really is whether fast-path or metadata-based source generator modes are being called.

I can't clear any clear recommendations here, but ultimately this a matter of trade-offs. How important is the added throughput of fast-path for your application, and is it pooling larger buffers to achieve it worth it?

[ladeak]

The metadata based source generation only helps to be faster with the first request (compared to non-source generated serialization), while the fast-path with all requests?

[eiriktsarpalis]

Primarily, metadata-based source generator is safe to use in trimmed/AOT applications (reflection is not).

[ladeak]

The more I read about this and test, I understand the additional throughput comes from fact that the serialized property names and fields of the classes are determined at compile time instead of runtime (and that saves CPU cycles for not needing to set the casing etc.). So, the increased throughput does not depend on the size of the serialized object (meaning I have two classes with the same size when serialized, but one with a single property and one with many), but rather on the number of properties/fields to be written. Is this correct to you understanding?

[eiriktsarpalis]

It's not just that, fast-path uses one method that inlines all serialization logic for a given object. OTOH the metadata based approaches generates a bunch of delegates that are fed into the shared converter infrastructure so there's a lot of indirection happening for every object.

[ladeak]

Thank you