How to use Body::wrap_stream?

[tzcnt]

First I would like to clarify that this question is NOT about "gRPC streaming". The following question regards sending the response to a single unary RPC.

I am asynchronously reading a large number of entries from a database query into a response message. I would like to avoid buffering the entire response in memory before sending it. It seems like the Body::wrap_stream method should allow me to stream those results as they become available, but I'm not sure how to use it.

If I use the default Prost codegen for a repeated RowType field, it creates a Vec and seemingly expects me to entirely fill it before completing. Is there a workaround that doesn't require me to roll my own codegen? What would the code for doing something like this even look like?

[davidpdrsn]

First I would like to clarify that this question is NOT about "gRPC streaming"

You have to send or return a stream to avoid buffering everything all at once. There are some examples here.

[tzcnt]

So, hypothetically, if I had a body that was a 4GB stream of u8s, and I used Body::wrap_stream to generate the body from that, Tonic would still completely buffer the entire 4GB message into memory before sending any packets over the wire?

[davidpdrsn]

Where are returning such a type from? I don't believe the generated RPC methods allows for returning that.

[tzcnt]

That's what I'm asking you. How can I, as a user, make use of the Body::wrap_stream method? If I cannot, then what is the purpose of this method?

In general, how can I send a large unary response (likely a repeated field) without needing to buffer the entire response in memory before sending? Is there a solution that could be implemented by just modifying the generated code? Or is this behavior buried deeper inside the Tonic internals?

[davidpdrsn]

That's what I'm asking you. How can I, as a user, make use of the Body::wrap_stream method? If I cannot, then what is the purpose of this method?

The body type is re-exported from hyper as its used in type signatures. But you can't actually use it when implementing RPCs.

In general, how can I send a large unary response (likely a repeated field) without needing to buffer the entire response in memory before sending? Is there a solution that could be implemented by just modifying the generated code? Or is this behavior buried deeper inside the Tonic internals?

Use a server to client stream like described in the guide.

[tzcnt]

Due to the fact that the encoding of a repeated field requires the length value at the beginning, for the use case where I am receiving an unknown number of entries from the database (and each individual entry may be of a variable size due to varint encoding), they cannot be serialized and transmitted "on-demand" within the context of a single message. I would need to receive all of the entries first, calculate the length properly, and then serialize them. This seems to be a restriction of the Protobuf format itself, not of Tonic.

Thus I will need to use a gRPC server-side streaming RPC.