Impossible to get a BytesMut (or owned bytes buffer) of a `Message` without copying it

[elenakrittik]

Calling Bytes::is_unique on a Message's Bytes instance always (seemingly at least) returns false, making it impossible to get a &mut [u8] of said Bytes instance without copying it. That, from what i could understand reading bytes's documentation, means that tungstenite keeps references to the messsage bytes somewhere inside. I need at least mutable (better owned) access to all bytes i receive, and copying everything passing over the websocket connection is a giant waste. Where does tungstenite store those references (or does it at all?), and how can i make it not do that (or otherwise give me the ownership of the message bytes)? Thank you.

[elenakrittik]

Dug a bit into this. The reason behind Bytes being perma unique seems to be this call to .freeze():

tungstenite-rs/src/protocol/frame/mod.rs

Line 227 in 5e15390

let frame = Frame::from_payload(header, payload.freeze());

Tungstenite doesn't seem to actually store these Bytes anywhere afterwards, so i don't understand why this was done

[daniel-abramov]

If I understand your question correctly, you're asking about the semantics of Bytes and BytesMut.

tungstenite reads the WebSocket frames into an underlying buffer (BytesMut in this case) until they can be returned to the user. Once the WebSocket message is fully read, we just split_to() it and get a new BytesMut that we return to the user (we convert it to Bytes by calling freeze()). The conversion via freeze() does not really change anything semantically (nor does it create an additional reference), i.e. whether BytesMut or Bytes, they both reference some storage. Since we split_to() the part of the buffer that contains the message, it is possible that at the time the user receives the message, their Bytes (or BytesMut) may be pointing to the same storage (irrespective of calling to freeze() or not).

fn main() {
    let mut buf = bytes::BytesMut::from("Hello, world!");
    let split = buf.split_to(5).freeze();
    println!("unique = {}, {:?}", split.is_unique(), split); // unique = false, b"Hello"
}

[elenakrittik]

Thank you for the reply. From my local experiments, it seems like it would not be a reasonable amount of changes to make things like Message::Binary contain BytesMut instead of Bytes (or otherwise be able to get a BytesMut from tungstenite)? At least in my case i had to completely delete Message::Text and any related types/functions

[daniel-abramov]

Returning BytesMut instead of Bytes would not change anything in this particular context. Even if we did not call freeze() and returned that same BytesMut, converting it to Vec<u8> would still involve copying. BytesMut represents a unique view into a potentially shared memory region.

[elenakrittik]

Returning BytesMut instead of Bytes would not change anything in this particular context. Even if we did not call freeze() and returned that same BytesMut, converting it to Vec<u8> would still involve copying. BytesMut represents a unique view into a potentially shared memory region.

I have no need to convert to Vec (though, as i said, that would be the ideal scenario), i just need mutable access to that "memory region" (the data transmitted over the connection). Having a BytesMut instead of Bytes is enough for that purpose

[daniel-abramov]

I see. Yeah, when I'm thinking about it, the current implementation of is_unique() and its usage inside bytes to determine when to copy is a bit confusing. Strictly speaking, there shouldn't be a reason to perform any copy internally when converting Bytes back to BytesMut, given that there are no references to that particular slice of a memory region. I need to think about how to solve it elegantly.