-
Notifications
You must be signed in to change notification settings - Fork 250
Description
First, a preface: If there's a fundamental reason why this can't be done, please let me know what it is -- I'm mostly interested in learning the contours of the choice made here, and I'm not terribly attached to anything necessarily changing.
When trying to deserialize messages from a borrowed slice, Reader#get_root takes an &'a self reference and returns a Reader<'a>-typed struct specialized to the message type. In effect, this means that the returned Reader<'a> only has a valid lifetime scoped to where the get_root call took place (this is what I mean when I say that it's "ephemeral"). However, in principle, the returned reader could have any lifetime upper-bounded by the lifetime of the original (serialized) bytes, with any necessary supporting information stored as owned data inside of the returned Reader<'a> struct.
My particular motivation for wanting this is that I'm in the process of writing something which uses lmdb_zero as a memory-mapped database, and I would like to be able to create temporary lifetime-parameterized structures in transactions which maintain shared references into the underlying memory-mapped key-value pairs. Since get_root yields readers with ephemeral lifetimes, the intersection of several of these lifetimes becomes empty if the Readers are not owned by some intermediate structure. Having such an intermediate structure isn't ideal because it leads to unnecessary memory usage in keeping around stale Readers which will never be read from again.
To try to restate as simply and concretely as possible, if I have a capnp message Foo which stores a single name (text) field:
fn point_to_name<'a>(mut serialized_bytes: &'a [u8]) -> &'a str {
let reader = capnp::serialize::read_message_from_flat_slice_no_alloc(
&mut serialized_bytes,
capnp::message::ReaderOptions::new(),
).unwrap();
let value_reader = reader.get_root::<foo_capnp::foo::Reader<'_>>().unwrap();
value_reader.get_name().unwrap()
}
will not pass the borrow checker, because the return value only has an ephemeral lifetime. I would expect that instead, the return value's lifetime is something which matches the incoming lifetime of the slice. I believe that other zero-copy deserialization libraries in Rust are capable of this (e.g: flatbuffers, rkyv) based on a quick scan of their documentation, however, I could be mistaken.
I believe that this structuring choice is also tangentially related to a number of other issues and PRs on this crate (#259, #243, maybe others?)