parametrize DeserializeRow with two lifetimes

`DeserializeRow` requires two types of data in order to perform
deserialization:
1) a reference to the CQL frame (a FrameSlice),
2) a slice of specifications of all columns in the row, being part of
   the ResultMetadata.

It's important to understand what is a _deserialized row_. It's not
just an implementor of DeserializeRow; there are some implementors of
`DeserializeRow` who are not yet final rows, but **partially**
deserialized types that support further deserialization - _row
deserializers_, such as `ColumnIterator`.
_Row deserializers_, as they still need to deserialize some row, are
naturally bound by 'metadata lifetime. However, _rows_ are completely
deserialized, so they should not be bound by 'metadata - only by 'frame.

When deserializing owned rows, both the frame and the metadata can have
any lifetime and it's not important. When deserializing borrowing rows,
however, they borrow from the frame, so their lifetime must necessarily
be bound by the lifetime of the frame. Metadata is only needed for the
deserialization, so its lifetime does not abstractly bound the
deserialized row.

Up to this commit, DeserializeRow was only parametrized by one lifetime:
'frame, which bounded both the frame slice and the metadata. (why?
the reason is the same as in DeserializeValue - see the previous commit)
This was done that way due to an assumption that both the metadata and
the frame (shared using Bytes) will be owned by the same entity.
However, the new idea of deserializing result metadata to a borrowed
form (to save allocations) makes result metadata's lifetime shorter than
the frame's lifetime. Not to unnecessarily shorten the deserialized
values' lifetime, a separate lifetime parameter is introduced for
result metadata: 'metadata.

The commit is large, but the changes are mostly mechanical, by adding
the second lifetime parameter.

RowIterator and TypedRowIterator are going to get the second lifetime
parameter, too, but for now they pass 'frame as both lifetime parameters
of DeserializeRow.

Author: wprzytula

scylla-cql/src/types/deserialize/mod.rs

@@ -4,7 +4,7 @@
 //!
 //! - A type that implements `DeserializeValue<'frame, 'metadata>` can be deserialized
 //!   from a single _CQL value_ - i.e. an element of a row in the query result,
-//! - A type that implements `DeserializeRow<'frame>` can be deserialized
+//! - A type that implements `DeserializeRow<'frame, 'metadata>` can be deserialized
 //!   from a single _row_ of a query result.
 //!
 //! Those traits are quite similar to each other, both in the idea behind them

scylla-cql/src/types/deserialize/result.rs

@@ -42,7 +42,7 @@ impl<'frame> RowIterator<'frame> {
 }
 
 impl<'frame> Iterator for RowIterator<'frame> {
-    type Item = Result<ColumnIterator<'frame>, DeserializationError>;
+    type Item = Result<ColumnIterator<'frame, 'frame>, DeserializationError>;
 
     #[inline]
     fn next(&mut self) -> Option<Self::Item> {
@@ -85,7 +85,7 @@ pub struct TypedRowIterator<'frame, R> {
 
 impl<'frame, R> TypedRowIterator<'frame, R>
 where
-    R: DeserializeRow<'frame>,
+    R: DeserializeRow<'frame, 'frame>,
 {
     /// Creates a new [TypedRowIterator] from given [RowIterator].
     ///
@@ -115,7 +115,7 @@ where
 
 impl<'frame, R> Iterator for TypedRowIterator<'frame, R>
 where
-    R: DeserializeRow<'frame>,
+    R: DeserializeRow<'frame, 'frame>,
 {
     type Item = Result<R, DeserializationError>;
 

scylla-cql/src/types/deserialize/row.rs

@@ -10,26 +10,29 @@ use crate::frame::response::result::{ColumnSpec, ColumnType, CqlValue, Row};
 
 /// Represents a raw, unparsed column value.
 #[non_exhaustive]
-pub struct RawColumn<'frame> {
+pub struct RawColumn<'frame, 'metadata> {
     pub index: usize,
-    pub spec: &'frame ColumnSpec<'frame>,
+    pub spec: &'metadata ColumnSpec<'metadata>,
     pub slice: Option<FrameSlice<'frame>>,
 }
 
 /// Iterates over columns of a single row.
 #[derive(Clone, Debug)]
-pub struct ColumnIterator<'frame> {
-    specs: std::iter::Enumerate<std::slice::Iter<'frame, ColumnSpec<'frame>>>,
+pub struct ColumnIterator<'frame, 'metadata> {
+    specs: std::iter::Enumerate<std::slice::Iter<'metadata, ColumnSpec<'metadata>>>,
     slice: FrameSlice<'frame>,
 }
 
-impl<'frame> ColumnIterator<'frame> {
+impl<'frame, 'metadata> ColumnIterator<'frame, 'metadata> {
     /// Creates a new iterator over a single row.
     ///
     /// - `specs` - information about columns of the serialized response,
     /// - `slice` - a [FrameSlice] which points to the serialized row.
     #[inline]
-    pub(crate) fn new(specs: &'frame [ColumnSpec], slice: FrameSlice<'frame>) -> Self {
+    pub(crate) fn new(
+        specs: &'metadata [ColumnSpec<'metadata>],
+        slice: FrameSlice<'frame>,
+    ) -> Self {
         Self {
             specs: specs.iter().enumerate(),
             slice,
@@ -44,8 +47,8 @@ impl<'frame> ColumnIterator<'frame> {
     }
 }
 
-impl<'frame> Iterator for ColumnIterator<'frame> {
-    type Item = Result<RawColumn<'frame>, DeserializationError>;
+impl<'frame, 'metadata> Iterator for ColumnIterator<'frame, 'metadata> {
+    type Item = Result<RawColumn<'frame, 'metadata>, DeserializationError>;
 
     #[inline]
     fn next(&mut self) -> Option<Self::Item> {
@@ -84,7 +87,7 @@ impl<'frame> Iterator for ColumnIterator<'frame> {
 /// The crate also provides a derive macro which allows to automatically
 /// implement the trait for a custom type. For more details on what the macro
 /// is capable of, see its documentation.
-pub trait DeserializeRow<'frame>
+pub trait DeserializeRow<'frame, 'metadata>
 where
     Self: Sized,
 {
@@ -100,7 +103,7 @@ where
     /// the row's type. Note that `deserialize` is not an unsafe function,
     /// so it should not use the assumption about `type_check` being called
     /// as an excuse to run `unsafe` code.
-    fn deserialize(row: ColumnIterator<'frame>) -> Result<Self, DeserializationError>;
+    fn deserialize(row: ColumnIterator<'frame, 'metadata>) -> Result<Self, DeserializationError>;
 }
 
 // raw deserialization as ColumnIterator
@@ -111,14 +114,14 @@ where
 // Implementing DeserializeRow for it allows us to simplify our interface. For example,
 // we have `QueryResult::rows<T: DeserializeRow>()` - you can put T = ColumnIterator
 // instead of having a separate rows_raw function or something like this.
-impl<'frame> DeserializeRow<'frame> for ColumnIterator<'frame> {
+impl<'frame, 'metadata> DeserializeRow<'frame, 'metadata> for ColumnIterator<'frame, 'metadata> {
     #[inline]
     fn type_check(_specs: &[ColumnSpec]) -> Result<(), TypeCheckError> {
         Ok(())
     }
 
     #[inline]
-    fn deserialize(row: ColumnIterator<'frame>) -> Result<Self, DeserializationError> {
+    fn deserialize(row: ColumnIterator<'frame, 'metadata>) -> Result<Self, DeserializationError> {
         Ok(row)
     }
 }
@@ -140,15 +143,17 @@ make_error_replace_rust_name!(
 /// While no longer encouraged (because the new framework encourages deserializing
 /// directly into desired types, entirely bypassing [CqlValue]), this can be indispensable
 /// for some use cases, i.e. those involving dynamic parsing (ORMs?).
-impl<'frame> DeserializeRow<'frame> for Row {
+impl<'frame, 'metadata> DeserializeRow<'frame, 'metadata> for Row {
     #[inline]
     fn type_check(_specs: &[ColumnSpec]) -> Result<(), TypeCheckError> {
         // CqlValues accept all types, no type checking needed.
         Ok(())
     }
 
     #[inline]
-    fn deserialize(mut row: ColumnIterator<'frame>) -> Result<Self, DeserializationError> {
+    fn deserialize(
+        mut row: ColumnIterator<'frame, 'metadata>,
+    ) -> Result<Self, DeserializationError> {
         let mut columns = Vec::with_capacity(row.size_hint().0);
         while let Some(column) = row
             .next()
@@ -181,17 +186,17 @@ impl<'frame> DeserializeRow<'frame> for Row {
 /// and needed conversions, issuing meaningful errors in case something goes wrong.
 macro_rules! impl_tuple {
     ($($Ti:ident),*; $($idx:literal),*; $($idf:ident),*) => {
-        impl<'frame, $($Ti),*> DeserializeRow<'frame> for ($($Ti,)*)
+        impl<'frame, 'metadata, $($Ti),*> DeserializeRow<'frame, 'metadata> for ($($Ti,)*)
         where
-            $($Ti: DeserializeValue<'frame, 'frame>),*
+            $($Ti: DeserializeValue<'frame, 'metadata>),*
         {
             fn type_check(specs: &[ColumnSpec]) -> Result<(), TypeCheckError> {
                 const TUPLE_LEN: usize = (&[$($idx),*] as &[i32]).len();
 
                 let column_types_iter = || specs.iter().map(|spec| spec.typ().clone().into_owned());
                 if let [$($idf),*] = &specs {
                     $(
-                        <$Ti as DeserializeValue<'frame, 'frame>>::type_check($idf.typ())
+                        <$Ti as DeserializeValue<'frame, 'metadata>>::type_check($idf.typ())
                             .map_err(|err| mk_typck_err::<Self>(column_types_iter(), BuiltinTypeCheckErrorKind::ColumnTypeCheckFailed {
                                 column_index: $idx,
                                 column_name: specs[$idx].name().to_owned(),
@@ -206,7 +211,7 @@ macro_rules! impl_tuple {
                 }
             }
 
-            fn deserialize(mut row: ColumnIterator<'frame>) -> Result<Self, DeserializationError> {
+            fn deserialize(mut row: ColumnIterator<'frame, 'metadata>) -> Result<Self, DeserializationError> {
                 const TUPLE_LEN: usize = (&[$($idx),*] as &[i32]).len();
 
                 let ret = (
@@ -217,7 +222,7 @@ macro_rules! impl_tuple {
                             $idx
                         )).map_err(deser_error_replace_rust_name::<Self>)?;
 
-                        <$Ti as DeserializeValue<'frame, 'frame>>::deserialize(column.spec.typ(), column.slice)
+                        <$Ti as DeserializeValue<'frame, 'metadata>>::deserialize(column.spec.typ(), column.slice)
                             .map_err(|err| mk_deser_err::<Self>(BuiltinDeserializationErrorKind::ColumnDeserializationFailed {
                                 column_index: column.index,
                                 column_name: column.spec.name().to_owned(),

scylla-cql/src/types/deserialize/row_tests.rs

@@ -251,18 +251,18 @@ fn val_str(s: &str) -> Option<Vec<u8>> {
     Some(s.as_bytes().to_vec())
 }
 
-fn deserialize<'frame, R>(
-    specs: &'frame [ColumnSpec],
+fn deserialize<'frame, 'metadata, R>(
+    specs: &'metadata [ColumnSpec<'metadata>],
     byts: &'frame Bytes,
 ) -> Result<R, DeserializationError>
 where
-    R: DeserializeRow<'frame>,
+    R: DeserializeRow<'frame, 'metadata>,
 {
-    <R as DeserializeRow<'frame>>::type_check(specs)
+    <R as DeserializeRow<'frame, 'metadata>>::type_check(specs)
         .map_err(|typecheck_err| DeserializationError(typecheck_err.0))?;
     let slice = FrameSlice::new(byts);
     let iter = ColumnIterator::new(specs, slice);
-    <R as DeserializeRow<'frame>>::deserialize(iter)
+    <R as DeserializeRow<'frame, 'metadata>>::deserialize(iter)
 }
 
 #[track_caller]

scylla-macros/src/deserialize/mod.rs

@@ -114,29 +114,12 @@ where
         let trait_: syn::Path = parse_quote!(#macro_internal::#trait_);
         let items = items.into_iter();
 
-        // This `if` is purely temporary. When in a next commit DeserializeRow receives a 'metadata lifetime,
-        // the handling of both traits is unified again.
-        if trait_
-            .segments
-            .last()
-            .is_some_and(|name| name.ident == "DeserializeValue")
-        {
-            parse_quote! {
-                impl<#frame_lifetime, #metadata_lifetime, #impl_generics>
-                    #trait_<#frame_lifetime, #metadata_lifetime> for #struct_name #ty_generics
-                where #(#predicates),*
-                {
-                    #(#items)*
-                }
-            }
-        } else {
-            parse_quote! {
-                impl<#frame_lifetime, #impl_generics>
-                    #trait_<#frame_lifetime> for #struct_name #ty_generics
-                where #(#predicates),*
-                {
-                    #(#items)*
-                }
+        parse_quote! {
+            impl<#frame_lifetime, #metadata_lifetime, #impl_generics>
+                #trait_<#frame_lifetime, #metadata_lifetime> for #struct_name #ty_generics
+            where #(#predicates),*
+            {
+                #(#items)*
             }
         }
     }

scylla-macros/src/deserialize/row.rs

@@ -206,7 +206,7 @@ impl<'sd> TypeCheckAssumeOrderGenerator<'sd> {
         // of the columns correspond fields' names/types.
 
         let macro_internal = self.0.struct_attrs().macro_internal_path();
-        let (constraint_lifetime, _) = self.0.constraint_lifetimes();
+        let (frame_lifetime, metadata_lifetime) = self.0.constraint_lifetimes();
 
         let required_fields_iter = || {
             self.0
@@ -242,7 +242,7 @@ impl<'sd> TypeCheckAssumeOrderGenerator<'sd> {
                             #name_verifications
 
                             // Verify the type
-                            <#required_fields_deserializers as #macro_internal::DeserializeValue<#constraint_lifetime, #constraint_lifetime>>::type_check(#required_fields_idents.typ())
+                            <#required_fields_deserializers as #macro_internal::DeserializeValue<#frame_lifetime, #metadata_lifetime>>::type_check(#required_fields_idents.typ())
                                 .map_err(|err| #macro_internal::mk_row_typck_err::<Self>(
                                     column_types_iter(),
                                     #macro_internal::DeserBuiltinRowTypeCheckErrorKind::ColumnTypeCheckFailed {
@@ -281,7 +281,7 @@ impl<'sd> DeserializeAssumeOrderGenerator<'sd> {
         let macro_internal = self.0.struct_attrs().macro_internal_path();
         let cql_name_literal = field.cql_name_literal();
         let deserializer = field.deserialize_target();
-        let (constraint_lifetime, _) = self.0.constraint_lifetimes();
+        let (frame_lifetime, metadata_lifetime) = self.0.constraint_lifetimes();
 
         let name_check: Option<syn::Stmt> = (!self.0.struct_attrs().skip_name_checks).then(|| parse_quote! {
             if col.spec.name() != #cql_name_literal {
@@ -301,7 +301,7 @@ impl<'sd> DeserializeAssumeOrderGenerator<'sd> {
 
                 #name_check
 
-                <#deserializer as #macro_internal::DeserializeValue<#constraint_lifetime, #constraint_lifetime>>::deserialize(col.spec.typ(), col.slice)
+                <#deserializer as #macro_internal::DeserializeValue<#frame_lifetime, #metadata_lifetime>>::deserialize(col.spec.typ(), col.slice)
                     .map_err(|err| #macro_internal::mk_row_deser_err::<Self>(
                         #macro_internal::BuiltinRowDeserializationErrorKind::ColumnDeserializationFailed {
                             column_index: #field_index,
@@ -315,7 +315,7 @@ impl<'sd> DeserializeAssumeOrderGenerator<'sd> {
 
     fn generate(&self) -> syn::ImplItemFn {
         let macro_internal = self.0.struct_attrs().macro_internal_path();
-        let (constraint_lifetime, _) = self.0.constraint_lifetimes();
+        let (frame_lifetime, metadata_lifetime) = self.0.constraint_lifetimes();
 
         let fields = self.0.fields();
         let field_idents = fields.iter().map(|f| f.ident.as_ref().unwrap());
@@ -327,7 +327,7 @@ impl<'sd> DeserializeAssumeOrderGenerator<'sd> {
         parse_quote! {
             fn deserialize(
                 #[allow(unused_mut)]
-                mut row: #macro_internal::ColumnIterator<#constraint_lifetime>,
+                mut row: #macro_internal::ColumnIterator<#frame_lifetime, #metadata_lifetime>,
             ) -> ::std::result::Result<Self, #macro_internal::DeserializationError> {
                 ::std::result::Result::Ok(Self {
                     #(#field_idents: #field_finalizers,)*
@@ -362,7 +362,7 @@ impl<'sd> TypeCheckUnorderedGenerator<'sd> {
     fn generate_type_check(&self, field: &Field) -> Option<syn::Block> {
         (!field.skip).then(|| {
             let macro_internal = self.0.struct_attrs().macro_internal_path();
-            let (constraint_lifetime, _) = self.0.constraint_lifetimes();
+            let (frame_lifetime, metadata_lifetime) = self.0.constraint_lifetimes();
             let visited_flag = Self::visited_flag_variable(field);
             let typ = field.deserialize_target();
             let cql_name_literal = field.cql_name_literal();
@@ -373,7 +373,7 @@ impl<'sd> TypeCheckUnorderedGenerator<'sd> {
             parse_quote! {
                 {
                     if !#visited_flag {
-                        <#typ as #macro_internal::DeserializeValue<#constraint_lifetime, #constraint_lifetime>>::type_check(spec.typ())
+                        <#typ as #macro_internal::DeserializeValue<#frame_lifetime, #metadata_lifetime>>::type_check(spec.typ())
                             .map_err(|err| {
                                 #macro_internal::mk_row_typck_err::<Self>(
                                     column_types_iter(),
@@ -516,7 +516,7 @@ impl<'sd> DeserializeUnorderedGenerator<'sd> {
     fn generate_deserialization(&self, column_index: usize, field: &Field) -> syn::Expr {
         assert!(!field.skip);
         let macro_internal = self.0.struct_attrs().macro_internal_path();
-        let (constraint_lifetime, _) = self.0.constraint_lifetimes();
+        let (frame_lifetime, metadata_lifetime) = self.0.constraint_lifetimes();
         let deserialize_field = Self::deserialize_field_variable(field);
         let deserializer = field.deserialize_target();
 
@@ -529,7 +529,7 @@ impl<'sd> DeserializeUnorderedGenerator<'sd> {
                 );
 
                 #deserialize_field = ::std::option::Option::Some(
-                    <#deserializer as #macro_internal::DeserializeValue<#constraint_lifetime, #constraint_lifetime>>::deserialize(col.spec.typ(), col.slice)
+                    <#deserializer as #macro_internal::DeserializeValue<#frame_lifetime, #metadata_lifetime>>::deserialize(col.spec.typ(), col.slice)
                         .map_err(|err| {
                             #macro_internal::mk_row_deser_err::<Self>(
                                 #macro_internal::BuiltinRowDeserializationErrorKind::ColumnDeserializationFailed {
@@ -557,7 +557,7 @@ impl<'sd> DeserializeUnorderedGenerator<'sd> {
 
     fn generate(&self) -> syn::ImplItemFn {
         let macro_internal = self.0.struct_attrs().macro_internal_path();
-        let (constraint_lifetime, _) = self.0.constraint_lifetimes();
+        let (frame_lifetime, metadata_lifetime) = self.0.constraint_lifetimes();
         let fields = self.0.fields();
 
         let deserialize_field_decls = fields
@@ -581,7 +581,7 @@ impl<'sd> DeserializeUnorderedGenerator<'sd> {
         parse_quote! {
             fn deserialize(
                 #[allow(unused_mut)]
-                mut row: #macro_internal::ColumnIterator<#constraint_lifetime>,
+                mut row: #macro_internal::ColumnIterator<#frame_lifetime, #metadata_lifetime>,
             ) -> ::std::result::Result<Self, #macro_internal::DeserializationError> {
 
                 // Generate fields that will serve as temporary storage