refactor(db): implement macro-based versioning system for database types

Introduces a declarative macro system to simplify adding new database versions. The new `versioned_type!` macro automatically generates versioned enums with all necessary trait implementations, reducing boilerplate from hundreds of lines to just a few lines per version.

Key changes:
- Add `versioned_type!` macro for generating versioned enums with automatic Compress/Decompress implementations
- Migrate existing V6/V7 transaction and block types to use the new macro system
- Add comprehensive documentation and examples for adding new versions
- Include tests to verify backward compatibility and round-trip serialization

This makes database format changes much more manageable - adding a new version now only requires:
1. Incrementing CURRENT_DB_VERSION
2. Adding one line to the versioned type declaration
3. Defining only the types that changed

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

Author: kariy

crates/storage/db/src/models/versioned/README.md

@@ -0,0 +1,173 @@
+# Database Versioning System
+
+This module provides a macro-based versioning system for database types to ensure backward compatibility when the primitive types change.
+
+## Problem
+
+When primitive types in `katana-primitives` change, it can break the database format, making databases created with previous Katana versions incompatible. The versioning system ensures that:
+
+1. The database is aware of format changes
+2. Old data can still be deserialized correctly
+3. New versions can be added with minimal boilerplate
+
+## Solution: Macro-Based Versioning
+
+The `versioned_type!` macro automatically generates versioned enums with all necessary trait implementations.
+
+## Usage
+
+### Basic Setup
+
+To create a versioned type, use the `versioned_type!` macro:
+
+```rust
+use crate::versioned_type;
+
+versioned_type! {
+    VersionedTx {
+        V6 => v6::Tx,
+        V7 => katana_primitives::transaction::Tx,
+    }
+}
+```
+
+This automatically generates:
+- The versioned enum with all variants
+- `From` trait implementations for conversions
+- `Compress` and `Decompress` implementations with fallback chain
+- Conversion to/from the latest version
+
+### Adding a New Version
+
+When the primitive types change in a breaking way:
+
+1. **Update the database version** in `crates/storage/db/src/version.rs`:
+   ```rust
+   pub const CURRENT_DB_VERSION: Version = Version::new(8); // Increment version
+   ```
+
+2. **Create a new version module** (e.g., `v7.rs`) that contains only the types that changed:
+   ```rust
+   // crates/storage/db/src/models/versioned/transaction/v7.rs
+   use serde::{Deserialize, Serialize};
+   
+   #[derive(Debug, Clone, Serialize, Deserialize)]
+   pub struct NewFieldType {
+       pub new_field: u64,
+       // ... other fields
+   }
+   
+   // Implement conversion to the current primitive type
+   impl From<NewFieldType> for katana_primitives::NewFieldType {
+       fn from(v7: NewFieldType) -> Self {
+           // Handle conversion
+       }
+   }
+   ```
+
+3. **Update the versioned type declaration**:
+   ```rust
+   versioned_type! {
+       VersionedTx {
+           V6 => v6::Tx,
+           V7 => v7::Tx,
+           V8 => katana_primitives::transaction::Tx,  // Latest version
+       }
+   }
+   ```
+
+That's it! The macro handles all the boilerplate.
+
+## How It Works
+
+### Serialization
+- New data is always serialized using the latest version variant
+- The versioned enum wrapper ensures version information is preserved
+
+### Deserialization
+The `Decompress` implementation tries deserialization in this order:
+1. First, as the versioned enum itself (for data that was already versioned)
+2. Then, as the latest version type (for recent unversioned data)
+3. Finally, falling back through older versions in reverse order
+
+This ensures maximum compatibility with both old and new data formats.
+
+### Conversions
+- `From<LatestType>` creates a versioned enum with the latest variant
+- `From<VersionedType>` converts any version to the latest type
+- Each old version module must implement conversion to the current types
+
+## Best Practices
+
+1. **Only define changed types**: In version modules, only include types that actually changed
+2. **Preserve field order**: When possible, maintain the same field order for serialization compatibility
+3. **Document changes**: Add comments explaining what changed in each version
+4. **Test thoroughly**: Add tests for round-trip serialization and cross-version compatibility
+
+## Example: Complete Version Addition
+
+Here's a complete example of adding V8 when a field type changes:
+
+```rust
+// 1. Create v7.rs with the old type definition
+// crates/storage/db/src/models/versioned/transaction/v7.rs
+mod v7 {
+    use serde::{Deserialize, Serialize};
+    
+    // This is how ResourceBounds looked in V7
+    #[derive(Debug, Clone, Serialize, Deserialize)]
+    pub struct ResourceBounds {
+        pub max_amount: u64,
+        pub max_price: u64,
+    }
+    
+    // Conversion to the new format
+    impl From<ResourceBounds> for katana_primitives::ResourceBounds {
+        fn from(v7: ResourceBounds) -> Self {
+            Self {
+                max_amount: v7.max_amount,
+                max_price_per_unit: v7.max_price, // Field renamed
+            }
+        }
+    }
+}
+
+// 2. Update the versioned type
+versioned_type! {
+    VersionedTx {
+        V6 => v6::Tx,
+        V7 => v7::Tx,  // Now points to our v7 module
+        V8 => katana_primitives::transaction::Tx,  // Latest
+    }
+}
+
+// 3. Update CURRENT_DB_VERSION to 8
+```
+
+## Testing
+
+Always test version compatibility:
+
+```rust
+#[test]
+fn test_v7_to_v8_migration() {
+    // Create V7 data
+    let v7_tx = v7::Tx { /* ... */ };
+    let versioned = VersionedTx::V7(v7_tx);
+    
+    // Convert to latest
+    let v8_tx: Tx = versioned.into();
+    
+    // Verify conversion worked correctly
+    assert_eq!(v8_tx.field, expected_value);
+}
+```
+
+## Benefits
+
+This macro-based approach reduces version addition from hundreds of lines to just:
+1. One line in the versioned type declaration
+2. A module with only the types that changed
+3. Conversion implementations for those types
+
+The rest is handled automatically!

crates/storage/db/src/models/versioned/block/mod.rs

@@ -1,63 +1,123 @@
-use katana_primitives::block::{self, Header};
-use serde::{Deserialize, Serialize};
+use katana_primitives::block::Header;
 
-use crate::codecs::{Compress, Decompress};
-use crate::error::CodecError;
+use crate::versioned_type;
 
 mod v6;
 
-#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
-#[cfg_attr(test, derive(::arbitrary::Arbitrary))]
-pub enum VersionedHeader {
-    V6(v6::Header),
-    V7(Header),
+// Use the macro to generate the versioned enum and all implementations
+versioned_type! {
+    VersionedHeader {
+        V6 => v6::Header,
+        V7 => Header,
+    }
 }
 
+// Manually implement Default for VersionedHeader since Header has Default
 impl Default for VersionedHeader {
     fn default() -> Self {
         Self::V7(Default::default())
     }
 }
 
-impl From<block::Header> for VersionedHeader {
-    fn from(header: block::Header) -> Self {
-        Self::V7(header)
+#[cfg(test)]
+mod tests {
+    use katana_primitives::block::Header;
+
+    use super::{v6, VersionedHeader};
+    use crate::codecs::{Compress, Decompress};
+
+    #[test]
+    fn test_versioned_header_v6_to_v7_conversion() {
+        let v6_header = v6::Header {
+            parent_hash: Default::default(),
+            number: 1,
+            state_diff_commitment: Default::default(),
+            transactions_commitment: Default::default(),
+            receipts_commitment: Default::default(),
+            events_commitment: Default::default(),
+            state_root: Default::default(),
+            transaction_count: 0,
+            events_count: 0,
+            state_diff_length: 0,
+            timestamp: 0,
+            sequencer_address: Default::default(),
+            l1_gas_prices: Default::default(),
+            l1_data_gas_prices: Default::default(),
+            l1_da_mode: katana_primitives::da::L1DataAvailabilityMode::Blob,
+            protocol_version: Default::default(),
+        };
+
+        let versioned = VersionedHeader::V6(v6_header.clone());
+
+        // Convert to latest version
+        let header: Header = versioned.into();
+        assert_eq!(header.number, 1);
+        // V6 doesn't have l2_gas_prices, so it should be set to MIN
+        assert_eq!(header.l2_gas_prices, katana_primitives::block::GasPrices::MIN);
     }
-}
 
-impl From<VersionedHeader> for block::Header {
-    fn from(versioned: VersionedHeader) -> Self {
-        match versioned {
-            VersionedHeader::V7(header) => header,
-            VersionedHeader::V6(header) => header.into(),
-        }
+    #[test]
+    fn test_versioned_header_v7_from_conversion() {
+        let header = Header { number: 42, ..Default::default() };
+        let versioned: VersionedHeader = header.clone().into();
+
+        assert!(matches!(versioned, VersionedHeader::V7(_)));
+
+        // Convert back
+        let recovered: Header = versioned.into();
+        assert_eq!(header, recovered);
     }
-}
 
-impl Compress for VersionedHeader {
-    type Compressed = Vec<u8>;
-    fn compress(self) -> Result<Self::Compressed, CodecError> {
-        postcard::to_stdvec(&self).map_err(|e| CodecError::Compress(e.to_string()))
+    #[test]
+    fn test_versioned_header_compress_decompress() {
+        let original = VersionedHeader::V7(Default::default());
+
+        // Compress
+        let compressed = original.clone().compress().unwrap();
+
+        // Decompress
+        let decompressed = VersionedHeader::decompress(&compressed).unwrap();
+
+        assert_eq!(original, decompressed);
     }
-}
 
-impl Decompress for VersionedHeader {
-    fn decompress<B: AsRef<[u8]>>(bytes: B) -> Result<Self, CodecError> {
-        let bytes = bytes.as_ref();
+    #[test]
+    fn test_backward_compatibility_header_decompression() {
+        // Create a V6 header
+        let v6_header = v6::Header {
+            parent_hash: Default::default(),
+            number: 99,
+            state_diff_commitment: Default::default(),
+            transactions_commitment: Default::default(),
+            receipts_commitment: Default::default(),
+            events_commitment: Default::default(),
+            state_root: Default::default(),
+            transaction_count: 0,
+            events_count: 0,
+            state_diff_length: 0,
+            timestamp: 0,
+            sequencer_address: Default::default(),
+            l1_gas_prices: Default::default(),
+            l1_data_gas_prices: Default::default(),
+            l1_da_mode: katana_primitives::da::L1DataAvailabilityMode::Blob,
+            protocol_version: Default::default(),
+        };
 
-        if let Ok(header) = postcard::from_bytes::<Self>(bytes) {
-            return Ok(header);
-        }
+        // Serialize it directly (simulating old database data)
+        let v6_bytes = postcard::to_stdvec(&v6_header).unwrap();
 
-        // Try deserializing as V7 first, then fall back to V6
-        if let Ok(header) = postcard::from_bytes::<Header>(bytes) {
-            return Ok(VersionedHeader::V7(header));
-        }
+        // Should be able to decompress as VersionedHeader
+        let versioned = VersionedHeader::decompress(&v6_bytes).unwrap();
 
-        if let Ok(header) = postcard::from_bytes::<v6::Header>(bytes) {
-            return Ok(VersionedHeader::V6(header));
+        match versioned {
+            VersionedHeader::V6(h) => assert_eq!(h.number, 99),
+            _ => panic!("Expected V6 header"),
         }
+    }
 
-        Err(CodecError::Decompress("failed to deserialize header: unknown format".to_string()))
+    #[test]
+    fn test_default_uses_latest_version() {
+        let versioned = VersionedHeader::default();
+        assert!(matches!(versioned, VersionedHeader::V7(_)));
     }
 }

crates/storage/db/src/models/versioned/example_v8.rs.example

@@ -0,0 +1,56 @@
+// Example: How to add version V8 when primitive types change
+// 
+// This file demonstrates the minimal code needed to add a new database version.
+// Rename this to v8.rs and uncomment when actually implementing V8.
+
+/*
+// Step 1: Define only the types that changed from V7 to V8
+use katana_primitives::{Felt, fee};
+use serde::{Deserialize, Serialize};
+
+// Example: ResourceBoundsMapping gained a new field in V8
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct ResourceBoundsMapping {
+    pub l1_gas: fee::ResourceBounds,
+    pub l2_gas: fee::ResourceBounds,
+    pub l3_gas: fee::ResourceBounds,  // New in V8!
+}
+
+// Provide conversion to the current primitive type
+impl From<ResourceBoundsMapping> for fee::ResourceBoundsMapping {
+    fn from(v8: ResourceBoundsMapping) -> Self {
+        // Handle the conversion appropriately
+        // This is just an example - adapt to your actual types
+        Self::All(v8.l1_gas, v8.l2_gas, v8.l3_gas)
+    }
+}
+
+// If the whole transaction structure changed, define it here
+// Otherwise, you can reuse types from katana_primitives for unchanged parts
+use katana_primitives::transaction::{InvokeTxV0, InvokeTxV1, /* ... */};
+
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct InvokeTxV3 {
+    // ... fields with the new ResourceBoundsMapping
+    pub resource_bounds: ResourceBoundsMapping,
+    // ... other fields
+}
+
+// ... Rest of the type definitions that changed
+
+// Step 2: Update the version declaration in the parent module:
+// In transaction/mod.rs:
+// versioned_type! {
+//     VersionedTx {
+//         V6 => v6::Tx,
+//         V7 => v7::Tx,
+//         V8 => v8::Tx,  // Add this line
+//         V9 => katana_primitives::transaction::Tx,  // Latest is now V9
+//     }
+// }
+
+// Step 3: Update CURRENT_DB_VERSION in crates/storage/db/src/version.rs:
+// pub const CURRENT_DB_VERSION: Version = Version::new(9);
+
+// That's it! The macro handles everything else automatically.
+*/