Better Control over stored data format

[Craig-Macomber]

The Problem / Limitation

The existing storage APIs, like use_synced_storage do not appear to to let the user control the encoding.

I care a lot about ensuring my actual persisted data format is human readable and human editable so I have been using pretty printed JSON as my data format.

I also like to fully cleanup the local storage entry (via removeItem) when not using it.

As far as I can tell neither of these is possible with use_synced_storage as it appears to unconditionally encode using postcard, compressed with yazi then run through some custom logic to ensure its a valid string, and there is no way to express and empty/cleared state.

Proposed Solutions

I think separating the storage location from the encoding would be a good way to do this.

Something like this could work:

/// A trait for a storage backing.
///
/// Unchanged.
pub trait StorageBacking: Clone + 'static {
    /// The key type used to store data in storage
    type Key: PartialEq + Clone + Debug + Send + Sync + 'static;
    /// Gets a value from storage for the given key.
    ///
    /// Question: Does None here indicate the storage was empty, or an error?
    fn get<T: DeserializeOwned + Clone + 'static>(key: &Self::Key) -> Option<T>;
    /// Sets a value in storage for the given key
    fn set<T: Serialize + Send + Sync + Clone + 'static>(key: Self::Key, value: &T);
}

/// A trait for the persistence portion of StorageBacking.
pub trait StoragePersistence: Clone + 'static {
    /// The key type used to store data in storage
    type Key: PartialEq + Clone + Debug + Send + Sync + 'static;
    /// The type of value which can be stored.
    type Value;
    /// Gets a value from storage for the given key
    fn store(key: &Self::Key) -> Self::Value;
    /// Sets a value in storage for the given key
    fn load(key: Self::Key, value: &Self::Value);
}

/// LocalStorage stores Option<String>.
impl StoragePersistence for LocalStorage {
    type Key = String;
    type Value = Option<String>;

    fn store(key: &Self::Key) -> Self::Value {
        // Use existing logic from storage code here.
        // Use the second half of https://github.com/DioxusLabs/sdk/blob/adf59211ea3caaaa404503660693f2cd9bb44968/packages/storage/src/client_storage/web.rs#L111
        todo!()
    }

    fn load(key: Self::Key, value: &Self::Value) {
        // Use existing logic from storage code here.
        // Use the second half of https://github.com/DioxusLabs/sdk/blob/adf59211ea3caaaa404503660693f2cd9bb44968/packages/storage/src/client_storage/web.rs#L119
        todo!()
    }
}

/// New trait which can be implemented to define a data format for storage.
pub trait StorageEncoder: Clone + 'static {
    /// The type of value which can be stored.
    type Value;
    fn deserialize<T: DeserializeOwned + Clone + 'static>(loaded: &Self::Value) -> T;
    fn serialize<T: Serialize + Send + Sync + Clone + 'static>(value: &T) -> Self::Value;
}

/// A way to create a StorageEncoder out of the two layers.
///
/// I'm not sure if this is the best way to abstract that.
#[derive(Clone)]
pub struct LayeredStorage<Persistence: StoragePersistence, Encoder: StorageEncoder> {
    persistence: PhantomData<Persistence>,
    encoder: PhantomData<Encoder>,
}

/// StorageBacking for LayeredStorage.
impl<Value, P: StoragePersistence<Value = Option<Value>>, E: StorageEncoder<Value = Value>>
    StorageBacking for LayeredStorage<P, E>
{
    type Key = P::Key;

    fn get<T: DeserializeOwned + Clone + 'static>(key: &Self::Key) -> Option<T> {
        let loaded = P::store(key);
        match loaded {
            Some(t) => E::deserialize(&t),
            None => None,
        }
    }

    fn set<T: Serialize + Send + Sync + Clone + 'static>(key: Self::Key, value: &T) {
        P::load(key, &Some(E::serialize(value)));
    }
}

/// Since StorageEncoder does not provide a way to clear, implement some options
impl<Value, P: StoragePersistence<Value = Option<Value>>, E: StorageEncoder<Value = Value>>
    LayeredStorage<P, E>
{
    pub fn clear(key: P::Key) {
        P::load(key, &None);
    }

    pub fn set_or_clear<T: Serialize + Send + Sync + Clone + 'static>(
        key: P::Key,
        value: &Option<T>,
    ) {
        match value {
            Some(t) => Self::set(key, t),
            None => Self::clear(key),
        }
    }
}

#[derive(Clone)]
struct DefaultEncoder;

impl StorageEncoder for DefaultEncoder {
    type Value = String;

    fn deserialize<T: DeserializeOwned + Clone + 'static>(loaded: &Self::Value) -> T {
        // Use existing logic from storage code here.
        // Use the first half of https://github.com/DioxusLabs/sdk/blob/adf59211ea3caaaa404503660693f2cd9bb44968/packages/storage/src/client_storage/web.rs#L119
        todo!()
    }

    fn serialize<T: Serialize + Send + Sync + Clone + 'static>(value: &T) -> Self::Value {
        // Use existing logic from storage code here.
        // Use the first half of https://github.com/DioxusLabs/sdk/blob/adf59211ea3caaaa404503660693f2cd9bb44968/packages/storage/src/client_storage/web.rs#L111
        todo!()
    }
}

/// StorageBacking using default encoder: handles LocalStorage and other built in storage implementations.
impl<P: StoragePersistence<Value = Option<String>>> StorageBacking for P {
    type Key = P::Key;

    fn get<T: DeserializeOwned + Clone + 'static>(key: &Self::Key) -> Option<T> {
        LayeredStorage::<P, DefaultEncoder>::get(key)
    }

    fn set<T: Serialize + Send + Sync + Clone + 'static>(key: Self::Key, value: &T) {
        LayeredStorage::<P, DefaultEncoder>::set(key, value)
    }
}

type HumanReadableStorage<Storage: StoragePersistence> =
    LayeredStorage<Storage, HumanReadableEncoding>;

#[derive(Clone)]
struct HumanReadableEncoding;

impl StorageEncoder for HumanReadableEncoding {
    type Value = String;

    fn deserialize<T: DeserializeOwned + Clone + 'static>(loaded: &Self::Value) -> T {
        let parsed: Result<T, serde_json::Error> = serde_json::from_str(loaded);
        // This design probably needs an error handling policy better than panic.
        parsed.unwrap()
    }

    fn serialize<T: Serialize + Send + Sync + Clone + 'static>(value: &T) -> Self::Value {
        serde_json::to_string_pretty(value).unwrap()
    }
}

fn example() {
    let s =
        use_synced_storage::<HumanReadableStorage<LocalStorage>, isize>("demo".to_string(), || 0);
}

[ealmloff]

This would be nice to support. If we do support multiple data formats, the encoder trait should be generic over the type that can be encoded/decoded instead of always using the serde::Serialize + serde::Deserialize bounds so it can be used with serializaiton frameworks outside of the serde ecosystem:

pub trait StorageEncoder<T>: Clone + 'static {
    /// The type of value which can be stored.
    type Value;
    fn deserialize(loaded: &Self::Value) -> T;
    fn serialize(value: &T) -> Self::Value;
}
impl<T: serde::Serialize + serde::DeserializeOwned + Clone + Sync + Send> StorageEncoder<T> for HumanReadableEncoding {
    type Value = String;

    fn deserialize(loaded: &Self::Value) -> T {
        let parsed: Result<T, serde_json::Error> = serde_json::from_str(loaded);
        // This design probably needs an error handling policy better than panic.
        parsed.unwrap()
    }

    fn serialize(value: &T) -> Self::Value {
        serde_json::to_string_pretty(value).unwrap()
    }
}

[Craig-Macomber]

I found some time to try and implement this. I'm going to attempt to put this together into an actual PR.

[Craig-Macomber]

I attempted to implement this, and got it working, but ran into a couple issues I called out in #85