PSA: This is what happens when you hold a shared RwLock across an await in web (0.4)

[kalzoo]

...that's when RwLock causes BorrowMutError in wasm-bindgen-futures.

Writeup here to save time for anyone else who sees this. I did not find anything similar in github or discord.

Big Picture

I was looking for a way to:

• Share state among components
• ...where all read queries from that state are synchronous and fast
• ...where but where some writes to shared state may be performed within futures
• ...suggesting read-write contention for the underlying data as other components try to render
• ...on the web target
• ...without panics

In summary, I tried:

• standard use_shared_state -> saw some BorrowMutErrors, even when using with and with_mut (although, in retrospect, those also could be explained by holding the RefCell across the await)
• using std::sync::RwLock in place of RefCell (described below)
• parking_lot::RwLock instead (may be incompatible with dioxus? because of necessary wasm target features)
• (considered tokio's async RwLock but since most uses of it are in synchronous renders, I'd rather avoid it if possible)

This feels like it will be a common pattern: an app wrapping a library which both makes network calls and holds local state. The docs examples show how to make async calls, but those don't contend for a resource.

---

The error

This may be a problem in wasm-bindgen-futures, but my assumption is that the problem was how I used it. Hence a discussion in this repo rather than an issue in this repo or that one.

1. web target, running in Chrome
2. using std::sync::RwLock
3. Modified use_shared_state_provider to use_shared_rw_state_provider specifically to avoid the race conditions causing BorrowMutError and get away from RefCell and its runtime panics
4. because I have a fair amount of async callbacks, even with a minimal prototype app and a few calls, those were popping up reliably.

Unfortunately, even with RwLock around state instead of RefCell, I saw this:

Stack Trace

Uncaught RuntimeError: unreachable
    at std::sys::wasm::common::abort_internal::hd00823c3a876104b (app_bg.wasm:0xf90498)
    at std::sys::wasm::locks::rwlock::RwLock::read::h8366b1c6af59c26e (app_bg.wasm:0x965551)
    at std::sync::rwlock::RwLock<T>::read::hd0724151485c7c5a (app_bg.wasm:0xf2cce8)
    at ui_dioxus::context::rw_context::use_shared_rw_state::{{closure}}::hd1d207b62d10bfd8 (app_bg.wasm:0xcebbae)
    at core::option::Option<T>::map::he960a324f0c9fcc0 (app_bg.wasm:0xd0f8e9)
    at ui_dioxus::context::rw_context::use_shared_rw_state::h4854370aa5b4e814 (app_bg.wasm:0xe6aa45)
    at ui_dioxus::components::data::UserAvatar::CurrentUserAvatar::h315f0af6141b9ae6 (app_bg.wasm:0x127d8c)
    at <dioxus_core::any_props::VProps<P> as dioxus_core::any_props::AnyProps>::render::{{closure}}::h7a5075d130d0aaa0 (app_bg.wasm:0x6f79f7)
    at core::ops::function::FnOnce::call_once::hab680fa7b0d43242 (app_bg.wasm:0xec22a4)
    at <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h90101a48d5e16dc4 (http://localhost:8080/assets/dioxus/app_bg.wasm)
$std::sys::wasm::common::abort_internal::hd00823c3a876104b @ app_bg.wasm:0xf90498
$std::sys::wasm::locks::rwlock::RwLock::read::h8366b1c6af59c26e @ app_bg.wasm:0x965551
$std::sync::rwlock::RwLock<T>::read::hd0724151485c7c5a @ app_bg.wasm:0xf2cce8
$ui_dioxus::context::rw_context::use_shared_rw_state::{{closure}}::hd1d207b62d10bfd8 @ app_bg.wasm:0xcebbae
$core::option::Option<T>::map::he960a324f0c9fcc0 @ app_bg.wasm:0xd0f8e9
$ui_dioxus::context::rw_context::use_shared_rw_state::h4854370aa5b4e814 @ app_bg.wasm:0xe6aa45
$ui_dioxus::components::data::UserAvatar::CurrentUserAvatar::h315f0af6141b9ae6 @ app_bg.wasm:0x127d8c
$<dioxus_core::any_props::VProps<P> as dioxus_core::any_props::AnyProps>::render::{{closure}}::h7a5075d130d0aaa0 @ app_bg.wasm:0x6f79f7
$core::ops::function::FnOnce::call_once::hab680fa7b0d43242 @ app_bg.wasm:0xec22a4
$<core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h90101a48d5e16dc4 @ app_bg.wasm:0xf20cf7
$std::panicking::try::do_call::hb9ec851a9698a6fd @ app_bg.wasm:0xd74cfd
$std::panicking::try::ha0280b2a0626b83c @ app_bg.wasm:0xb83da2
$std::panic::catch_unwind::hc43cbb9578ae59a0 @ app_bg.wasm:0xf20b39
$<dioxus_core::any_props::VProps<P> as dioxus_core::any_props::AnyProps>::render::h64fdded864fb460d @ app_bg.wasm:0x4eab2d
$dioxus_core::scope_arena::<impl dioxus_core::virtual_dom::VirtualDom>::run_scope::hf70eda5a344f26b0 @ app_bg.wasm:0x2bb082
$dioxus_core::virtual_dom::VirtualDom::render_with_deadline::{{closure}}::he9473b3cb5fc7483 @ app_bg.wasm:0x58c424
$dioxus_core::virtual_dom::VirtualDom::render_immediate::ha185377da1eba3e0 @ app_bg.wasm:0x7ec7dc
$dioxus_web::run_with_props::{{closure}}::hbb708d0917d1805f @ app_bg.wasm:0x13fe20
$wasm_bindgen_futures::task::singlethread::Task::run::h63e6b8d25355d82f @ app_bg.wasm:0x9f792d
$wasm_bindgen_futures::queue::QueueState::run_all::h5f4528297c40709f @ app_bg.wasm:0x8e045a
$wasm_bindgen_futures::queue::Queue::new::{{closure}}::h8c10f31c71166eba @ app_bg.wasm:0xde9f9e
$<dyn core::ops::function::FnMut<(A,)>+Output = R as wasm_bindgen::closure::WasmClosure>::describe::invoke::hd92f9926fb751ad1 @ app_bg.wasm:0xcbedf8
__wbg_adapter_63 @ app.js:334
real @ app.js:235
app.js:542 panicked at /Users/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasm-bindgen-futures-0.4.39/src/task/singlethread.rs:102:37:
already borrowed: BorrowMutError

Stack:

Error
    at http://localhost:8080/assets/dioxus/app.js:545:17
    at logError (http://localhost:8080/assets/dioxus/app.js:252:18)
    at imports.wbg.__wbg_new_abda76e883ba8a5f (http://localhost:8080/assets/dioxus/app.js:544:62)
    at console_error_panic_hook::Error::new::hae9dbd6c2313cc34 (http://localhost:8080/assets/dioxus/app_bg.wasm:wasm-function[32268]:0xed1030)
    at console_error_panic_hook::hook_impl::h6d14d5ba407451f4 (http://localhost:8080/assets/dioxus/app_bg.wasm:wasm-function[8536]:0xadb632)
    at console_error_panic_hook::hook::he9ae8d58c518ee7e (http://localhost:8080/assets/dioxus/app_bg.wasm:wasm-function[40771]:0xf54d9b)
    at core::ops::function::Fn::call::h439817351db926d0 (http://localhost:8080/assets/dioxus/app_bg.wasm:wasm-function[35037]:0xf0138f)
    at std::panicking::rust_panic_with_hook::h62090d3fc0630473 (http://localhost:8080/assets/dioxus/app_bg.wasm:wasm-function[14992]:0xcaf073)
    at std::panicking::begin_panic_handler::{{closure}}::h823c80ef219a166d (http://localhost:8080/assets/dioxus/app_bg.wasm:wasm-function[19091]:0xd702b9)
    at std::sys_common::backtrace::__rust_end_short_backtrace::hcc7222b556723586 (http://localhost:8080/assets/dioxus/app_bg.wasm:wasm-function[46886]:0xf90048)


(anonymous) @ app.js:542
logError @ app.js:252
imports.wbg.__wbg_error_f851667af71bcfc6 @ app.js:539
$console_error_panic_hook::error::h1a9c9411aad2e813 @ app_bg.wasm:0xdaf582
$console_error_panic_hook::hook_impl::h6d14d5ba407451f4 @ app_bg.wasm:0xadb723
$console_error_panic_hook::hook::he9ae8d58c518ee7e @ app_bg.wasm:0xf54d9b
$core::ops::function::Fn::call::h439817351db926d0 @ app_bg.wasm:0xf0138f
$std::panicking::rust_panic_with_hook::h62090d3fc0630473 @ app_bg.wasm:0xcaf073
$std::panicking::begin_panic_handler::{{closure}}::h823c80ef219a166d @ app_bg.wasm:0xd702b9
$std::sys_common::backtrace::__rust_end_short_backtrace::hcc7222b556723586 @ app_bg.wasm:0xf90048
$rust_begin_unwind @ app_bg.wasm:0xf07c2b
$core::panicking::panic_fmt::hd79411a297d06dc8 @ app_bg.wasm:0xf4e7c9
$core::cell::panic_already_borrowed::h8c62ffbd80fe44ce @ app_bg.wasm:0xecb41a
$core::cell::RefCell<T>::borrow_mut::hd237d497749f5ab7 @ app_bg.wasm:0xd83e4c
$wasm_bindgen_futures::task::singlethread::Task::run::h63e6b8d25355d82f @ app_bg.wasm:0x9f780d
$wasm_bindgen_futures::queue::QueueState::run_all::h5f4528297c40709f @ app_bg.wasm:0x8e045a
$wasm_bindgen_futures::queue::Queue::new::{{closure}}::h8c10f31c71166eba @ app_bg.wasm:0xde9f9e
$<dyn core::ops::function::FnMut<(A,)>+Output = R as wasm_bindgen::closure::WasmClosure>::describe::invoke::hd92f9926fb751ad1 @ app_bg.wasm:0xcbedf8
__wbg_adapter_63 @ app.js:334
real @ app.js:235
app_bg.wasm:0xf9049c Uncaught RuntimeError: unreachable
    at __rust_start_panic (app_bg.wasm:0xf9049c)
    at rust_panic (app_bg.wasm:0xf8f184)
    at std::panicking::rust_panic_with_hook::h62090d3fc0630473 (app_bg.wasm:0xcaf0a3)
    at std::panicking::begin_panic_handler::{{closure}}::h823c80ef219a166d (app_bg.wasm:0xd702b9)
    at std::sys_common::backtrace::__rust_end_short_backtrace::hcc7222b556723586 (app_bg.wasm:0xf90048)
    at rust_begin_unwind (app_bg.wasm:0xf07c2b)
    at core::panicking::panic_fmt::hd79411a297d06dc8 (app_bg.wasm:0xf4e7c9)
    at core::cell::panic_already_borrowed::h8c62ffbd80fe44ce (app_bg.wasm:0xecb41a)
    at core::cell::RefCell<T>::borrow_mut::hd237d497749f5ab7 (app_bg.wasm:0xd83e4c)
    at wasm_bindgen_futures::task::singlethread::Task::run::h63e6b8d25355d82f (app_bg.wasm:0x9f780d)

This occurred, reliably, when I clicked a button spawning a future which used a write lock on that shared state. It acquired the lock and made some progress within the future until a different component attempts to read the lock and then panics. I would expect that other component to just block on rendering until the write is complete. Turns out, though, that there were awaits hidden within the async call I was making

The RefCell is not within dioxus but appears to be this one, panicking here

I know that there are caveats to using std::sync::Mutex in an async context to include panics. I thought I had been careful not to hold the lock across an await point. In any case this was not the behavior that I had expected (re-borrowing of a RefCell within a core library).

Relevant issue: wasm-bindgen/wasm-bindgen#2562, where a maintainer suggests a problem with the runtime. I suppose that could be the cause here, although it seems less likely within Chrome.

`use_shared_rw_state` implementation

Modified from use_shared_state to mix in use_rw. I planned to contribute a PR with this, if it's not outmoded by 0.5 and if I can solve this particular issue. There's some dead code; intent was to keep the diff minimal.

use self::error::{UseSharedRwStateError, UseSharedRwStateResult};
use dioxus::prelude::{ScopeId, ScopeState};
use std::{
    collections::HashSet,
    future::Future,
    sync::{Arc, RwLock, RwLockReadGuard, RwLockWriteGuard},
};

#[macro_export]
macro_rules! debug_location {
    () => {{
        #[cfg(debug_assertions)]
        {
            std::panic::Location::caller()
        }
        #[cfg(not(debug_assertions))]
        {
            ()
        }
    }};
}

pub mod error {
    #[derive(thiserror::Error, Debug)]
    pub enum UseSharedRwStateError {
        #[error("the RwLock for {type_name} has been poisoned")]
        Poisoned { type_name: &'static str },
    }

    pub type UseSharedRwStateResult<T> = Result<T, UseSharedRwStateError>;
}

type ProvidedState<T> = Arc<RwLock<ProvidedStateInner<T>>>;

// Tracks all the subscribers to a shared State
pub(crate) struct ProvidedStateInner<T> {
    value: T,
    notify_any: Arc<dyn Fn(ScopeId)>,
    consumers: HashSet<ScopeId>,
    gen: usize,
}

impl<T> ProvidedStateInner<T> {
    pub(crate) fn notify_consumers(&mut self) {
        self.gen += 1;
        for consumer in self.consumers.iter() {
            (self.notify_any)(*consumer);
        }
    }

    pub(crate) fn value_mut(&mut self) -> &mut T {
        &mut self.value
    }

    pub(crate) fn value(&self) -> &T {
        &self.value
    }
}

/// This hook provides some relatively light ergonomics around shared state.
///
/// It is not a substitute for a proper state management system, but it is capable enough to provide use_state - type
/// ergonomics in a pinch, with zero cost.
///
/// # Example
///
/// ```rust
/// # use dioxus::prelude::*;
/// #
/// # fn app(cx: Scope) -> Element {
/// #     render! {
/// #         Parent{}
/// #     }
/// # }
///
/// #[derive(Clone, Copy)]
/// enum Theme {
///     Light,
///     Dark,
/// }
///
/// // Provider
/// fn Parent<'a>(cx: Scope<'a>) -> Element<'a> {
///     use_shared_rw_state_provider(cx, || Theme::Dark);
///     let theme = use_shared_rw_state::<Theme>(cx).unwrap();
///
///     render! {
///         button{
///             onclick: move |_| {
///                 let current_theme = *theme.read();
///                 *theme.write() = match current_theme {
///                     Theme::Dark => Theme::Light,
///                     Theme::Light => Theme::Dark,
///                 };
///             },
///             "Change theme"
///         }
///         Child{}
///     }
/// }
///
/// // Consumer
/// fn Child<'a>(cx: Scope<'a>) -> Element<'a> {
///     let theme = use_shared_rw_state::<Theme>(cx).unwrap();
///     let current_theme = *theme.read();
///
///     render! {
///         match current_theme {
///             Theme::Dark => {
///                 "Dark mode"
///             }
///             Theme::Light => {
///                 "Light mode"
///             }
///         }
///     }
/// }
/// ```
///
/// # How it works
///
/// Any time a component calls `write`, every consumer of the state will be notified - excluding the provider.
///
/// Right now, there is not a distinction between read-only and write-only, so every consumer will be notified.
#[must_use]
pub fn use_shared_rw_state<T: 'static>(cx: &ScopeState) -> Option<&UseSharedRwState<T>> {
    let state_owner: &mut Option<UseSharedRwStateOwner<T>> = &mut *cx.use_hook(move || {
        let scope_id = cx.scope_id();
        let root = cx.consume_context::<ProvidedState<T>>()?;

        root.write().unwrap().consumers.insert(scope_id);

        let state = UseSharedRwState::new(root);
        let owner = UseSharedRwStateOwner { state, scope_id };
        Some(owner)
    });
    state_owner.as_mut().map(|s| {
        s.state.gen = s.state.inner.read().unwrap().gen;
        &s.state
    })
}

/// This wrapper detects when the hook is dropped and will unsubscribe when the component is unmounted
struct UseSharedRwStateOwner<T> {
    state: UseSharedRwState<T>,
    scope_id: ScopeId,
}

impl<T> Drop for UseSharedRwStateOwner<T> {
    fn drop(&mut self) {
        // we need to unsubscribe when our component is unmounted
        let mut root = self.state.inner.write().unwrap();
        root.consumers.remove(&self.scope_id);
    }
}

/// State that is shared between components through the context system
pub struct UseSharedRwState<T> {
    pub(crate) inner: Arc<RwLock<ProvidedStateInner<T>>>,
    gen: usize,
}

impl<T> UseSharedRwState<T> {
    fn new(inner: Arc<RwLock<ProvidedStateInner<T>>>) -> Self {
        let gen = inner.read().unwrap().gen;
        Self { inner, gen }
    }

    /// Notify all consumers of the state that it has changed. (This is called automatically when you call "write")
    pub fn notify_consumers(&self) {
        self.inner.write().unwrap().notify_consumers();
    }

    /// Try reading the shared state
    pub fn try_read(&self) -> UseSharedRwStateResult<RwLockReadGuard<'_, ProvidedStateInner<T>>> {
        match self.inner.try_read() {
            Ok(value) => Ok(value),
            Err(source) => Err(UseSharedRwStateError::Poisoned {
                type_name: std::any::type_name::<Self>(),
            }),
        }
    }

    /// Read the shared value
    pub fn read(&self) -> RwLockReadGuard<'_, ProvidedStateInner<T>> {
        match self.try_read() {
            Ok(value) => value,
            Err(message) => panic!(
                "Reading the shared state failed: {}\n({:?})",
                message, message
            ),
        }
    }

    /// Try writing the shared state
    pub fn try_write(&self) -> UseSharedRwStateResult<RwLockWriteGuard<'_, ProvidedStateInner<T>>> {
        match self.inner.try_write() {
            Ok(mut value) => {
                value.notify_consumers();
                Ok(value)
            }
            Err(source) => Err(UseSharedRwStateError::Poisoned {
                type_name: std::any::type_name::<Self>(),
            }),
        }
    }

    /// Calling "write" will force the component to re-render
    ///
    ///
    // TODO: We prevent unncessary notifications only in the hook, but we should figure out some more global lock
    pub fn write(&self) -> RwLockWriteGuard<'_, ProvidedStateInner<T>> {
        match self.try_write() {
            Ok(value) => value,
            Err(message) => panic!(
                "Writing to shared state failed: {}\n({:?})",
                message, message
            ),
        }
    }

    /// Tries writing the value without forcing a re-render
    pub fn try_write_silent(
        &self,
    ) -> UseSharedRwStateResult<RwLockWriteGuard<'_, ProvidedStateInner<T>>> {
        match self.inner.try_write() {
            Ok(value) => Ok(value),
            Err(source) => Err(UseSharedRwStateError::Poisoned {
                type_name: std::any::type_name::<Self>(),
            }),
        }
    }

    /// Writes the value without forcing a re-render
    pub fn write_silent(&self) -> RwLockWriteGuard<'_, ProvidedStateInner<T>> {
        match self.try_write_silent() {
            Ok(value) => value,
            Err(message) => panic!(
                "Writing to shared state silently failed: {}\n({:?})",
                message, message
            ),
        }
    }

    /// Take a reference to the inner value temporarily and produce a new value
    pub fn with<O>(&self, immutable_callback: impl FnOnce(&T) -> O) -> O {
        immutable_callback(&(self.read().value))
    }

    /// Very much todo
    // pub async fn with_async<O>(&self, immutable_callback: impl FnOnce(&T) -> F) -> O
    // where
    //     F: Future<Output = O>,
    // {
    //     immutable_callback(&(self.read().value)).await
    // }

    /// Take a mutable reference to the inner value temporarily and produce a new value
    pub fn with_mut<O>(&self, mutable_callback: impl FnOnce(&mut T) -> O) -> O {
        mutable_callback(&mut self.write().value)
    }
}

impl<T> Clone for UseSharedRwState<T> {
    fn clone(&self) -> Self {
        Self {
            inner: self.inner.clone(),
            gen: self.gen,
        }
    }
}

impl<T> PartialEq for UseSharedRwState<T> {
    fn eq(&self, other: &Self) -> bool {
        self.gen == other.gen
    }
}

/// Provide some state for components down the hierarchy to consume without having to drill props. See [`use_shared_rw_state`] to consume the state
///
///
/// # Example
///
/// ```rust
/// # use dioxus::prelude::*;
/// #
/// # fn app(cx: Scope) -> Element {
/// #     render! {
/// #         Parent{}
/// #     }
/// # }
///
/// #[derive(Clone, Copy)]
/// enum Theme {
///     Light,
///     Dark,
/// }
///
/// // Provider
/// fn Parent<'a>(cx: Scope<'a>) -> Element<'a> {
///     use_shared_rw_state_provider(cx, || Theme::Dark);
///     let theme = use_shared_rw_state::<Theme>(cx).unwrap();
///
///     render! {
///         button{
///             onclick: move |_| {
///                 let current_theme = *theme.read();
///                 *theme.write() = match current_theme {
///                     Theme::Dark => Theme::Light,
///                     Theme::Light => Theme::Dark,
///                 };
///             },
///             "Change theme"
///         }
///         // Children components that consume the state...
///     }
/// }
/// ```
pub fn use_shared_rw_state_provider<T: 'static>(cx: &ScopeState, f: impl FnOnce() -> T) {
    cx.use_hook(|| {
        let state: ProvidedState<T> = Arc::new(RwLock::new(ProvidedStateInner {
            value: f(),
            notify_any: cx.schedule_update_any(),
            consumers: HashSet::new(),
            gen: 0,
        }));

        cx.provide_context(state);
    });
}

Q: Is this useful to contribute? Is there a benefit to the added reliability of a RwLock over a RefCell?

The Fix

I refactored the async logic into two parts:

• Static method to perform the "slow" async work without the lock and package the result
• Synchronously ingest that result within the lock critical section

Relevant Links

• https://discord.com/channels/899851952891002890/1142892007853928628/1142892007853928628

[ealmloff]

There are a few different ways to handle the problem of holding dynamically borrowed state (RwLock and refcell) over await points:

1. Clone the state out of the lock, run the async task and then set the original state to the new state:

let state = use_ref(cx, || String::new());

// clone whatever state you need for async out of the lock
let mut current_state = state.read().clone();
cx.spawn(async move {
    // Do some async work
    sleep(100).await;
    current_state += "hello world";
    
    // And then modify the original state synchronously
    *state.write() = current_state;
});

2. Borrow the state on only when polling the future: https://github.com/DioxusLabs/dioxus/blob/master/packages/html/src/eval.rs#L78-L94

[ealmloff]

Q: Is this useful to contribute? Is there a benefit to the added reliability of a RwLock over a RefCell?

The git version of dioxus uses signals for local, context, and global state management. Signals can be optionally Send + Sync:
https://github.com/DioxusLabs/dioxus/blob/master/packages/signals/examples/send.rs. In the git version of dioxus, you can just create a send signal and provide and consume it with the context API without a special shared state hook

RefCell and RwLock work very similarly. RwLock is just a send version of RefCell. Both can have issues with race conditions in async tasks because they both perform borrow checking at runtime