Retry mechanism broken on WASM

[darioAnongba]

Describe the bug
The retry mechanism get_with_retry does not work on the browser (wasm32-unknown-unknown) and fails with the following error:

error output:
        panicked at library/std/src/sys/pal/wasm/../unsupported/time.rs:31:9:
        time not implemented on this platform

The reason is that WebAssembly in browsers doesn’t have direct support for timing mechanisms like std::time::Instant or std::thread::sleep.

To Reproduce
Can run the tests on this repo for sync: https://github.com/darioAnongba/bdk-wasm

Expected behavior

Build environment

• BDK tag/commit:
  • bdk_wallet = { version = "1.0.0-beta.5" }
  • bdk_esplora = { version = "0.19.0", features = ["async-https"] }
• OS+version: macOS 14.7
• Rust/Cargo version: 1.79
• Rust/Cargo target: wasm32-unknown-unknown

Additional context
Using gloo-timers instead ?

use gloo_timers::future::sleep;
use std::time::Duration;

const BASE_BACKOFF_MILLIS: u64 = 100;

async fn get_with_retry(&self, url: &str) -> Result<Response, Error> {
    let mut delay = BASE_BACKOFF_MILLIS;
    let mut attempts = 0;

    loop {
        match self.client.get(url).send().await {
            Ok(resp) if attempts < self.max_retries && is_status_retryable(resp.status()) => {
                sleep(Duration::from_millis(delay)).await;
                attempts += 1;
                delay *= 2;
            }
            Ok(resp) => return Ok(resp),
            Err(e) => return Err(e), // Handle error if send fails
        }
    }
}

or 2 functions

#[cfg(not(target_arch = "wasm32"))]
async fn get_with_retry(...) {
    // Original retry logic with task::sleep
}

#[cfg(target_arch = "wasm32")]
async fn get_with_retry(...) {
    // Non-retrying version for WebAssembly
}

[ValuedMammal]

I think it is also related to bitcoindevkit/rust-esplora-client#102

[ValuedMammal]

@darioAnongba Is it possible to test syncing in bdk-wasm with this branch of bdk based on a proposed change to rust-esplora-client that makes the async client generic over the sleep implementation? https://github.com/ValuedMammal/bdk/tree/deps/esplora-client-103

[darioAnongba]

yes I will test it ASAP and let you know

[oleonardolima]

I'm not sure if it might be useful, but you can take a look at a partial implementation/usage of gloo-timers here, which builds on top of the PR ValuedMammal mentioned above, bitcoindevkit/rust-esplora-client#103.

[darioAnongba]

Thanks @oleonardolima and @ValuedMammal but sorry I tried my best but couldn't implement a Sleeper that would work...
It works in your examples but when using sync or full_scan fails with:

the method `full_scan` exists for struct `Ref<'_, AsyncClient<GlooTimersSleeper>>`, but its trait bounds were not satisfied
the following trait bounds were not satisfied:
`TimeoutFuture: Send`
which is required by `AsyncClient<GlooTimersSleeper>: EsploraAsyncExt`
`TimeoutFuture: Sync`
which is required by `AsyncClient<GlooTimersSleeper>: EsploraAsyncExt`

I wasn't able to implement a Sleeper that is Send + Sync

The simple Sleeper is:

#[derive(Debug, Clone)]
struct GlooTimersSleeper;

impl Sleeper for GlooTimersSleeper {
    type Sleep = TimeoutFuture;

    fn sleep(dur: Duration) -> Self::Sleep {
        sleep(dur)
    }
}

I tried using something JSFuture instead of gloo_timers, wrapping Sleep in Pin and also using spawn_local from wasm_bindgen_futures but nothing worked...

[ValuedMammal]

Hm yes I see that TimeoutFuture cannot be Send. I'm wondering if there's a known solution/workaround to this or if it's better to forgo the retry functionality when targeting WASM.

[darioAnongba]

I find it strange that passing your own Reqwest client to the Esplora Client cannot fix this as Reqwest does not implement an embedded retry mechanism.

[RCasatta]

One option could be to use javascript bindings, see

https://github.com/Blockstream/lwk/blob/a2246b5e821c974731bc63de5681a058fa767683/lwk_wollet/src/clients/esplora_wasm_client.rs#L537

[darioAnongba]

Guys I managed to make it work. Tried with 100 stop gap full sync on testnet Blockstream and got a lot of 429 that triggered the timer many times. Here is the code:

#[derive(Clone)]
pub struct BrowserSleeper;

impl Sleeper for BrowserSleeper {
    type Sleep = SendSyncWrapper<TimeoutFuture>;

    fn sleep(dur: Duration) -> Self::Sleep {
        SendSyncWrapper(sleep(dur))
    }
}

and the wrapper (let me know your opinion on it):

// Wrap a future that is not `Send` or `Sync` and make it `Send` and `Sync`
pub struct SendSyncWrapper<F>(pub F);

unsafe impl<F> Send for SendSyncWrapper<F> {}
unsafe impl<F> Sync for SendSyncWrapper<F> {}

impl<F> Future for SendSyncWrapper<F>
where
    F: Future,
{
    type Output = F::Output;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        // SAFETY: Since we're in a single-threaded WASM environment, this is safe.
        unsafe {
            let this = self.get_unchecked_mut();
            Pin::new_unchecked(&mut this.0).poll(cx)
        }
    }
}

What are your plans to integrate the latest version of rust-esplora-client into the next beta release ? Is there going to be a beta.6 ? Is it going into master anytime soon ?

[notmandatory]

Yes we're going to have a 1.0.0-beta.6 which should be the final release candidate for the final 1.0 and it will include an updated version of the rust-esplora-client.