Error: Middleware(Middleware error: Request object is not cloneable. Are you passing a streaming body?

[egargale]

Hi and thanks for your code. I am trying to cache json responses from an REST API. I am trying both the DarkBird and CaCache Cache.Managers The API request is

pub async fn process_file_through_api(file_path: &PathBuf, client: &ClientWithMiddleware) -> Result<()> {
    //
    let url = "http://localhost:8000/general/v0/general";
    println!("Converting PDF file {}...", file_path.display());

    // Read the file content
    let file_content = fs::read(&file_path).await.unwrap();

    let form = reqwest::multipart::Form::new()
        .text("strategy", "hi_res")
        .text("languages", "eng")   
        .text("pdf_infer_table_structure", "false")
        .part("files", multipart::Part::bytes(file_content).file_name(file_path.file_name().unwrap().to_str().unwrap().to_string()));
    
    let response = client.post(url)
        .header("Accept", "application/json")
        .header("api-key", "XXXXXXX".to_string())
        .multipart(form)
        .timeout(Duration::from_secs(3 * 60))  // 3 minutes in seconds
        .send()
        .await?;

Ok(())
}

The request use reqwest::multipart to construct a form to submitted. The code works fine with reqwest without middleware but fails with both CaCache and Darkbird.

Here is the trace:

Error: Middleware(Middleware error: Request object is not cloneable. Are you passing a streaming body?

Caused by:
    Request object is not cloneable. Are you passing a streaming body?

Stack backtrace:
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\backtrace.rs:331
   3: std::backtrace::Backtrace::capture
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\backtrace.rs:297
   4: anyhow::kind::Boxed::new
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\anyhow-1.0.77\src\kind.rs:116
   5: http_cache_reqwest::from_box_error
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\http-cache-reqwest-0.12.0\src\lib.rs:193
   6: core::ops::function::FnOnce::call_once<enum2$<reqwest_middleware::error::Error> (*)(alloc::boxed::Box<dyn$<core::error::Error,core::marker::Send,core::marker::Sync>,alloc::alloc::Global>),tuple$<alloc::boxed::Box<dyn$<core::error::Error,core::marker::Send
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\core\src\ops\function.rs:250
   7: enum2$<core::result::Result<tuple$<>,alloc::boxed::Box<dyn$<core::error::Error,core::marker::Send,core::marker::Sync>,alloc::alloc::Global> > >::map_err<tuple$<>,alloc::boxed::Box<dyn$<core::error::Error,core::marker::Send,core::marker::Sync>,alloc::alloc
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\core\src\result.rs:829
   8: http_cache_reqwest::impl$2::handle::async_block$0<http_cache::managers::cacache::CACacheManager>
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\http-cache-reqwest-0.12.0\src\lib.rs:214
   9: core::future::future::impl$1::poll<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,enum2$<core::result::Result<reqwest::async_impl::response::Response,enum2$<reqwest_middleware::error::Error> > > > >,core::marker::Send>,alloc::alloc::Glo
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\core\src\future\future.rs:125
  10: core::future::future::impl$1::poll<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,enum2$<core::result::Result<reqwest::async_impl::response::Response,enum2$<reqwest_middleware::error::Error> > > > >,core::marker::Send>,alloc::alloc::Glo
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\core\src\future\future.rs:125
  11: reqwest_middleware::client::impl$1::execute_with_extensions::async_fn$0
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\reqwest-middleware-0.2.4\src\client.rs:160
  12: reqwest_middleware::client::impl$4::send::async_fn$0
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\reqwest-middleware-0.2.4\src\client.rs:314
  13: testbirdhttp::process_file_through_api::async_fn$0
             at .\src\main.rs:94
  14: testbirdhttp::scan_folder::async_fn$0
             at .\src\main.rs:129
  15: testbirdhttp::main::async_block$0
             at .\src\main.rs:170
  16: tokio::runtime::park::impl$4::block_on::closure$0<enum2$<testbirdhttp::main::async_block_env$0> >
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\park.rs:282
  17: tokio::runtime::coop::with_budget
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\coop.rs:107
  18: tokio::runtime::coop::budget
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\coop.rs:73
  19: tokio::runtime::park::CachedParkThread::block_on<enum2$<testbirdhttp::main::async_block_env$0> >
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\park.rs:282
  20: tokio::runtime::context::blocking::BlockingRegionGuard::block_on<enum2$<testbirdhttp::main::async_block_env$0> >
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\context\blocking.rs:66
  21: tokio::runtime::scheduler::multi_thread::impl$0::block_on::closure$0<enum2$<testbirdhttp::main::async_block_env$0> >
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\scheduler\multi_thread\mod.rs:87
  22: tokio::runtime::context::runtime::enter_runtime<tokio::runtime::scheduler::multi_thread::impl$0::block_on::closure_env$0<enum2$<testbirdhttp::main::async_block_env$0> >,enum2$<core::result::Result<tuple$<>,enum2$<reqwest_middleware::error::Error> > > >
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\context\runtime.rs:65
  23: tokio::runtime::scheduler::multi_thread::MultiThread::block_on<enum2$<testbirdhttp::main::async_block_env$0> >
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\scheduler\multi_thread\mod.rs:86
  24: tokio::runtime::runtime::Runtime::block_on<enum2$<testbirdhttp::main::async_block_env$0> >
             at C:\Users\enric\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.35.1\src\runtime\runtime.rs:350
  25: testbirdhttp::main
             at .\src\main.rs:178
  26: core::ops::function::FnOnce::call_once<enum2$<core::result::Result<tuple$<>,enum2$<reqwest_middleware::error::Error> > > (*)(),tuple$<> >
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\core\src\ops\function.rs:250
  27: std::sys_common::backtrace::__rust_begin_short_backtrace<enum2$<core::result::Result<tuple$<>,enum2$<reqwest_middleware::error::Error> > > (*)(),enum2$<core::result::Result<tuple$<>,enum2$<reqwest_middleware::error::Error> > > >
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\std\src\sys_common\backtrace.rs:154
  28: std::rt::lang_start::closure$0<enum2$<core::result::Result<tuple$<>,enum2$<reqwest_middleware::error::Error> > > >
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\std\src\rt.rs:166
  29: std::rt::lang_start_internal::closure$2
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\rt.rs:148
  30: std::panicking::try::do_call
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\panicking.rs:504
  31: std::panicking::try
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\panicking.rs:468
  32: std::panic::catch_unwind
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\panic.rs:142
  33: std::rt::lang_start_internal
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library\std\src\rt.rs:148
  34: std::rt::lang_start<enum2$<core::result::Result<tuple$<>,enum2$<reqwest_middleware::error::Error> > > >
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962\library\std\src\rt.rs:165
  35: main
  36: invoke_main
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
  37: __scrt_common_main_seh
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
  38: BaseThreadInitThunk
  39: RtlUserThreadStart)
error: process didn't exit successfully: `target\debug\testbirdhttp.exe` (exit code: 1)

Any suggestion ?
Thanks and Happy New Year

[06chaynes]

This is on the latest version of http-cache-reqwest correct?

Also do you mind sharing the portion where you are constructing the client so I can see how it's been configured?

The request needs to be cloned, including the body, as of right now I don't have a way around that unfortunately. Im guessing that multipart form generates a streaming body which probably doesn't impl Clone.

But this should be skipping the part where the clone occurs given it's a post request and not cacheable, so I'll have to dig a bit deeper I think.

[06chaynes]

I was able to recreate this on my end and have fixed the graceful fallback logic, should be fixed with PR #116