GET request with reqwest hangs indefinitely when exposed via PyO3

[ChristopherRabotin]

Hi there,

I have a structure called MetaFile whose sole job is to download the provided URI and store it locally, using blocking in reqwest (which specifically does not use, or at least explicitly use, async).

It works great when called from Rust. Here is the problem: when calling exactly the same function from Python, the call just hangs indefinitely.

The exact code I'm working with is here: https://github.com/nyx-space/anise/blob/7b9b0d926fcb331a256500b8ad0e586e0bda81c2/anise/src/almanac/meta.rs#L193 .

Here is the relevant Rust code:

                                        let client = reqwest::blocking::Client::builder()
                                            .connect_timeout(Duration::from_secs(30))
                                            .timeout(Duration::from_secs(30))
                                            .build()
                                            .unwrap();

                                        println!("built");

                                        match client.get(url.clone()).send() {
                                            Ok(resp) => {
                                                println!("resp");
                                                if resp.status().is_success() {
// ...

When called from Python, it blocks forever on the start of this match.

>>> MetaFile("http://google.com/robots.txt").process()
built

Any idea what could be happening here?

Thanks

P.S.: This is a cross-post with seanmonstar/reqwest#2084 .

[adamreichold]

I cannot readily pinpoint this in the linked code, but the most common source of deadlocks in PyO3-based extension is the GIL which is automatically held when calling into #[pymethod]. Could you try wrapping your whole method body using Python::allow_threads, e.g.

fn process(&mut self, py: Python) -> Result<(), MetaAlmanacError> {
  py.allow_threads(|| { /* same code as before */ })
}

[ChristopherRabotin]

You're a magician! Thank you, that worked out of the box!