CancelledError raised on failed requests when on a loop inside asyncio.timeout

[marcoffee]

It looks like httpx is not handling asyncio.Timeout cancellation when run inside a loop.
I was trying to replace asyncio.wait_for with asyncio.timeout on async requests with retries and suddenly I started getting asyncio.CancelledErrors instead of the expected asyncio.TimeoutError

Here is a minimal code snippet that causes such error here:

import asyncio
import httpx


async def repro () -> None:
    try:
        async with asyncio.timeout(10):
            while True:
                try:
                    async with httpx.AsyncClient() as cli:
                        await cli.get("https://httpstat.us/200?sleep=5000", timeout=2)

                except Exception as exc:
                    print(repr(exc))

    except BaseException as exc:
        print(repr(exc))

with asyncio.Runner() as runner:
    runner.run(repro())

Here, it prints:

ReadTimeout('')
ReadTimeout('')
ReadTimeout('')
CancelledError()

Which is unexpected, since the asyncio.timeout decorator should convert the CancelledError to a TimeoutError.
I tampered a little with the asyncio.Timeout class code and found out that, when it tries to uncancel the running task, it already has 7 cancelations attempts.
Then, I tampered a little more and discovered that those cancellation attempts (two per cli.get call) are being triggered inside anyio. Here is a stack trace that I printed every time the Task.cancel was called.

 File "<ipython-input-1-506dcbbecc9d>", line 23, in <module>
    runner.run(repro2())
  File "<PYTHON-LOCATION>/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
  File "<PYTHON-LOCATION>/asyncio/base_events.py", line 640, in run_until_complete
    self.run_forever()
  File "<PYTHON-LOCATION>/asyncio/base_events.py", line 607, in run_forever
    self._run_once()
  File "<PYTHON-LOCATION>/asyncio/base_events.py", line 1919, in _run_once
    handle._run()
  File "<PYTHON-LOCATION>/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "<PYTHON-LOCATION>/asyncio/tasks.py", line 353, in __wakeup
    self.__step()
  File "<PYTHON-LOCATION>/asyncio/tasks.py", line 270, in __step
    result = coro.send(None)
  File "<PYTHON-LOCATION>/site-packages/anyio/_core/_sockets.py", line 171, in try_connect
    tg.cancel_scope.cancel()
  File "<PYTHON-LOCATION>/site-packages/anyio/_backends/_asyncio.py", line 480, in cancel
    self._deliver_cancellation()
  File "<PYTHON-LOCATION>/site-packages/anyio/_backends/_asyncio.py", line 435, in _deliver_cancellation
    task.cancel()

Here is the code that prints such trace:

import asyncio
import httpx
import traceback


class MyTask (asyncio.Task):
    def cancel (self, *args, **kwargs) -> bool:
        if self.get_name() == "repro-task":
            print("cancellation attempt")
            traceback.print_stack()

        return super().cancel(*args, **kwargs)


async def repro () -> None:
    try:
        async with asyncio.timeout(10):
            while True:
                try:
                    async with httpx.AsyncClient() as cli:
                        await cli.get("https://httpstat.us/200?sleep=5000", timeout=2)

                except Exception as exc:
                    print(repr(exc))

    except BaseException as exc:
        print(repr(exc))

async def repro2 () -> None:
    await MyTask(repro(), name="repro-task")

with asyncio.Runner() as runner:
    runner.run(repro2())

Maybe this is an issue with some internal httpx async call, maybe this is an issue with anyio itself.
If it is indeed from httpx, I would like to open an issue.

[nikicat]

I've encountered this error as well. @marcoffee, have you determined the cause of the error and found a solution or workaround for it?

[nikicat]

I believe this PR fixes the error.

[lovelydinosaur]

Have you tried the equivalent in trio to see what behaviour you get then?

[nikicat]

No.

[lovelydinosaur]

Probably a useful bit of info, since it would isolate if this issue is specific to asyncio.

[nikicat]

I think it's specific to Python 3.11 asyncio as only there exists a Task.uncancel() method. But I'm not fully sure that trio isn't affected too.

[lovelydinosaur]

Can confirm that trio works okay (which I'd expect)...

import trio
import httpx


async def repro () -> None:
    try:
        with trio.fail_after(10):
            while True:
                try:
                    async with httpx.AsyncClient() as cli:
                        await cli.get("https://httpstat.us/200?sleep=5000", timeout=2)

                except Exception as exc:
                    print(repr(exc))

    except BaseException as exc:
        print(repr(exc))


trio.run(repro)

Output:

python ./example.py 
ReadTimeout('')
ReadTimeout('')
ReadTimeout('')
ReadTimeout('')
TooSlowError()