Impossible to identify asyncio.TimeoutError from other TimeoutError

[ArtsiomAntropau]

Feature or enhancement

Proposal:

Abstract

Sometimes there might be a case when a block inside async with asyncio.timeout(10) can raise TimeoutError and there are no proper way to identify one TimeoutError from another.

async with asyncio.timeout(10):
    if ...:
        raise TimeoutError

then here engineers will not be able to identify from where TimeoutError was received

Simplest example

import asyncio


async def main():
    try:
        async with asyncio.timeout(1):
            raise TimeoutError
    except asyncio.TimeoutError:  # will catch both, but should only exception from asyncio.timeout
        print('catched asyncio.TimeoutError')

asyncio.run(main())

More complex example

try:
    async with asyncio.timeout(10):
        if ...:
            raise TimeoutError
except TimeoutError:  # will catch both
    pass
except asyncio.TimeoutError:  # will catch both, but should only exception from asyncio.timeout
    pass

Proposal

asyncio should raise asyncio.TimeoutError from asyncio.exceptions and use some kind of inheritance, e.g.

class TimeoutError(builtins.TimeoutError):
    pass

then engineers will be able to use native TimeoutError from builtins as well as asyncio.TimeoutError for their try/except or isinstance checks.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-124308: Make standard library TimeoutErrors unique rather than aliases #124320

[rruuaanng]

Feature or enhancement

Proposal:

Abstract

Sometimes there might be a case when a block inside async with asyncio.timeout(10) can raise TimeoutError and there are no proper way to identify one TimeoutError from another.

async with asyncio.timeout(10):
    if ...:
        raise TimeoutError

then here engineers will not be able to identify from where TimeoutError was received

Simplest example

import asyncio


async def main():
    try:
        async with asyncio.timeout(1):
            raise TimeoutError
    except asyncio.TimeoutError:  # will catch both, but should only exception from asyncio.timeout
        print('catched asyncio.TimeoutError')

asyncio.run(main())

More complex example

try:
    async with asyncio.timeout(10):
        if ...:
            raise TimeoutError
except TimeoutError:  # will catch both
    pass
except asyncio.TimeoutError:  # will catch both, but should only exception from asyncio.timeout
    pass

Proposal

asyncio should raise asyncio.TimeoutError from asyncio.exceptions and use some kind of inheritance, e.g.

class TimeoutError(builtins.TimeoutError):
    pass

then engineers will be able to use native TimeoutError from builtins as well as asyncio.TimeoutError for their try/except or isinstance checks.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

The asyncio.TimeoutError you mentioned inherits exception, and TimeoutError inherits OSError.
Maybe you can refer to this code

import asyncio

try:
    # raise asyncio.TimeoutError
    raise TimeoutError
except (asyncio.TimeoutError, OSError) as e:
    if isinstance(e, asyncio.TimeoutError):
        print('asyncio.TimeoutError')
    elif isinstance(e, OSError):
        print('TimeoutError')

It can distinguish asyncio.TimeoutError from the built-in TimeoutError.

[ZeroIntensity]

@rruuaanng, that workaround is wrong :(

Running that on my end results in:

>>> import asyncio
>>> 
>>> try:
...     # raise asyncio.TimeoutError
...     raise TimeoutError
... except (asyncio.TimeoutError, OSError) as e:
...     if isinstance(e, asyncio.TimeoutError):
...         print('asyncio.TimeoutError')
...     elif isinstance(e, OSError):
...         print('TimeoutError')
... 
asyncio.TimeoutError

This is because asyncio.TimeoutError is exactly the same as TimeoutError (quite literally asyncio.TimeoutError is TimeoutError is true), there's no way to distinguish between the two in an except clause. As far as I can see, this was originally intentional. See this issue (cc @kumaraditya303).

The only workaround that I could think of would be to check if the traceback frame originated from asyncio, but that's less than ideal. So, I think this is a reasonable feature; there's no good workaround, and the implementation is tiny (and has basically no maintenance cost).

[rruuaanng]

@rruuaanng, that workaround is wrong :(

Running that on my end results in:

>>> import asyncio
>>> 
>>> try:
...     # raise asyncio.TimeoutError
...     raise TimeoutError
... except (asyncio.TimeoutError, OSError) as e:
...     if isinstance(e, asyncio.TimeoutError):
...         print('asyncio.TimeoutError')
...     elif isinstance(e, OSError):
...         print('TimeoutError')
... 
asyncio.TimeoutError

This is because asyncio.TimeoutError is exactly the same as TimeoutError (quite literally asyncio.TimeoutError is TimeoutError is true), there's no way to distinguish between the two in an except clause. As far as I can see, this was originally intentional. See this issue (cc @kumaraditya303).

The only workaround that I could think of would be to check if the traceback frame originated from asyncio, but that's less than ideal. So, I think this is a reasonable feature; there's no good workaround, and the implementation is tiny (and has basically no maintenance cost).

Hmm, Maybe I wrote it wrong. I thought maybe this was for compatibility reasons, since the behavior of TimeError is the same, there is really no difference.

[ArtsiomAntropau]

Previously, we’ve had

class TimeoutError(Exception):
    pass

so thats understandable why it was bad (no any connections with builtins.TimeoutError), but the way how it was refactored (TimeoutError = builtins.TimeoutError) caused another problem asyncio.TimeoutError is builtins.TimeoutError which causes inability to support TimeoutError inside async with asyncio.timeout(): block if we need differentiate them.

I’ve caught that in our production application and that really got me into a dead end. For sure option with checking context/traceback in that case … is not how engineers want to work with that.

[rruuaanng]

Previously, we’ve had

class TimeoutError(Exception):
    pass

so thats understandable why it was bad (no any connections with builtins.TimeoutError), but the way how it was refactored (TimeoutError = builtins.TimeoutError) caused another problem asyncio.TimeoutError is builtins.TimeoutError which causes inability to support TimeoutError inside async with asyncio.timeout(): block if we need differentiate them.

I’ve caught that in our production application and that really got me into a dead end. For sure option with checking context/traceback in that case … is not how engineers want to work with that.

Perhaps you can modify the interpreter exception to meet your application needs instead of using the official compiler implementation.

[kumaraditya303]

FWIW, it was a deliberate change not just for asyncio but other modules too, you can create your own custom exception for your application

[ArtsiomAntropau]

@kumaraditya303 if here (in asyncio) we will have inherited TimeoutError, then engineers will be able to handle both builtins.TimeoutError or any custom timeout error inside the block as well as asyncio.TimeoutError.

Code inside this block might use other third-party libs that can raise both their custom exceptions or builtins.TimeoutError, so exactly asyncio is responsible for this “split” and only asyncio prevents us here.

We should not and can not restrict usage of some exceptions inside asyncio.timeout block imho …

[ZeroIntensity]

TimeoutError is technically deprecated, and has been since 3.11. Unfortunately, I think we've done a pretty bad job of actually enforcing that deprecation. A quick GitHub search shows a bit over 60k people still using asyncio.TimeoutError. Similarly, typeshed does not use @deprecated on TimeoutError, so I think removing it would be the wrong approach. (Overall, I've noticed that we don't have a good way to emit deprecation warnings for exceptions, but that's something for the future.) So, if we want to do anything here, we need to un-deprecate asyncio.TimeoutError. Same goes for concurrent.futures.TimeoutError.

While we're here, aliasing to the global TimeoutError was not done for multiprocessing, so bpo-42413 is still a problem there (issubclass(multiprocessing.TimeoutError, TimeoutError) is still False on the newer versions).

I'll put together a PR addressing each of these. I think there are certainly some use cases where being able to differentiate between a timeout error raised by asyncio or something else is useful (especially considering we already have the existing asyncio.TimeoutError).

[ZeroIntensity]

I've created #124320 to make standard library TimeoutErrors unique.

[gvanrossum]

Honestly this was designed like this on purpose. I’m not sure I see the use case. I might be convinced that asyncio should add the Timeout instance as an attribute to the exception though?