Dictionary unpacking with object that doesn't inherit from collections.abc.Mapping

[bryanforbes]

While writing the type stubs for asyncpg, I've come across an issue that I can't quite determine how to fix. asyncpg.Record is a C class that implements all of the methods of collections.abc.Mapping and works at runtime with the ** operator, and I have typed it in a pyi file as such. However both mypy and pyright flag an error when using it with **. It seems that the only way to use an object with ** according to mypy and pyright is if the object inherits from collections.abc.Mapping:

from __future__ import annotations

from collections.abc import ItemsView, Iterator, KeysView, Mapping, ValuesView
from typing import Any


class MyMapping(Mapping[str, Any]):
    def __getitem__(self, __key: str) -> str:
        ...

    def __len__(self) -> int:
        ...

    def __iter__(self) -> Iterator[Any]:
        ...


class Record:
    def __getitem__(self, __key: str) -> str:
        ...

    def get(self, __key: str, default: Any) -> Any:
        ...

    def items(self) -> ItemsView[str, str]:
        ...

    def keys(self) -> KeysView[str]:
        ...

    def values(self) -> ValuesView[str]:
        ...

    def __contains__(self, __o: object) -> bool:
        ...

    def __len__(self) -> int:
        ...

    def __iter__(self) -> Iterator[str]:
        ...


def some_func(**kwargs: Any) -> None:
    ...


m = MyMapping()
r = Record()


some_func(**m)
some_func(**r)  # Error: Argument after ** must be a mapping, not "Record"

Is this a bug in the current type checking logic of mypy and pyright? Or is there another solution I'm not thinking of?

[erictraut]

To my knowledge, there is no protocol class defined for the concept of Mapping[K, V]. The class typing.Mapping is not a protocol, probably for historical reasons (since it predates PEP 544).

I see a couple of potential solutions:

1. You could "lie" in the asyncpg stubs and indicate that Record subclasses from typing.Mapping[str, str].
2. We could introduce a new protocol in typeshed, perhaps _Mapping[K, V] or _UnpackableMapping[K, V], that defines the interface for a generic mapping class. Then type checkers could use this protocol definition when applying the dictionary unpack operator.

[AlexWaygood]

The class typing.Mapping is not a protocol, probably for historical reasons (since it predates PEP 544).

Many ABCs in collections.abc/typing were converted to protocols in typeshed after PEP 544 was accepted, but several (including Mapping, Sequence, AbstractSet, etc) were deliberately not converted to protocols. I believe this is because they represent complex interfaces where it is important to keep certain methods consistent with each other, in ways that are hard to fully express using structural typing. As such, only the simpler ABCs were converted to become protocols.

[bryanforbes]

I'd be happy to add such a protocol to typeshed if it leads to more precise type-checking. stdlib/_typeshed/__init__.pyi would be the place to put it.

Thanks, @AlexWaygood. Would you like me to create a PR? And which name that @erictraut proposed would you suggest?

[AlexWaygood]

It actually looks like we already have the necessary protocol in typeshed, @erictraut. SupportsKeysAndGetItem describes the minimal interface required for an object to be considered an unpackable mapping by the Python runtime:

>>> class Foo:
...     def __init__(self):
...         self.data = {}
...     def __getitem__(self, key):
...         return self.data[key]
...     def keys(self):
...         return self.data.keys()
...
>>> {**Foo()}
{}

[erictraut]

Thanks Alex. I must have learned about SupportsKeyAndGetItem at some point because pyright supports it for dictionary expansion within a dict expression (e.g. {**foo}), but it didn't support it for argument dictionary expansion (e.g. bar(**foo)). I just added the support to pyright for the latter case.

[bryanforbes]

Thanks for the feedback and help, @AlexWaygood and @erictraut! Since pyright has fixed this and I've filed an issue for mypy, I believe the answer is that this was a bug in type checking and bar(**foo) should work if foo matches the SupportsKeysAndGetItem protocol. Once this is fixed in mypy, I'll come back here, make a comment that it's been fixed, and select it as the answer.

[bryanforbes]

Now that mypy has released support for SupportsKeysAndGetItem being destructured in 1.3, the answer to this question is to upgrade mypy or pyright and it will work. Thanks for the help, @AlexWaygood and @erictraut!

[hauntsaninja]

Thank you for adding mypy support!