Custom Annotated class with default metadata

[vitalik]

Hello

I'm bumping my head trying to develop a custom Annotated class...

Annotated allows to add some metadata to type hint that can be checked at runtime:

from typing import Annotated

some: Annotated[int, 'must be even']

so metadata is always required - but I want to develop a similar type that initialises metadata with some default value:

some: MyAnnotated[int] # <- this must now be equal to Annotated[int, '<default-meta>']

I'm able to make it work with this code:

from typing import Generic, TypeVar, Annotated, Any

T = TypeVar('T')


class MyAnnotated(Generic[T]):
    @classmethod
    def __class_getitem__(cls, param: Any) -> T:
        if isinstance(param, tuple):
            return Annotated[param[0], param[1]]  # type: ignore
        return Annotated[param, '<default-meta>']  # type: ignore


assert MyAnnotated[int, 'my-meta'] == Annotated[int, 'my-meta']
assert MyAnnotated[int] == Annotated[int, '<default-meta>']

this works as expected - but VScode (pylance) does not understand it and cannot add autocompletion:

while works fine with default Annotated class:

is there a way to fool pylance somehow ?

The high level idea is to make some type container that allows to set default:

T = TypeVar("T")
Vec: TypeAlias = Annotated[T, MaxLen(10)]


v1: Vec[int]              # <-- will have meta max-len = 10
v2: Vec[int, MaxLen(20))  # <-- will have meta max-len = 20 - but this example fails with validation error

[hmc-cs-mdrissi]

I tried a couple approaches and all look forbidden. The approach that's closest that I could imagine working would be this,

class MyAnnotated:
    def __class_getitem__(cls, key: tuple[type[T], Unpack[tuple[object, ...]]] | type[T]) -> T:
        ...

Main issue is type checkers do not support custom class_getitem magic and I think this would likely require a pep (or some discussion at least). I think this could open some type level operators and that would be possible justification more then 1 argument Annotated. The current definition is very close to just having First operator that takes a tuple of types and returns First type. Similar trick could be used for taking callable and getting it's return type. Most languages lack type level operators although some functional ones have it (haskell and idris being two).

Your use case that requires supporting both a type passed by itself + tuple[type, other stuff] is also awkward to express in type system. While possible it takes strange usage of variadic generics to do it. That part by itself,

def f(x: tuple[type[T], Unpack[tuple[object, ...]]] | type[T]) -> T:
    ...


reveal_type(f(int))  # Shows int
reveal_type(f((int, "hello", 4.0)))  # Shows int

does work fine in pyright. Mypy gets first one right, but gives an error with second case (a bug to be filed). Mypy's PEP 646 implementation is still in progress.

An alternative likely easier route is a feature request to allow normal Annotated to take just one argument. Then you could just do,

if TYPE_CHECKING:
  MyAnnotated = Annotated
else:
  # Your implementation that adds metadata by default

[vitalik]

@hmc-cs-mdrissi thank you

if TYPE_CHECKING:

Yeah, I was looking into this - but then it gives another problem - annotated with default is not validating:

do you see any way how to fool mypy/pylance into thinking that MyAnnotated is a special container that allows one or two arguments ?

[hmc-cs-mdrissi]

No. My explorations focused on type checking side and I don’t see a way to make a custom type like annotated with 1 argument allowed. A generic with 1/2 arguments is easy with type var tuple but it’s also not helpful as it won’t behave right in that you want first argument to be treated like main type (how Annotated works). Maybe there is some trick I’m missing but I think supporting both 1/2 argument similar to Annotated will require some change to be done to each type checker + a discussion/maybe pep.

Supporting custom class getitem would allow it although it’s overkill for this one use case and likely need pep/constraints. I think easiest option would be to propose to allow Annnoated with only 1 argument. That may not need a pep but still needs agreement from cpython + various type checkers that it’s reasonable.

[gandhis1]

It's not quite the same thing, but regarding the general concept of Annotated that accepts 1 argument - I wonder if using NewType would be a better approach. But that will not embed any metadata inside of it, outside of the name of the type itself.