[ty] Support typevar-specialized dynamic types in generic type aliases (#21730)

## Summary

For a type alias like the one below, where `UnknownClass` is something
with a dynamic type, we previously lost track of the fact that this
dynamic type was explicitly specialized *with a type variable*. If that
alias is then later explicitly specialized itself (`MyAlias[int]`), we
would miscount the number of legacy type variables and emit a
`invalid-type-arguments` diagnostic
([playground](https://play.ty.dev/886ae6cc-86c3-4304-a365-510d29211f85)).
```py
T = TypeVar("T")

MyAlias: TypeAlias = UnknownClass[T] | None
```
The solution implemented here is not pretty, but we can hopefully get
rid of it via astral-sh/ty#1711. Also, once we
properly support `ParamSpec` and `Concatenate`, we should be able to
remove some of this code.

This addresses many of the `invalid-type-arguments` false-positives in
astral-sh/ty#1685. With this change, there are
still some diagnostics of this type left. Instead of implementing even
more (rather sophisticated) workarounds for these cases as well, it
might be much easier to wait for full `ParamSpec`/`Concatenate` support
and then try again.

A disadvantage of this implementation is that we lose track of some
`@Todo` types and replace them with `Unknown`. We could spend more
effort and try to preserve them, but I'm unsure if this is the best use
of our time right now.

## Test Plan

New Markdown tests.

Author: sharkdp

crates/ty_python_semantic/resources/mdtest/pep613_type_aliases.md

@@ -149,6 +149,101 @@ def _(x: MyAlias):
     reveal_type(x)  # revealed: int | list[str] | set[str]
 ```
 
+## Typevar-specialized dynamic types
+
+We still recognize type aliases as being generic if a symbol of a dynamic type is explicitly
+specialized with a type variable:
+
+```py
+from typing import TypeVar, TypeAlias
+
+from unknown_module import UnknownClass  # type: ignore
+
+T = TypeVar("T")
+
+MyAlias1: TypeAlias = UnknownClass[T] | None
+
+def _(a: MyAlias1[int]):
+    reveal_type(a)  # revealed: Unknown | None
+```
+
+This also works with multiple type arguments:
+
+```py
+U = TypeVar("U")
+V = TypeVar("V")
+
+MyAlias2: TypeAlias = UnknownClass[T, U, V] | int
+
+def _(a: MyAlias2[int, str, bytes]):
+    reveal_type(a)  # revealed: Unknown | int
+```
+
+If we specialize with fewer or more type arguments than expected, we emit an error:
+
+```py
+def _(
+    # error: [invalid-type-arguments] "No type argument provided for required type variable `V`"
+    too_few: MyAlias2[int, str],
+    # error: [invalid-type-arguments] "Too many type arguments: expected 3, got 4"
+    too_many: MyAlias2[int, str, bytes, float],
+): ...
+```
+
+We can also reference these type aliases from other type aliases:
+
+```py
+MyAlias3: TypeAlias = MyAlias1[str] | MyAlias2[int, str, bytes]
+
+def _(c: MyAlias3):
+    reveal_type(c)  # revealed: Unknown | None | int
+```
+
+Here, we test some other cases that might involve `@Todo` types, which also need special handling:
+
+```py
+from typing_extensions import Callable, Concatenate, TypeAliasType
+
+MyAlias4: TypeAlias = Callable[Concatenate[dict[str, T], ...], list[U]]
+
+def _(c: MyAlias4[int, str]):
+    # TODO: should be (int, / ...) -> str
+    reveal_type(c)  # revealed: Unknown
+
+T = TypeVar("T")
+
+MyList = TypeAliasType("MyList", list[T], type_params=(T,))
+
+MyAlias5 = Callable[[MyList[T]], int]
+
+def _(c: MyAlias5[int]):
+    # TODO: should be (list[int], /) -> int
+    reveal_type(c)  # revealed: (Unknown, /) -> int
+
+K = TypeVar("K")
+V = TypeVar("V")
+
+MyDict = TypeAliasType("MyDict", dict[K, V], type_params=(K, V))
+
+MyAlias6 = Callable[[MyDict[K, V]], int]
+
+def _(c: MyAlias6[str, bytes]):
+    # TODO: should be (dict[str, bytes], /) -> int
+    reveal_type(c)  # revealed: (Unknown, /) -> int
+
+ListOrDict: TypeAlias = MyList[T] | dict[str, T]
+
+def _(x: ListOrDict[int]):
+    # TODO: should be list[int] | dict[str, int]
+    reveal_type(x)  # revealed: Unknown | dict[str, int]
+
+MyAlias7: TypeAlias = Callable[Concatenate[T, ...], None]
+
+def _(c: MyAlias7[int]):
+    # TODO: should be (int, / ...) -> None
+    reveal_type(c)  # revealed: Unknown
+```
+
 ## Imported
 
 `alias.py`:

crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md

@@ -223,7 +223,8 @@ T = TypeVar("T")
 IntAndT = TypeAliasType("IntAndT", tuple[int, T], type_params=(T,))
 
 def f(x: IntAndT[str]) -> None:
-    reveal_type(x)  # revealed: @Todo(Generic manual PEP-695 type alias)
+    # TODO: This should be `tuple[int, str]`
+    reveal_type(x)  # revealed: Unknown
 ```
 
 ### Error cases

crates/ty_python_semantic/src/types.rs

@@ -763,7 +763,7 @@ impl<'db> DataclassParams<'db> {
 #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, salsa::Update, get_size2::GetSize)]
 pub enum Type<'db> {
     /// The dynamic type: a statically unknown set of values
-    Dynamic(DynamicType),
+    Dynamic(DynamicType<'db>),
     /// The empty set of values
     Never,
     /// A specific function object
@@ -889,7 +889,10 @@ impl<'db> Type<'db> {
     }
 
     pub const fn is_unknown(&self) -> bool {
-        matches!(self, Type::Dynamic(DynamicType::Unknown))
+        matches!(
+            self,
+            Type::Dynamic(DynamicType::Unknown | DynamicType::UnknownGeneric(_))
+        )
     }
 
     pub(crate) const fn is_never(&self) -> bool {
@@ -959,7 +962,10 @@ impl<'db> Type<'db> {
 
     pub(crate) fn is_todo(&self) -> bool {
         self.as_dynamic().is_some_and(|dynamic| match dynamic {
-            DynamicType::Any | DynamicType::Unknown | DynamicType::Divergent(_) => false,
+            DynamicType::Any
+            | DynamicType::Unknown
+            | DynamicType::UnknownGeneric(_)
+            | DynamicType::Divergent(_) => false,
             DynamicType::Todo(_) | DynamicType::TodoStarredExpression | DynamicType::TodoUnpack => {
                 true
             }
@@ -1146,7 +1152,7 @@ impl<'db> Type<'db> {
         }
     }
 
-    pub(crate) const fn as_dynamic(self) -> Option<DynamicType> {
+    pub(crate) const fn as_dynamic(self) -> Option<DynamicType<'db>> {
         match self {
             Type::Dynamic(dynamic_type) => Some(dynamic_type),
             _ => None,
@@ -1160,7 +1166,7 @@ impl<'db> Type<'db> {
         }
     }
 
-    pub(crate) const fn expect_dynamic(self) -> DynamicType {
+    pub(crate) const fn expect_dynamic(self) -> DynamicType<'db> {
         self.as_dynamic().expect("Expected a Type::Dynamic variant")
     }
 
@@ -7851,14 +7857,18 @@ impl<'db> Type<'db> {
         typevars: &mut FxOrderSet<BoundTypeVarInstance<'db>>,
         visitor: &FindLegacyTypeVarsVisitor<'db>,
     ) {
+        let is_matching_typevar = |bound_typevar: &BoundTypeVarInstance<'db>| {
+            matches!(
+                bound_typevar.typevar(db).kind(db),
+                TypeVarKind::Legacy | TypeVarKind::TypingSelf | TypeVarKind::ParamSpec
+            ) && binding_context.is_none_or(|binding_context| {
+                bound_typevar.binding_context(db) == BindingContext::Definition(binding_context)
+            })
+        };
+
         match self {
             Type::TypeVar(bound_typevar) => {
-                if matches!(
-                    bound_typevar.typevar(db).kind(db),
-                    TypeVarKind::Legacy | TypeVarKind::TypingSelf | TypeVarKind::ParamSpec
-                ) && binding_context.is_none_or(|binding_context| {
-                    bound_typevar.binding_context(db) == BindingContext::Definition(binding_context)
-                }) {
+                if is_matching_typevar(&bound_typevar) {
                     typevars.insert(bound_typevar);
                 }
             }
@@ -7998,6 +8008,14 @@ impl<'db> Type<'db> {
                 }
             },
 
+            Type::Dynamic(DynamicType::UnknownGeneric(generic_context)) => {
+                for variable in generic_context.variables(db) {
+                    if is_matching_typevar(&variable) {
+                        typevars.insert(variable);
+                    }
+                }
+            }
+
             Type::Dynamic(_)
             | Type::Never
             | Type::AlwaysTruthy
@@ -8029,6 +8047,26 @@ impl<'db> Type<'db> {
         }
     }
 
+    /// Bind all unbound legacy type variables to the given context and then
+    /// add all legacy typevars to the provided set.
+    pub(crate) fn bind_and_find_all_legacy_typevars(
+        self,
+        db: &'db dyn Db,
+        binding_context: Option<Definition<'db>>,
+        variables: &mut FxOrderSet<BoundTypeVarInstance<'db>>,
+    ) {
+        self.apply_type_mapping(
+            db,
+            &TypeMapping::BindLegacyTypevars(
+                binding_context
+                    .map(BindingContext::Definition)
+                    .unwrap_or(BindingContext::Synthetic),
+            ),
+            TypeContext::default(),
+        )
+        .find_legacy_typevars(db, None, variables);
+    }
+
     /// Replace default types in parameters of callables with `Unknown`.
     pub(crate) fn replace_parameter_defaults(self, db: &'db dyn Db) -> Type<'db> {
         self.apply_type_mapping(
@@ -8177,7 +8215,7 @@ impl<'db> Type<'db> {
             Self::SpecialForm(special_form) => special_form.definition(db),
             Self::Never => Type::SpecialForm(SpecialFormType::Never).definition(db),
             Self::Dynamic(DynamicType::Any) => Type::SpecialForm(SpecialFormType::Any).definition(db),
-            Self::Dynamic(DynamicType::Unknown) => Type::SpecialForm(SpecialFormType::Unknown).definition(db),
+            Self::Dynamic(DynamicType::Unknown | DynamicType::UnknownGeneric(_)) => Type::SpecialForm(SpecialFormType::Unknown).definition(db),
             Self::AlwaysTruthy => Type::SpecialForm(SpecialFormType::AlwaysTruthy).definition(db),
             Self::AlwaysFalsy => Type::SpecialForm(SpecialFormType::AlwaysFalsy).definition(db),
 
@@ -8839,11 +8877,18 @@ pub struct DivergentType {
 impl get_size2::GetSize for DivergentType {}
 
 #[derive(Copy, Clone, Debug, Eq, Hash, PartialEq, salsa::Update, get_size2::GetSize)]
-pub enum DynamicType {
+pub enum DynamicType<'db> {
     /// An explicitly annotated `typing.Any`
     Any,
     /// An unannotated value, or a dynamic type resulting from an error
     Unknown,
+    /// Similar to `Unknown`, this represents a dynamic type that has been explicitly specialized
+    /// with legacy typevars, e.g. `UnknownClass[T]`, where `T` is a legacy typevar. We keep track
+    /// of the type variables in the generic context in case this type is later specialized again.
+    ///
+    /// TODO: Once we implement <https://github.com/astral-sh/ty/issues/1711>, this variant might
+    /// not be needed anymore.
+    UnknownGeneric(GenericContext<'db>),
     /// Temporary type for symbols that can't be inferred yet because of missing implementations.
     ///
     /// This variant should eventually be removed once ty is spec-compliant.
@@ -8862,7 +8907,7 @@ pub enum DynamicType {
     Divergent(DivergentType),
 }
 
-impl DynamicType {
+impl DynamicType<'_> {
     fn normalized(self) -> Self {
         if matches!(self, Self::Divergent(_)) {
             self
@@ -8880,11 +8925,11 @@ impl DynamicType {
     }
 }
 
-impl std::fmt::Display for DynamicType {
+impl std::fmt::Display for DynamicType<'_> {
     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
         match self {
             DynamicType::Any => f.write_str("Any"),
-            DynamicType::Unknown => f.write_str("Unknown"),
+            DynamicType::Unknown | DynamicType::UnknownGeneric(_) => f.write_str("Unknown"),
             // `DynamicType::Todo`'s display should be explicit that is not a valid display of
             // any other type
             DynamicType::Todo(todo) => write!(f, "@Todo{todo}"),

crates/ty_python_semantic/src/types/bound_super.rs

@@ -172,7 +172,7 @@ impl<'db> BoundSuperError<'db> {
 
 #[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, get_size2::GetSize)]
 pub enum SuperOwnerKind<'db> {
-    Dynamic(DynamicType),
+    Dynamic(DynamicType<'db>),
     Class(ClassType<'db>),
     Instance(NominalInstanceType<'db>),
 }

crates/ty_python_semantic/src/types/class_base.rs

@@ -18,7 +18,7 @@ use crate::types::{
 /// automatically construct the default specialization for that class.
 #[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, salsa::Update, get_size2::GetSize)]
 pub enum ClassBase<'db> {
-    Dynamic(DynamicType),
+    Dynamic(DynamicType<'db>),
     Class(ClassType<'db>),
     /// Although `Protocol` is not a class in typeshed's stubs, it is at runtime,
     /// and can appear in the MRO of a class.
@@ -62,7 +62,7 @@ impl<'db> ClassBase<'db> {
         match self {
             ClassBase::Class(class) => class.name(db),
             ClassBase::Dynamic(DynamicType::Any) => "Any",
-            ClassBase::Dynamic(DynamicType::Unknown) => "Unknown",
+            ClassBase::Dynamic(DynamicType::Unknown | DynamicType::UnknownGeneric(_)) => "Unknown",
             ClassBase::Dynamic(
                 DynamicType::Todo(_) | DynamicType::TodoUnpack | DynamicType::TodoStarredExpression,
             ) => "@Todo",