PEP 728: Address feedback from Carl (python#4393)

* Address feedback from Carl

Discussion:
  https://discuss.python.org/t/pep-728-typeddict-with-typed-extra-items/45443/153

* Address review feedback

* Update peps/pep-0728.rst

---------

Co-authored-by: Carl Meyer <[email protected]>

Author: PIG208

peps/pep-0728.rst

@@ -36,8 +36,8 @@ The current behavior of TypedDict prevents users from defining a
 TypedDict type when it is expected that the type contains no extra items.
 
 Due to the possible presence of extra items, type checkers cannot infer more
-precise return types for ``.items()`` and ``.values()`` on a TypedDict. This can
-also be resolved by
+precise return types for ``.items()`` and ``.values()`` on a TypedDict.
+This can be resolved by
 `defining a closed TypedDict type <https://github.com/python/mypy/issues/7981>`__.
 
 Another possible use case for this is a sound way to
@@ -126,12 +126,11 @@ that the old typing behavior can be supported in combination with ``Unpack``.
 Rationale
 =========
 
-A type that allows extra items of type ``str`` on a TypedDict can be loosely
-described as the intersection between the TypedDict and ``Mapping[str, str]``.
+Suppose we want a type that allows extra items of type ``str`` on a TypedDict.
 
 `Index Signatures
 <https://www.typescriptlang.org/docs/handbook/2/objects.html#index-signatures>`__
-in TypeScript achieve this:
+in TypeScript allow this:
 
 .. code-block:: typescript
 
@@ -140,9 +139,8 @@ in TypeScript achieve this:
         [key: string]: string
     }
 
-This proposal aims to support a similar feature without introducing general
-intersection of types or syntax changes, offering a natural extension to the
-existing assignability rules.
+This proposal aims to support a similar feature without syntax changes,
+offering a natural extension to the existing assignability rules.
 
 We propose to add a class parameter ``extra_items`` to TypedDict.
 It accepts a :term:`typing:type expression` as the argument; when it is present,
@@ -510,12 +508,13 @@ checks::
 
     details: MovieWithYear = {"name": "Kill Bill Vol. 1", "year": 2003}
     movie: Movie = details  # Not OK. 'year' is not required in 'Movie',
-                            # so it shouldn't be required in 'MovieWithYear' either
+                            # but it is required in 'MovieWithYear'
 
-Because ``'year'`` is absent in ``Movie``, ``extra_items`` is considered the
-corresponding key. ``'year'`` being required violates this rule:
+where ``MovieWithYear`` (B) is not assignable to ``Movie`` (A)
+according to this rule:
 
-    * For each required key in ``A``, the corresponding key is required in ``B``.
+    * For each non-required key in ``A``, if the item is not read-only in ``A``,
+      the corresponding key is not required in ``B``.
 
 When ``extra_items`` is specified to be read-only on a TypedDict type, it is
 possible for an item to have a :term:`narrower <typing:narrow>` type than the
@@ -606,9 +605,6 @@ still holds true.
     Operations with arbitrary str keys (instead of string literals or other
     expressions with known string values) should generally be rejected.
 
-This means that indexed accesses and assignments with arbitrary keys can still
-be rejected even when ``extra_items`` is specified.
-
 Operations that already apply to ``NotRequired`` items should generally also
 apply to extra items, following the same rationale from the `typing spec
 <https://typing.python.org/en/latest/spec/typeddict.html#supported-and-unsupported-operations>`__:
@@ -617,9 +613,10 @@ apply to extra items, following the same rationale from the `typing spec
     cases potentially unsafe operations may be accepted if the alternative is to
     generate false positive errors for idiomatic code.
 
-Some operations are allowed due to the TypedDict being
-:term:`typing:assignable` to ``Mapping[str, VT]`` or ``dict[str, VT]``.
-The two following sections will expand on that.
+Some operations, including indexed accesses and assignments with arbitrary str keys,
+may be allowed due to the TypedDict being :term:`typing:assignable` to
+``Mapping[str, VT]`` or ``dict[str, VT]``. The two following sections will expand
+on that.
 
 Interaction with Mapping[str, VT]
 ---------------------------------
@@ -628,8 +625,8 @@ A TypedDict type is :term:`typing:assignable` to a type of the form ``Mapping[st
 when all value types of the items in the TypedDict
 are assignable to ``VT``. For the purpose of this rule, a
 TypedDict that does not have ``extra_items=`` or ``closed=`` set is considered
-to have an item with a value of type ``object``. This extends the current
-assignability rule from the `typing spec
+to have an item with a value of type ``ReadOnly[object]``. This extends the
+current assignability rule from the `typing spec
 <https://typing.python.org/en/latest/spec/typeddict.html#assignability>`__.
 
 For example::
@@ -647,12 +644,26 @@ For example::
     int_mapping: Mapping[str, int] = extra_int  # Not OK. 'int | str' is not assignable with 'int'
     int_str_mapping: Mapping[str, int | str] = extra_int  # OK
 
-Type checkers should be able to infer the precise return types of ``values()``
-and ``items()`` on such TypedDict types::
+Type checkers should infer the precise signatures of ``values()`` and ``items()``
+on such TypedDict types::
+
+    def foo(movie: MovieExtraInt) -> None:
+        reveal_type(movie.items())  # Revealed type is 'dict_items[str, str | int]'
+        reveal_type(movie.values())  # Revealed type is 'dict_values[str, str | int]'
+
+By extension of this assignability rule, type checkers may allow indexed accesses
+with arbitrary str keys when ``extra_items`` or ``closed=True`` is specified.
+For example::
+
+    def bar(movie: MovieExtraInt, key: str) -> None:
+        reveal_type(movie[key])  # Revealed type is 'str | int'
+
+.. _pep728-type-narrowing:
 
-    def fun(movie: MovieExtraStr) -> None:
-        reveal_type(movie.items())  # Revealed type is 'dict_items[str, str]'
-        reveal_type(movie.values())  # Revealed type is 'dict_values[str, str]'
+Defining the type narrowing behavior for TypedDict is out-of-scope for this PEP.
+This leaves flexibility for a type checker to be more/less restrictive about
+indexed accesses with arbitrary str keys. For example, a type checker may opt
+for more restriction by requiring an explicit ``'x' in d`` check.
 
 Interaction with dict[str, VT]
 ------------------------------
@@ -687,20 +698,32 @@ For example::
     regular_dict: dict[str, int] = not_required_num_dict  # OK
     f(not_required_num_dict)  # OK
 
-In this case, methods that are previously unavailable on a TypedDict are allowed::
+In this case, methods that are previously unavailable on a TypedDict are allowed,
+with signatures matching ``dict[str, VT]``
+(e.g.: ``__setitem__(self, key: str, value: VT) -> None``)::
 
-    not_required_num.clear()  # OK
+    not_required_num_dict.clear()  # OK
 
-    reveal_type(not_required_num.popitem())  # OK. Revealed type is tuple[str, int]
+    reveal_type(not_required_num_dict.popitem())  # OK. Revealed type is 'tuple[str, int]'
 
-However, ``dict[str, VT]`` is not necessarily assignable to a TypedDict type,
+    def f(not_required_num_dict: IntDictWithNum, key: str):
+      not_required_num_dict[key] = 42  # OK
+      del not_required_num_dict[key]  # OK
+
+:ref:`Notes on indexed accesses <pep728-type-narrowing>` from the previous section
+still apply.
+
+``dict[str, VT]`` is not assignable to a TypedDict type,
 because such dict can be a subtype of dict::
 
     class CustomDict(dict[str, int]):
         pass
 
-    not_a_regular_dict: CustomDict = {"num": 1}
-    int_dict: IntDict = not_a_regular_dict  # Not OK
+    def f(might_not_be_a_builtin_dict: dict[str, int]):
+        int_dict: IntDict = might_not_be_a_builtin_dict # Not OK
+
+    not_a_builtin_dict: CustomDict = {"num": 1}
+    f(not_a_builtin_dict)
 
 Runtime behavior
 ----------------