diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index 88be22ce078..a245a113f67 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -1,6 +1,6 @@ PEP: 767 Title: Annotating Read-Only Attributes -Author: Eneg +Author: Łukasz Modzelewski Sponsor: Carl Meyer Discussions-To: https://discuss.python.org/t/pep-767-annotating-read-only-attributes/73408 Status: Draft @@ -9,6 +9,7 @@ Topic: Typing Created: 18-Nov-2024 Python-Version: 3.15 Post-History: `09-Oct-2024 `__ + `05-Dec-2024 `__ Abstract @@ -29,10 +30,11 @@ Motivation The Python type system lacks a single concise way to mark an attribute read-only. This feature is present in other statically and gradually typed languages -(such as `C# `_ -or `TypeScript `_), -and is useful for removing the ability to reassign or ``del``\ ete an attribute -at a type checker level, as well as defining a broad interface for structural subtyping. +(such as `C# `__ +or `TypeScript `__), +and is useful for removing the ability to externally assign to or ``del``\ ete +an attribute at a type checker level, as well as defining a broad interface +for structural subtyping. .. _classes: @@ -58,7 +60,7 @@ Today, there are three major ways of achieving read-only attributes, honored by - Overriding ``number`` is not possible - the specification of ``Final`` imposes that the name cannot be overridden in subclasses. -* read-only proxy via ``@property``:: +* marking the attribute "_internal", and exposing it via read-only ``@property``:: class Foo: _number: int @@ -70,7 +72,7 @@ Today, there are three major ways of achieving read-only attributes, honored by def number(self) -> int: return self._number - - Overriding ``number`` is possible. *Type checkers disagree about the specific rules*. [#overriding_property]_ + - Overriding ``number`` is possible, but limited to using ``@property``. [#overriding_property]_ - Read-only at runtime. [#runtime]_ - Requires extra boilerplate. - Supported by :mod:`dataclasses`, but does not compose well - the synthesized @@ -90,8 +92,7 @@ Today, there are three major ways of achieving read-only attributes, honored by - Read-only at runtime. [#runtime]_ - No per-attribute control - these mechanisms apply to the whole class. - Frozen dataclasses incur some runtime overhead. - - ``NamedTuple`` is still a ``tuple``. Most classes do not need to inherit - indexing, iteration, or concatenation. + - Most classes do not need indexing, iteration, or concatenation, inherited from ``NamedTuple``. .. _protocols: @@ -123,8 +124,10 @@ This syntax has several drawbacks: * It is somewhat verbose. * It is not obvious that the quality conveyed here is the read-only character of a property. * It is not composable with :external+typing:term:`type qualifiers `. -* Not all type checkers agree [#property_in_protocol]_ that all of the above five - objects are assignable to this structural type. +* Currently, Pyright disagrees that some of the above five objects + are assignable to this structural type. + `[Pyright] `_ + `[mypy] `_ Rationale ========= @@ -155,7 +158,7 @@ A class with a read-only instance attribute can now be defined as:: return f"Hello, {obj.name}!" * A subclass of ``Member`` can redefine ``.id`` as a writable attribute or a - :term:`descriptor`. It can also :external+typing:term:`narrow` the type. + :term:`descriptor`. It can also :external+typing:term:`narrow` its type. * The ``HasName`` protocol has a more succinct definition, and is agnostic to the writability of the attribute. * The ``greet`` function can now accept a wide variety of compatible objects, @@ -165,9 +168,12 @@ A class with a read-only instance attribute can now be defined as:: Specification ============= +Usage +----- + The :external+py3.13:data:`typing.ReadOnly` :external+typing:term:`type qualifier` becomes a valid annotation for :term:`attributes ` of classes and protocols. -It can be used at class-level or within ``__init__`` to mark individual attributes read-only:: +It can be used at class-level and within ``__init__`` to mark individual attributes read-only:: class Book: id: ReadOnly[int] @@ -176,100 +182,14 @@ It can be used at class-level or within ``__init__`` to mark individual attribut self.id = id self.name: ReadOnly[str] = name -Type checkers should error on any attempt to reassign or ``del``\ ete an attribute -annotated with ``ReadOnly``. -Type checkers should also error on any attempt to delete an attribute annotated as ``Final``. -(This is not currently specified.) - -Use of ``ReadOnly`` in annotations at other sites where it currently has no meaning -(such as local/global variables or function parameters) is considered out of scope -for this PEP. - -Akin to ``Final`` [#final_mutability]_, ``ReadOnly`` does not influence how -type checkers perceive the mutability of the assigned object. Immutable :term:`ABCs ` -and :mod:`containers ` may be used in combination with ``ReadOnly`` -to forbid mutation of such values at a type checker level: - -.. code-block:: python - - from collections import abc - from dataclasses import dataclass - from typing import Protocol, ReadOnly - - - @dataclass - class Game: - name: str - - - class HasGames[T: abc.Collection[Game]](Protocol): - games: ReadOnly[T] - - - def add_games(shelf: HasGames[list[Game]]) -> None: - shelf.games.append(Game("Half-Life")) # ok: list is mutable - shelf.games[-1].name = "Black Mesa" # ok: "name" is not read-only - shelf.games = [] # error: "games" is read-only - del shelf.games # error: "games" is read-only and cannot be deleted - - - def read_games(shelf: HasGames[abc.Sequence[Game]]) -> None: - shelf.games.append(...) # error: "Sequence" has no attribute "append" - shelf.games[0].name = "Blue Shift" # ok: "name" is not read-only - shelf.games = [] # error: "games" is read-only - +Use of bare ``ReadOnly`` (without ``[]``) is not allowed. -All instance attributes of frozen dataclasses and ``NamedTuple`` should be -implied to be read-only. Type checkers may inform that annotating such attributes -with ``ReadOnly`` is redundant, but it should not be seen as an error: - -.. code-block:: python - - from dataclasses import dataclass - from typing import NewType, ReadOnly - - - @dataclass(frozen=True) - class Point: - x: int # implicit read-only - y: ReadOnly[int] # ok, redundant - - - uint = NewType("uint", int) - - - @dataclass(frozen=True) - class UnsignedPoint(Point): - x: ReadOnly[uint] # ok, redundant; narrower type - y: Final[uint] # not redundant, Final imposes extra restrictions; narrower type - -.. _init: - -Initialization --------------- - -Assignment to a read-only attribute can only occur in the class declaring the attribute. -There is no restriction to how many times the attribute can be assigned to. -Depending on the kind of the attribute, they can be assigned to at different sites: - -Instance Attributes -''''''''''''''''''' - -Assignment to an instance attribute must be allowed in the following contexts: - -* In ``__init__``, on the instance received as the first parameter (likely, ``self``). -* In ``__new__``, on instances of the declaring class created via a call - to a super-class' ``__new__`` method. -* At declaration in the body of the class. - -Additionally, a type checker may choose to allow the assignment: +Type checkers should error on any attempt to *externally mutate* an attribute +annotated with ``ReadOnly``. -* In ``__new__``, on instances of the declaring class, without regard - to the origin of the instance. - (This choice trades soundness, as the instance may already be initialized, - for the simplicity of implementation.) -* In ``@classmethod``\ s, on instances of the declaring class created via - a call to the class' or super-class' ``__new__`` method. +We define "externally" here as occurring outside the body of the class declaring +the attribute, or its subclasses. +"Mutate" means to assign to or ``del``\ ete the attribute. .. code-block:: python @@ -289,8 +209,7 @@ Additionally, a type checker may choose to allow the assignment: self.songs = list(songs) # multiple assignments are fine def clear(self) -> None: - # error: assignment to read-only "songs" outside initialization - self.songs = [] + self.songs = [] # ok band = Band(name="Bôa", songs=["Duvet"]) @@ -299,10 +218,6 @@ Additionally, a type checker may choose to allow the assignment: band.songs.append("Twilight") # ok: list is mutable - class SubBand(Band): - def __init__(self) -> None: - self.songs = [] # error: cannot assign to a read-only attribute of a base class - .. code-block:: python # a simplified immutable Fraction class @@ -333,61 +248,76 @@ Additionally, a type checker may choose to allow the assignment: self.numerator, self.denominator = f.as_integer_ratio() return self -Class Attributes -'''''''''''''''' -Read-only class attributes are attributes annotated as both ``ReadOnly`` and ``ClassVar``. -Assignment to such attributes must be allowed in the following contexts: +It should also be error to delete an attribute annotated as ``Final``. +(This is not currently specified.) -* At declaration in the body of the class. -* In ``__init_subclass__``, on the class object received as the first parameter (likely, ``cls``). +Use of ``ReadOnly`` in annotations at other sites where it currently has no meaning +(such as local/global variables or function parameters) is considered out of scope +for this PEP. + +``ReadOnly`` does not influence the mutability of the attribute's value. Immutable +protocols and :mod:`collections ` may be used in combination +with ``ReadOnly`` to forbid mutation of those values at a type checker level: .. code-block:: python - class URI: - protocol: ReadOnly[ClassVar[str]] = "" + from collections import abc + from dataclasses import dataclass + from typing import Protocol, ReadOnly - def __init_subclass__(cls, protocol: str = "") -> None: - cls.protocol = protocol - class File(URI, protocol="file"): ... + @dataclass + class Game: + name: str + + + class HasGames[T: abc.Collection[Game]](Protocol): + games: ReadOnly[T] -When a class-level declaration has an initializing value, it can serve as a `flyweight `_ -default for instances: + + def add_games(shelf: HasGames[list[Game]]) -> None: + shelf.games.append(Game("Half-Life")) # ok: list is mutable + shelf.games[-1].name = "Black Mesa" # ok: "name" is not read-only + shelf.games = [] # error: "games" is read-only + del shelf.games # error: "games" is read-only and cannot be deleted + + + def read_games(shelf: HasGames[abc.Sequence[Game]]) -> None: + # shelf.games.append(...) error, "Sequence" has no "append"! + shelf.games[0].name = "Blue Shift" # ok: "name" is not read-only + shelf.games = [] # error: "games" is read-only + + +All instance attributes of frozen dataclasses and ``NamedTuple`` should be +implied to be read-only. Type checkers may inform that annotating such attributes +with ``ReadOnly`` is redundant, but it should not be seen as an error: .. code-block:: python - class Patient: - number: ReadOnly[int] = 0 + from dataclasses import dataclass + from typing import NewType, ReadOnly - def __init__(self, number: int | None = None) -> None: - if number is not None: - self.number = number -.. note:: - This feature conflicts with :data:`~object.__slots__`. An attribute with - a class-level value cannot be included in slots, effectively making it a class variable. + @dataclass(frozen=True) + class Point: + x: int # implicit read-only + y: ReadOnly[int] # ok, redundant -Type checkers may choose to warn on read-only attributes which could be left uninitialized -after an instance is created (except in :external+typing:term:`stubs `, -protocols or ABCs):: - class Patient: - id: ReadOnly[int] # error: "id" is not initialized on all code paths - name: ReadOnly[str] # error: "name" is never initialized + uint = NewType("uint", int) - def __init__(self) -> None: - if random.random() > 0.5: - self.id = 123 + @dataclass(frozen=True) + class UnsignedPoint(Point): + x: ReadOnly[uint] # ok, redundant; narrower type + y: Final[uint] # not redundant, Final imposes extra restrictions; narrower type - class HasName(Protocol): - name: ReadOnly[str] # ok Subtyping --------- -The inability to reassign read-only attributes makes them covariant. +The inability to externally mutate read-only attributes makes them covariant. This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: * Read-only attributes can be redeclared as writable attributes, descriptors @@ -406,7 +336,7 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: game = Game(title="DOOM", year=1993) game.year = 1994 - game.title = "DOOM II" # ok: attribute is not read-only + game.title = "DOOM II" # ok: attribute is no longer read-only class TitleProxy(HasTitle): @@ -419,17 +349,13 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: * If a read-only attribute is not redeclared, it remains read-only:: + @dataclass class Game(HasTitle): year: int - def __init__(self, title: str, year: int) -> None: - super().__init__(title) - self.title = title # error: cannot assign to a read-only attribute of base class - self.year = year - - game = Game(title="Robot Wants Kitty", year=2010) game.title = "Robot Wants Puppy" # error: "title" is read-only + game.year = 2012 # ok * Subtypes can :external+typing:term:`narrow` the type of read-only attributes:: @@ -464,8 +390,9 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: def pprint(self) -> None: print(self.foo, self.bar, self.baz) -* In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural - subtype must support ``.name`` access, and the returned value is assignable to ``T``:: +* In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that values + that inhabit the protocol must support ``.name`` access, and the returned value + is assignable to ``T``:: class HasName(Protocol): name: ReadOnly[str] @@ -493,6 +420,15 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: has_name = NamedClassVar() has_name = NamedDescriptor() + Type checkers should not assume that access to a protocol's read-only attributes + is supported by the protocol's type (``type[HasName]``). Even if an attribute + exists on the protocol's type, no assumptions should be made about its type. + + Accurately modeling the behavior and type of ``type[HasName].name`` is difficult, + therefore it was left out from this PEP to reduce its complexity; + future enhancements to the typing specification may refine this behavior. + + Interaction with Other Type Qualifiers -------------------------------------- @@ -513,10 +449,25 @@ Interaction with Other Type Qualifiers This is consistent with the interaction of ``ReadOnly`` and :class:`typing.TypedDict` defined in :pep:`705`. -An attribute cannot be annotated as both ``ReadOnly`` and ``Final``, as the two -qualifiers differ in semantics, and ``Final`` is generally more restrictive. -``Final`` remains allowed as an annotation of attributes that are only implied -to be read-only. It can be also used to redeclare a ``ReadOnly`` attribute of a base class. +Read-only class attributes can be *internally* assigned to in the same places +a normal class variable can: + +.. code-block:: python + + class URI: + protocol: ReadOnly[ClassVar[str]] = "" + + def __init_subclass__(cls, protocol: str = "") -> None: + cls.protocol = protocol + + class File(URI, protocol="file"): ... + + URI.protocol = "http" # error: "protocol" is read-only + +``Final`` attributes are implicitly read-only. Annotating an attribute as both +``Final`` and ``ReadOnly`` is redundant and should be flagged as such by type checkers. +``Final`` may be used to override both implicit and explicit read-only attributes +of a base class. Backwards Compatibility @@ -557,8 +508,8 @@ following the footsteps of :pep:`705#how-to-teach-this`: `type qualifiers `_ section: The ``ReadOnly`` type qualifier in class attribute annotations indicates - that the attribute of the class may be read, but not reassigned or ``del``\ eted. - For usage in ``TypedDict``, see `ReadOnly `_. + that outside of the class, the attribute may be read but not assigned to + or ``del``\ eted. For usage in ``TypedDict``, see `ReadOnly `_. Rejected Ideas @@ -575,58 +526,44 @@ quality of such properties. This PEP makes ``ReadOnly`` a better alternative for defining read-only attributes in protocols, superseding the use of properties for this purpose. +Assignment Only in ``__init__`` and Class Scope +----------------------------------------------- -Assignment Only in ``__init__`` and Class Body ----------------------------------------------- - -An earlier version of this PEP proposed that read-only attributes could only be -assigned to in ``__init__`` and the class' body. A later discussion revealed that -this restriction would severely limit the usability of ``ReadOnly`` within -immutable classes, which typically do not define ``__init__``. - -:class:`fractions.Fraction` is one example of an immutable class, where the -initialization of its attributes happens within ``__new__`` and classmethods. -However, unlike in ``__init__``, the assignment in ``__new__`` and classmethods -is potentially unsound, as the instance they work on can be sourced from -an arbitrary place, including an already finalized instance. +An earlier version of this PEP specified that internal mutation of read-only +attributes could only happen in ``__init__`` and at class-level. This was done +to follow suit the specification of C#'s `readonly `__. -We find it imperative that this type checking feature is useful to the foremost -use site of read-only attributes - immutable classes. Thus, the PEP has changed -since to allow assignment in ``__new__`` and classmethods under a set of rules -described in the :ref:`init` section. +Later revision of this PEP loosened the restriction to also include ``__new__``, +``__init_subclass__`` and ``@classmethod``\ s, as it was revealed that the initial +version would severely limit the usability of ``ReadOnly`` within immutable classes, +which typically do not define ``__init__``. +Further revision removed this restriction entirely, as it turned out unnecessary +to achieve soundness of the effects of ``ReadOnly`` as described in this PEP. +In turn, this allowed to simplify the PEP, and should reduce the complexity +of type checker implementations. -Open Issues -=========== +Allowing Bare ``ReadOnly`` With Initializing Value +-------------------------------------------------- -Extending Initialization ------------------------- +An earlier version of this PEP allowed the use of bare ``ReadOnly`` when the attribute +being annotated had an initializing value. The type of the attribute was supposed +to be determined by type checkers using their usual type inference rules. -Mechanisms such as :func:`dataclasses.__post_init__` or attrs' `initialization hooks `_ -augment object creation by providing a set of special hooks which are called -during initialization. +`This thread `_ +surfaced a few non-trivial issues with this feature, like undesirable inference +of ``Literal[...]`` from literal values, differences in type checker inference rules, +or complexity of implementation due to class-level and ``__init__``-level assignments. +We decided to always require a type for ``ReadOnly[...]``, as *explicit is better than implicit*. -The current initialization rules defined in this PEP disallow assignment to -read-only attributes in such methods. It is unclear whether the rules could be -satisfyingly shaped in a way that is inclusive of those 3rd party hooks, while -upkeeping the invariants associated with the read-only-ness of those attributes. - -The Python type system has a long and detailed `specification `_ -regarding the behavior of ``__new__`` and ``__init__``. It is rather unfeasible -to expect the same level of detail from 3rd party hooks. - -A potential solution would involve type checkers providing configuration in this -regard, requiring end users to manually specify a set of methods they wish -to allow initialization in. This however could easily result in users mistakenly -or purposefully breaking the aforementioned invariants. It is also a fairly -big ask for a relatively niche feature. Footnotes ========= .. [#overriding_property] Pyright in strict mode disallows non-property overrides. - Mypy does not impose this restriction and allows an override with a plain attribute. + Mypy permits an override with a plain attribute. + Non-property overrides are technically unsafe, as they may break class-level ``Foo.number`` access. `[Pyright playground] `_ `[mypy playground] `_ @@ -634,15 +571,6 @@ Footnotes This PEP focuses solely on the type-checking behavior. Nevertheless, it should be desirable the name is read-only at runtime. -.. [#property_in_protocol] - Pyright disallows class variable and non-property descriptor overrides. - `[Pyright] `_ - `[mypy] `_ - `[Pyre] `_ - -.. [#final_mutability] - As noted above the second-to-last code example of https://typing.python.org/en/latest/spec/qualifiers.html#semantics-and-examples - Copyright =========