From 393506cc109b75f46a35208fd5a0bb35c2a43946 Mon Sep 17 00:00:00 2001 From: Alice Purcell Date: Wed, 8 Nov 2023 12:54:02 +0000 Subject: [PATCH 1/4] Clarifications --- peps/pep-0705.rst | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/peps/pep-0705.rst b/peps/pep-0705.rst index f45afb47fa8..321ec3f2985 100644 --- a/peps/pep-0705.rst +++ b/peps/pep-0705.rst @@ -21,7 +21,8 @@ Abstract :pep:`589` defines the structural type :class:`~typing.TypedDict` for dictionaries with a fixed set of keys. As ``TypedDict`` is a mutable type, it is difficult to correctly annotate methods which accept read-only parameters in a way that doesn't prevent valid inputs. -This PEP proposes a new type qualifier, ``typing.ReadOnly``, to support these usages. + +This PEP proposes a new type qualifier, ``typing.ReadOnly``, to support these usages. It makes no Python grammar changes. Correct usage of read-only keys of TypedDicts is intended to be enforced only by static type checkers, and will not be enforced by Python itself at runtime. Motivation ========== @@ -125,7 +126,7 @@ It is possible to work around this issue with generics (as of Python 3.11), but Rationale ========= -These problems can be resolved by removing the ability to update one or more of the items in a ``TypedDict``. This does not mean the items are immutable: a reference to the underlying dictionary could still exist with a different but compatible type in which those items have mutator operations. As such, these are not "final" items; using this term would risk confusion with final attributes, which are fully immutable. These items are "read-only", and we introduce a new ``typing.ReadOnly`` type qualifier for this purpose. +These problems can be resolved by removing the ability to update one or more of the items in a ``TypedDict``. This does not mean the items are immutable: a reference to the underlying dictionary could still exist with a different but compatible type in which those items have mutator operations. These items are "read-only", and we introduce a new ``typing.ReadOnly`` type qualifier for this purpose. The ``movie_string`` function in the first motivating example can then be typed as follows:: @@ -324,7 +325,7 @@ In addition to existing type checking rules, type checkers should error if a Typ a2: A = { "x": 3, "y": 4 } a1.update(a2) # Type check error: "x" is read-only in A -Unless the declared value is of bottom type:: +Unless the declared value is of bottom type (:class:`~typing.Never`):: class B(TypedDict): x: NotRequired[typing.Never] @@ -333,6 +334,8 @@ Unless the declared value is of bottom type:: def update_a(a: A, b: B) -> None: a.update(b) # Accepted by type checker: "x" cannot be set on b +Note: Nothing will ever match the ``Never`` type, so an item annotated with it must be absent. + Keyword argument typing ----------------------- @@ -368,18 +371,18 @@ There are no known security consequences arising from this PEP. How to teach this ================= -Suggestion for changes to the :mod:`typing` module, in line with current practice: +Suggested changes to the :mod:`typing` module documentation, in line with current practice: * Add this PEP to the others listed. * Add ``typing.ReadOnly``, linked to TypedDict and this PEP. * Add the following text to the TypedDict entry: -Individual items can be excluded from mutate operations using ReadOnly, allowing them to be read but not changed. This is useful when the exact type of the value is not known yet, and so modifying it would break structural subtypes. *insert example* +The ReadOnly type qualifier indicates that an item declared in a TypedDict definition may be read but not mutated (added, modified or removed). This is useful when the exact type of the value is not known yet, and so modifying it would break structural subtypes. *insert example* Reference implementation ======================== -pyright 1.1.332 fully implements this proposal. +`pyright 1.1.333 fully implements this proposal `_. Rejected alternatives ===================== @@ -399,6 +402,15 @@ Calling the type ``Readonly`` ``Read-only`` is generally hyphenated, and it appears to be common convention to put initial caps onto words separated by a dash when converting to CamelCase. This appears consistent with the definition of CamelCase on Wikipedia: CamelCase uppercases the first letter of each word. That said, Python examples or counter-examples, ideally from the core Python libraries, or better explicit guidance on the convention, would be greatly appreciated. +Reusing the ``Final`` annotation +-------------------------------- + +The :class:`~typing.Final` annotation prevents an attribute from being modified, like the proposed ``ReadOnly`` qualifier does for TypedDict items. However, it is also documented as preventing redefinition in subclasses too; from :pep:`591`: + + The ``typing.Final`` type qualifier is used to indicate that a variable or attribute should not be reassigned, redefined, or overridden. + +This does not fit with the intended use of ReadOnly. Rather than introduce confusion by having ``Final`` behave differently in different contexts, we chose to introduce a new qualifier. + A readonly flag --------------- @@ -422,7 +434,7 @@ However, this led to confusion when inheritance was introduced:: b: B = { "key1": 1, "key2": 2 } b["key1"] = 4 # Accepted by type checker: "key1" is not read-only -It would be reasonable for someone familiar with ``frozen``, on seeing just the definition of B, to assume that the whole type was read-only. On the other hand, it would be reasonable for someone familiar with ``total`` to assume that read-only only applies to the current type. +It would be reasonable for someone familiar with ``frozen`` (from :mod:`dataclasses`), on seeing just the definition of B, to assume that the whole type was read-only. On the other hand, it would be reasonable for someone familiar with ``total`` to assume that read-only only applies to the current type. The original proposal attempted to eliminate this ambiguity by making it both a type check and a runtime error to define ``B`` in this way. This was still a source of surprise to people expecting it to work like ``total``. From f86c343e7deedce43c9738f6c2ea47817748165e Mon Sep 17 00:00:00 2001 From: Alice Purcell Date: Wed, 8 Nov 2023 13:43:19 +0000 Subject: [PATCH 2/4] Define runtime behavior changes --- peps/pep-0705.rst | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/peps/pep-0705.rst b/peps/pep-0705.rst index 321ec3f2985..033b8296d6d 100644 --- a/peps/pep-0705.rst +++ b/peps/pep-0705.rst @@ -357,6 +357,30 @@ Keyword argument typing fn: Function = impl # Accepted by type checker: function signatures are identical +Runtime behavior +---------------- + +``TypedDict`` types will gain two new attributes, ``__readonly_keys__`` and ``__mutable_keys__``, which will be frozensets containing all read-only and non-read-only keys, respectively:: + + class Example(TypedDict): + a: int + b: ReadOnly[int] + c: int + d: ReadOnly[int] + + assert Example.__readonly_keys__ == frozenset({'b', 'd'}) + assert Example.__mutable_keys__ == frozenset({'a', 'c'}) + +``typing.get_type_hints`` will strip out any ``ReadOnly`` type qualifiers, unless ``include_extras`` is ``True``:: + + assert get_type_hints(Example)['b'] == int + assert get_type_hints(Example, include_extras=True)['b'] == ReadOnly[int] + +``typing.get_origin`` and ``typing.get_args`` will be updated to recognize ``ReadOnly``:: + + assert get_origin(ReadOnly[int]) is ReadOnly + assert get_args(ReadOnly[int]) == (int,) + Backwards compatibility ======================= From 068903278edb1e2a6645a1a6a9c0fb198129e4df Mon Sep 17 00:00:00 2001 From: Alice Date: Wed, 8 Nov 2023 17:53:41 +0000 Subject: [PATCH 3/4] Fix typing.Never link Co-authored-by: Hugo van Kemenade --- peps/pep-0705.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0705.rst b/peps/pep-0705.rst index 033b8296d6d..76cf799adb0 100644 --- a/peps/pep-0705.rst +++ b/peps/pep-0705.rst @@ -325,7 +325,7 @@ In addition to existing type checking rules, type checkers should error if a Typ a2: A = { "x": 3, "y": 4 } a1.update(a2) # Type check error: "x" is read-only in A -Unless the declared value is of bottom type (:class:`~typing.Never`):: +Unless the declared value is of bottom type (:data:`~typing.Never`):: class B(TypedDict): x: NotRequired[typing.Never] From ca0e1a395613b18feb9a7c2918dea6d53093eac8 Mon Sep 17 00:00:00 2001 From: Alice Date: Tue, 14 Nov 2023 18:06:53 -0500 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: Hugo van Kemenade --- peps/pep-0705.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/peps/pep-0705.rst b/peps/pep-0705.rst index 76cf799adb0..222da40e298 100644 --- a/peps/pep-0705.rst +++ b/peps/pep-0705.rst @@ -401,7 +401,7 @@ Suggested changes to the :mod:`typing` module documentation, in line with curren * Add ``typing.ReadOnly``, linked to TypedDict and this PEP. * Add the following text to the TypedDict entry: -The ReadOnly type qualifier indicates that an item declared in a TypedDict definition may be read but not mutated (added, modified or removed). This is useful when the exact type of the value is not known yet, and so modifying it would break structural subtypes. *insert example* +The ``ReadOnly`` type qualifier indicates that an item declared in a ``TypedDict`` definition may be read but not mutated (added, modified or removed). This is useful when the exact type of the value is not known yet, and so modifying it would break structural subtypes. *insert example* Reference implementation ======================== @@ -429,11 +429,11 @@ Calling the type ``Readonly`` Reusing the ``Final`` annotation -------------------------------- -The :class:`~typing.Final` annotation prevents an attribute from being modified, like the proposed ``ReadOnly`` qualifier does for TypedDict items. However, it is also documented as preventing redefinition in subclasses too; from :pep:`591`: +The :class:`~typing.Final` annotation prevents an attribute from being modified, like the proposed ``ReadOnly`` qualifier does for ``TypedDict`` items. However, it is also documented as preventing redefinition in subclasses too; from :pep:`591`: The ``typing.Final`` type qualifier is used to indicate that a variable or attribute should not be reassigned, redefined, or overridden. -This does not fit with the intended use of ReadOnly. Rather than introduce confusion by having ``Final`` behave differently in different contexts, we chose to introduce a new qualifier. +This does not fit with the intended use of ``ReadOnly``. Rather than introduce confusion by having ``Final`` behave differently in different contexts, we chose to introduce a new qualifier. A readonly flag ---------------