PEP 705: Clarifications, and define runtime behavior changes (#3526)

Co-authored-by: Hugo van Kemenade <[email protected]>

Author: alicederyn

peps/pep-0705.rst

@@ -21,7 +21,8 @@ Abstract
 
 :pep:`589` defines the structural type :class:`~typing.TypedDict` for dictionaries with a fixed set of keys.
 As ``TypedDict`` is a mutable type, it is difficult to correctly annotate methods which accept read-only parameters in a way that doesn't prevent valid inputs.
-This PEP proposes a new type qualifier, ``typing.ReadOnly``, to support these usages.
+
+This PEP proposes a new type qualifier, ``typing.ReadOnly``, to support these usages. It makes no Python grammar changes. Correct usage of read-only keys of TypedDicts is intended to be enforced only by static type checkers, and will not be enforced by Python itself at runtime.
 
 Motivation
 ==========
@@ -125,7 +126,7 @@ It is possible to work around this issue with generics (as of Python 3.11), but
 Rationale
 =========
 
-These problems can be resolved by removing the ability to update one or more of the items in a ``TypedDict``. This does not mean the items are immutable: a reference to the underlying dictionary could still exist with a different but compatible type in which those items have mutator operations. As such, these are not "final" items; using this term would risk confusion with final attributes, which are fully immutable. These items are "read-only", and we introduce a new ``typing.ReadOnly`` type qualifier for this purpose.
+These problems can be resolved by removing the ability to update one or more of the items in a ``TypedDict``. This does not mean the items are immutable: a reference to the underlying dictionary could still exist with a different but compatible type in which those items have mutator operations. These items are "read-only", and we introduce a new ``typing.ReadOnly`` type qualifier for this purpose.
 
 The ``movie_string`` function in the first motivating example can then be typed as follows::
 
@@ -324,7 +325,7 @@ In addition to existing type checking rules, type checkers should error if a Typ
     a2: A = { "x": 3, "y": 4 }
     a1.update(a2)  # Type check error: "x" is read-only in A
 
-Unless the declared value is of bottom type::
+Unless the declared value is of bottom type (:data:`~typing.Never`)::
 
     class B(TypedDict):
         x: NotRequired[typing.Never]
@@ -333,6 +334,8 @@ Unless the declared value is of bottom type::
     def update_a(a: A, b: B) -> None:
         a.update(b)  # Accepted by type checker: "x" cannot be set on b
 
+Note: Nothing will ever match the ``Never`` type, so an item annotated with it must be absent.
+
 Keyword argument typing
 -----------------------
 
@@ -354,6 +357,30 @@ Keyword argument typing
 
     fn: Function = impl  # Accepted by type checker: function signatures are identical
 
+Runtime behavior
+----------------
+
+``TypedDict`` types will gain two new attributes, ``__readonly_keys__`` and ``__mutable_keys__``, which will be frozensets containing all read-only and non-read-only keys, respectively::
+
+    class Example(TypedDict):
+        a: int
+        b: ReadOnly[int]
+        c: int
+        d: ReadOnly[int]
+
+    assert Example.__readonly_keys__ == frozenset({'b', 'd'})
+    assert Example.__mutable_keys__ == frozenset({'a', 'c'})
+
+``typing.get_type_hints`` will strip out any ``ReadOnly`` type qualifiers, unless ``include_extras`` is ``True``::
+
+    assert get_type_hints(Example)['b'] == int
+    assert get_type_hints(Example, include_extras=True)['b'] == ReadOnly[int]
+
+``typing.get_origin`` and ``typing.get_args`` will be updated to recognize ``ReadOnly``::
+
+    assert get_origin(ReadOnly[int]) is ReadOnly
+    assert get_args(ReadOnly[int]) == (int,)
+
 
 Backwards compatibility
 =======================
@@ -368,18 +395,18 @@ There are no known security consequences arising from this PEP.
 How to teach this
 =================
 
-Suggestion for changes to the :mod:`typing` module, in line with current practice:
+Suggested changes to the :mod:`typing` module documentation, in line with current practice:
 
 * Add this PEP to the others listed.
 * Add ``typing.ReadOnly``, linked to TypedDict and this PEP.
 * Add the following text to the TypedDict entry:
 
-Individual items can be excluded from mutate operations using ReadOnly, allowing them to be read but not changed. This is useful when the exact type of the value is not known yet, and so modifying it would break structural subtypes. *insert example*
+The ``ReadOnly`` type qualifier indicates that an item declared in a ``TypedDict`` definition may be read but not mutated (added, modified or removed). This is useful when the exact type of the value is not known yet, and so modifying it would break structural subtypes. *insert example*
 
 Reference implementation
 ========================
 
-pyright 1.1.332 fully implements this proposal.
+`pyright 1.1.333 fully implements this proposal <https://github.com/microsoft/pyright/releases/tag/1.1.333>`_.
 
 Rejected alternatives
 =====================
@@ -399,6 +426,15 @@ Calling the type ``Readonly``
 
 ``Read-only`` is generally hyphenated, and it appears to be common convention to put initial caps onto words separated by a dash when converting to CamelCase. This appears consistent with the definition of CamelCase on Wikipedia: CamelCase uppercases the first letter of each word. That said, Python examples or counter-examples, ideally from the core Python libraries, or better explicit guidance on the convention, would be greatly appreciated.
 
+Reusing the ``Final`` annotation
+--------------------------------
+
+The :class:`~typing.Final` annotation prevents an attribute from being modified, like the proposed ``ReadOnly`` qualifier does for ``TypedDict`` items. However, it is also documented as preventing redefinition in subclasses too; from :pep:`591`:
+
+    The ``typing.Final`` type qualifier is used to indicate that a variable or attribute should not be reassigned, redefined, or overridden.
+
+This does not fit with the intended use of ``ReadOnly``. Rather than introduce confusion by having ``Final`` behave differently in different contexts, we chose to introduce a new qualifier.
+
 A readonly flag
 ---------------
 
@@ -422,7 +458,7 @@ However, this led to confusion when inheritance was introduced::
     b: B = { "key1": 1, "key2": 2 }
     b["key1"] = 4  # Accepted by type checker: "key1" is not read-only
 
-It would be reasonable for someone familiar with ``frozen``, on seeing just the definition of B, to assume that the whole type was read-only. On the other hand, it would be reasonable for someone familiar with ``total`` to assume that read-only only applies to the current type.
+It would be reasonable for someone familiar with ``frozen`` (from :mod:`dataclasses`), on seeing just the definition of B, to assume that the whole type was read-only. On the other hand, it would be reasonable for someone familiar with ``total`` to assume that read-only only applies to the current type.
 
 The original proposal attempted to eliminate this ambiguity by making it both a type check and a runtime error to define ``B`` in this way. This was still a source of surprise to people expecting it to work like ``total``.