TypedDict does not support type refinement in subclasses

[sm-Fifteen]

Are you reporting a bug, or opening a feature request?

Feature Request

Please insert below the code you are checking with mypy.

from typing_extensions import Literal, TypedDict
from typing import Optional

class Super:
    type: str

class Foo(Super):
    type: Literal['foo']

class SuperDict(TypedDict):
    type: str

class Bar(SuperDict):
    type: Literal['bar']

class SuperTuple(NamedTuple):
    type: str

class Baz(SuperTuple):
    type: Literal['baz']

What is the actual behavior/output?

Even though Literal['bar'] is compatible with type str, as shown in the other two examples, the following error is returned.

test.py:14: error: Cannot overwrite TypedDict field "type" while extending

What is the behavior/output you expect?

PEP 589 states the following:

Additional notes on TypedDict class inheritance:

• Changing a field type of a parent TypedDict class in a subclass is not allowed
• Multiple inheritance does not allow conflict types for the same name field

The current behavior of TypedDict is in-line with the specification, but that restriction prevents type refinement for subtypes, a restriction that is not shared by regular classes or NamedTuple subclasses. PEP 589 does not appear to mention why that additional restriction is given to typed dicts specifically, and this restricts how well TypedDict is able to express certain structures.

Could this limitation be lifted or is there a specific reason for it to apply?

[ilevkivskyi]

Could this limitation be lifted or is there a specific reason for it to apply?

I don't think there is any fundamental reason to prohibit this, it is just simplicity of implementation and maintenance. I think if people will ask for this, then we can reconsider. Making as low priority for now.

[ierezell]

Just a post to get information about any advancement on this ?
I'm often using TypedDicts and most of the time I feel i'm doing Hacks to make it work.

Thanks for the efforts, typing in python helps a lot !
Have a great day

[SalomonSmeke]

+1! I am working around this by using a normal class with attributes instead of a mapping, but I have to constantly hush the linter about "too many attributes".

[erictraut]

Allowing type refinement of a mutable dict key is dangerous from a type perspective and should probably remain an error. Consider the following:

def mutate_superdict(b: SuperDict):
    b["type"] = "error"

def func(b: Bar):
    b["type"] = "bar"
    mutate_superdict(b)

    # The revealed type doesn't match reality!
    reveal_type(b["type"])  # Literal["bar"]
    print(b["type"])  # "error"

func({"type": "bar"})

It's the same reason why you cannot assign a Dict[str, Literal["bar"]] to variable of type Dict[str, str].

Incidentally, pyright is consistent with mypy here and does not allow type refinements for TypedDict keys.

As for generating errors about "too many attributes", I'm failing to understand why that limitation is required. PEP 589 is mostly silent on that topic, with the exception of this statement about constructors:

Extra keys included in TypedDict object construction should also be caught.

And even here, I'm not sure why this limitation is required for type safety.

[SalomonSmeke]

Oh no. Its a quirk of my workaround, not anything related to mypy.

What you said makes sense to me, though. Im happy to take back the +1

[kkom]

@erictraut – thanks for this comment, very insightful!

Following on, doesn't this argument apply to subclassing any mutable type? E.g. the plain Foo(Super) class from the beginning of the example:

class Super:
    def __init__(self, type: str) -> None:
        self.type: str = type


class Foo(Super):
    def __init__(self, type: Literal["foo"]) -> None:
        self.type: Literal["foo"] = type  # hinting this as e.g. `int` would cause a type error (at least does in Pyright)


f = Foo(type="foo")


def mutate_super(s: Super) -> None:
    s.type = "error"


def func(f: Foo) -> None:
    f.type = "foo"
    mutate_super(f)
    # The revealed type doesn't match reality!
    reveal_type(f.type)  # Literal["bar"]
    print(f.type)  # "error"


func(Foo(type="foo"))

Is there something special about TypedDict?

Or is it simply that it's a newer addition to Python and its inheritance is being analysed more carefully than of the rest of the language?

[erictraut]

@kkom, you make a good point. This can be unsafe with regular subclassing as well. I think there are two reasons for the apparent inconsistency. The first, as you point out, is that TypedDict is a newer addition to the Python type system. The older parts of the type system needed to make pragmatic tradeoffs to accommodate standard practices in existing Python code. The second reason is that TypedDict defines a structural type (effectively, a protocol), not a nominal type. With nominal types, the author of a subclass knows which classes it derives from, so it can be implemented in a way that won't violate assumptions of those base classes. By contrast, a structural type can be applied to any class that conforms to its interface. It's more dangerous to make assumptions about implementations in this case.

[Cheaterman]

I mostly get what's being said here, but I believe changing a TypedDict's shape based on a discriminant field is a good idea? It should probably (at least) be allowed when the superclass has a discriminant field which is a union of literals, then the union can be narrowed to a single literal for each type and extra fields can be added? It's a fairly common idiom in other (typed) languages, and is often a thing in untyped Python as well.

[ldeld]

Could type refinement potentially be implemented for keys marked with ReadOnly, since the type checker will enforce that these keys are not modified?

class SuperDict(TypedDict):
    type: ReadOnly[str]

class Bar(SuperDict):
    type: Literal['bar']
    
  
def mutate_superdict(b: SuperDict):
    b["type"] = "error"  # error: ReadOnly TypedDict key "type" TypedDict is mutated