Implement PEP 585 (Generic builtins and __future__.annotations)

[Starwort]

• Are you reporting a bug, or opening a feature request? Reporting a bug
• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate if you try to simplify your case to a minimal repro.

from __future__ import annotations
from collections import defaultdict

squares: defaultdict[tuple, int] = defaultdict(lambda: 999999)

• What is the actual behaviour/output? MyPy tells me "defaultdict" is not subscriptable, use "typing.DefaultDict" instead
• What is the behaviour/output you expect? MyPy ignores the subscript error (and recognises it as equivalent) because of from __future__ import annotations
• What is the version of mypy and Python you are using? 0.740 on 3.8.0 (Ubuntu 18.04)
• No additional MyPy flags

[ilevkivskyi]

Looks like this should be not hard to support.

[TH3CHARLie]

@ilevkivskyi I'd like to take a shot to support this and I may need some hints

[ilevkivskyi]

@TH3CHARLie Sure, you can try this. I think the fix should contain two parts. First, whenever encountering the future import a file, you should set some flag (probably in the semantic analyzer), and second, at the point where the error is generated check for this flag, and suppress the error if it is set.

[JukkaL]

I think that this needs more discussion. Firstly, it seems pretty clear that list[int] is not a valid type, according to PEP 484 and PEP 563. Similarly, dict, set, tuple and frozenset shouldn't be subscriptable, even when using from __future__ import annotations.

However, mypy allows types such as collections.OrderedDict and collections.Iterable to be subscripted, even though they have aliases in typing and this will fail at runtime. For defaultdict and a few others (beyond the built-in collections mentioned above) require the alias in typing to be used.

Some types, such sa queue.Queue, don't have an alias in typing, and mypy allows these to be subcripted even though it doesn't work at runtime. Otherwise it would be impossible to use Queue as a generic type.

The current behavior seems inconsistent, but I'm not sure what's the desired semantics (with or without from __future__ import annotations). Thoughts?

[ilevkivskyi]

Since the possible runtime failures happen soon and are quite obvious, I would stay on the side of having some false negatives if this removes some false positives (from a user point of view).

I.e. I am fine with unconditionally allowing list[int] if there is the future import.

[JukkaL]

if this removes some false positives (from a user point of view)

In my opinion this not a false positive, since we are only following the spec.

I.e. I am fine with unconditionally allowing list[int] if there is the future import.

I think that this is such as a big change that it deserves a PEP, especially as not supporting this syntax was explicitly mentioned in PEP 563 (if I'm reading it correctly). One of the main points of PEP 484 was that the same annotation syntax would be used by all tools. I think that any new syntax we experiment with should be clearly marked as a non-standard extension, such as through enabling it via mypy_extensions or an opt-in flag. Otherwise there is a risk of user confusion, and if we set an example, it may lower the barrier for other tools to invent their own extensions as well outside the PEP process. (For example, IDEs might complain about our syntax extensions, or start using ~int for optional types, etc.)

A potential way to experiment with this would be to allow list[int] only after from mypy_extensions import list.

[ilevkivskyi]

In my opinion this not a false positive, since we are only following the spec.

Sure, there is your opinion, but there is also a user opinion. Seriously what is the point of prohibiting list[int] with the future import apart from annoying people? In fact:

• It works at runtime (in most cases)
• Mypy already understands it well
• There is zero ambiguity about what it means
• Some people expect this to work in mypy
• Pyre and pytype already accept list[int] (PyCharm doesn't however)
• It is used in typeshed

IMO the main point of PEP 585 is to actually enable list[int] at runtime. There is big difference between ~int and list[int], the former is indeed an experimental syntax, while the latter is something everyone always wanted but didn't have because of runtime limitations.

Anyway, I am not going to argue about this, I don't think it is something really important.

[JukkaL]

I agree that supporting list[int] etc. could be quite nice. I'll try to find time to write a PEP about this.

[Starwort]

Since this thread seems stagnant, I'll post this:
PEP 585 is all about parametrisation of standard collections. In fact, it poses this as an example:

from __future__ import annotations

def find(haystack: dict[str, list[int]]) -> int:
    ...

IMO this shows absolute support for parametrising generics in this way, and additionally, this was written in March - so no need to write a new PEP :P

[JukkaL]

PEP 585 hasn't been accepted yet, though. Allowing list[int] (and so on) only in type annotations that aren't evaluated at runtime could be done separately.