Join of types depends on order of types

[farmio]

Bug Report

When using yield from to return an Iterator of any generics mypy fails to identify the type and falls back to object unless a Union is involved.

To Reproduce

Using list[...] as example. Behavior is the same with custom Generics.

from __future__ import annotations

from typing import Any, Iterator

l_int: list[int] = [1, 2]

l_str: list[str]
l_str = ["a", "b"]

l_union: list[str] | list[bytes]
l_union = ["a", "b"]


def iter_yield() -> Iterator[list[Any]]:
    yield l_int
    yield l_str  # this passes


def iter_yield_from() -> Iterator[list[Any]]:
    reveal_type(l_int)  # builtins.list[builtins.int]
    reveal_type(l_str)  # builtins.list[builtins.str]
    reveal_type(l_union)  # Union[builtins.list[builtins.str], builtins.list[builtins.bytes]]

    yield from (
        l_int,
        l_str,
    )  # error: Incompatible types in "yield from" (actual type "object", expected type "List[Any]")

    yield from (
        l_int,
        l_str,
        l_union,
    )  # error: Incompatible types in "yield from" (actual type "object", expected type "List[Any]")

    yield from (
        l_int,
        l_union,
        l_str,
    )  # union on first or second position passes 🙃

Expected Behavior

I'd expect to infer the type to list[Any] or list[str | int] for yield from ( l_int, l_str,).

Actual Behavior

(Write what happened.)

Your Environment

• Mypy version used: 0.812
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files):

# setup.cfg
[mypy]
python_version = 3.8
strict = true
warn_unreachable = true
implicit_reexport = true
ignore_missing_imports = true

• Python version used: 3.9.1
• Operating system and version: macOS 10.13.6

[hauntsaninja]

So most of this is expected to me, since List is invariant (try using Sequence instead!).

The one bit that isn't expected to me is that the result of the type join depends on the order of the types, i.e., that the second and third reveals don't match below:

def check(l_int: list[int], l_str: list[str], l_union: list[str] | list[bytes]):
    reveal_type([l_int, l_str])
    reveal_type([l_int, l_str, l_union])
    reveal_type([l_int, l_union, l_str])

[farmio]

I'm not sure where to try to use Sequence. As stated above list is only used for example. The problem in my code occurs with custom generics.

Here is an example:

from __future__ import annotations

from typing import Any, Generic, Iterator, TypeVar

ValueType = TypeVar("ValueType")


class A(Generic[ValueType]):
    def test(self, a: ValueType) -> ValueType:
        return a


class B(A[int]):
    pass


class C(A[str]):
    pass


b: B
c: C
bc: B | C


def check() -> Iterator[A[Any]]:
    yield from (b, c, bc)  # error: Incompatible types in "yield from" (actual type "object", expected type "A[Any]")
    yield from (b, bc, c)  # passes

I just want to iterate over a couple of instances derived from A.

But my understanding of mypy is not even close enough to say if this made any difference.

[hauntsaninja]

So my recommendation would have been to make your generic class covariant (https://mypy.readthedocs.io/en/stable/generics.html#variance-of-generic-types), which in theory would make mypy infer A[object], instead of object. Unfortunately, it looks like mypy still doesn't do the join correctly; the issue for that is #5269 (and I think there's a PR open for it).

Generally speaking, mypy doesn't infer Any types automatically, since that's an easy way to lead to false negatives. I think your best option is to use # type: ignore, or to cast, e.g.: yield from (cast(A[Any], b), c, bc)

[farmio]

Thank you very much, I'll look into variance.

As a workaround

def check()  -> Iterator[A[Any]]:
    yield b
    yield c

works perfectly fine without ignore.

Or even use c: Union[C | C] which is a little bit too hacky for my taste 🙃

[hauntsaninja]

Closing as a duplicate of #3339