latest pytest crashes on import on python 3.5.0

[njsmith]

We have a CI job that tests on python 3.5.0, because 3.5.0 had a lot of issues with both async/await and typing, so we want to make sure that we're working around them correctly. I guess pytest does not have a similar CI job, because the latest release crashes on import, somewhere in the typing module :-)

  File "/home/travis/virtualenv/python3.5.0/lib/python3.5/site-packages/_pytest/_code/__init__.py", line 2, in <module>
    from .code import Code  # noqa
  File "/home/travis/virtualenv/python3.5.0/lib/python3.5/site-packages/_pytest/_code/code.py", line 388, in <module>
    class ExceptionInfo(Generic[_E]):
  File "/home/travis/virtualenv/python3.5.0/lib/python3.5/site-packages/_pytest/_code/code.py", line 594, in ExceptionInfo
    def match(self, regexp: Union[str, Pattern]) -> bool:
  File "/opt/python/3.5.0/lib/python3.5/typing.py", line 534, in __getitem__
    dict(self.__dict__), parameters, _root=True)
  File "/opt/python/3.5.0/lib/python3.5/typing.py", line 491, in __new__
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
  File "/opt/python/3.5.0/lib/python3.5/typing.py", line 491, in <genexpr>
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
TypeError: issubclass() arg 1 must be a class

https://travis-ci.org/python-trio/trio/jobs/572601545

I'm guessing this is actually a bug in typing, and maybe the answer is just that pytest should announce that it's dropping support for 3.5.0. But I don't see anything about that in the release notes, so I figured you'd appreciate a heads-up.

[bluetech]

Thanks for letting us know. I'll try to get Python 3.5.0 going locally and see how difficult it is to fix, and report back.

[njsmith]

FYI, we also have an issue open to discuss the advantages/disadvantages of dropping support for 3.5.0 and 3.5.1, which might be relevant reading: python-trio/trio#880

[mgedmin]

I'm also seeing this error, but strangely only on Windows, and only in my Jenkins. I'm unable to reproduce locally, or on Appveyor.

Here's a representative traceback: https://jenkins.gedmin.as/job/mgp2pdf-on-windows/1666/console

[mgedmin]

Ooh, the issue @njsmith mentioned provides a Clue:

The typing module also went through a lot of changes over the lifespan of 3.5.x. I haven't looked into the details here, but @oremanj says "it looks like 3.5.2 was the first version with a typing module that's even kinda usable for async code".

I've never upgraded Python 3.5.0 to any newer bugfix release on my Jenkins.

(Actually I'm a dum dum because the issue title very clearly refers to 3.5.0, and I just read it as 3.5)

[bluetech]

I created a PR to fix this. Tested it in a python:3.5.0 docker container. I think it should be merged to fix the problem for 5.1.0, but I hope we can consider dropping support for Python 3.5.0 and 3.5.1 -- their typing support is indeed busted, and according to @njsmith's analysis in the trio issue, they should not be used much compared to the later point releases, since they were not included in any major Linux distribution release.

[nicoddemus]

(Moving the discussion from #5752 (comment) by @bluetech to here for better visibility)

The reason I didn't close the issue is that I thought it'd be best to fix this regression quickly for 5.1.x, and then use the issue to discuss how to proceed. The options as I see them are:

Keep Python 3.5.{0,1} support - in this case we should probably add it to CI (or change the 3.5 job to use it), move the hacks to _compat.py and also update flake8-typing-imports to detect these edge cases. Otherwise it is bound to regress again.

Drop support for Python 3.5.{0,1} - would make typing easier since it's missing a lot of things.

For selfish reasons, my preference is obviously 2 :)

IMHO I think we should fix this for problem with Python 3.5.{0,1} in pytest 5.1.1 regardless, because I think it is fair that those Python users get a working 5.1.X version.

For pytest 5.1.2+:

1. If we want to drop support for Python 3.5.{0,1}, we then need to update python_requires to exclude Python 3.5.{0,1} so users using those Python versions don't install pytest 5.1.2+.
2. If we want to keep supporting Python 3.5.{0,1}, then I agree we need to add it to the CI.

Personally, if very few users use Python 3.5.{0,1} because it has known quirky issues, I'm OK with pytest 5.1.2+ no longer supporting those Python versions.

@njsmith what's your opinion, given that trio has decided to support Python 3.5.{0,1} explicitly? Do you think there are many users using Python 3.5.{0,1}?

[njsmith]

The main reason we support early versions of 3.5.x is because it's easier to explain to users "this works on 3.5+", versus "this works on some versions of 3.5.x, and all versions of 3.6+". Plus it hasn't been too difficult to support so far – though it will probably get worse as we get more serious about type hints. Or if pytest drops support, that would also probably count as being too much hassle to bother with :-).

aiohttp for example only supports 3.5.3+. I forget why they chose that exact version. 3.5.1->3.5.2 had a major change in __aiter__ that makes 3.5.2 an obvious break point for us, and 3.5.2 is also what Ubuntu 16.04 LTS ships.

[nicoddemus]

The main reason we support early versions of 3.5.x is because it's easier to explain to users "this works on 3.5+", versus "this works on some versions of 3.5.x, and all versions of 3.6+"

I see, thanks!

Particularly what bothers me about dropping support in patch versions is about breaking expectations. If feasible, I would rather we support Python 3.5.{0,1} through pytest 5.1.X, introducing a warning that only Python 3.5.2+ will be supported in pytest 5.2.0+.

Would love to hear others' opinions though.

[bluetech]

If feasible, I would rather we support Python 3.5.{0,1} through pytest 5.1.X, introducing a warning that only Python 3.5.2+ will be supported in pytest 5.2.0+.

I agree. Typing changes are targeted at features, so after the fixup PR is merged, it shouldn't regress again for pytest 5.1.x. For pytest 5.2.x we can add a python_requires so it won't trip up people who just do pip install pytest.

[charris]

NumPy and SciPy are seeing segfaults with Pytest 5.1 and Python 3.5.6-dbg. For example https://travis-ci.org/numpy/numpy/jobs/573017630. Python 3.5.6 passes.

EDIT: This is not quite the same as the reported issue as it is dbg specific. Should I open a new issue for it?

EDIT: Opened #5759 for this.