Explicit minimum supported PyPy?

[henryiii]

I'd like to explicitly have a minimum PyPy version stated, supported, and tested.

My proposal would be to support 7.3.0+ for now. Reasoning:

• A PyPy dev ( @mattip ) stated "PyPy users are pretty good about updating" (indicating there's no need to support 1+ year old PyPy's for now) in Transition PyPy to PEP 3149 compliant SOABI tags pypa/wheel#372
• Older versions of PyPy are buggy
• Wheel support on PyPI has been very recent, 7.2 or 7.3 I believe
• The separate wheel repository for NumPy, etc. only supports 7.3 (though they have a legacy archive with 7.1 and 7.2)
• NumPy's PyPI wheels will be 7.3 only when they are released
• cibuildwheel is 7.3 only
• 7.3.x wheels work on any 7.3.x release (the first time they've added this ABI promise)
• It's not trivial to test with less than 7.2, since PyPy is often just the latest version, rather than coming in versions, and no wheels are available.

In the past, such as in 2.5.0, our support was hit or miss; we don't support the latest PyPy3, for example, in 2.5.0.

If we choose an older minimum version, we need to add CI for it.

[YannickJadoul]

Agreed on this. I'd even ask a stronger question: is there anything in the 7.3.1 patch (and/or upcoming 7.3.2) that's been fixed for pybind11? If so, I'd even specify the latest patch version of 7.3. Wheels are already compatible, so for PyPy users, updating shouldn't be too hard?

Additional reason: with a bit of luck, we can clean up and dump some old workarounds :-)

[Julian]

(1 more longtime PyPy user though not developer who can vouch for "I always update really quickly, so only supporting the newest version is 100% fine")

[mattip]

+1 for dropping pre-7.3.1 right now.

[wjakob]

+1 from me as well.

Unrelated comment for @mattip: there are some really awful parts of pybind11 <-> PyPy interface where we have to fall back to evaluating strings during method dispatch (https://github.com/pybind/pybind11/blob/master/include/pybind11/pybind11.h#L2115) when users extend C++ classes in Python and override methods. This has substantial overheads. Do you think that PyPy could export the required frame information to the C API world so that a common implementation could be used for CPython and PyPy?

[arigo]

A less horribly slow method would be to pre-compile the Python source code, not re-evaluating it from a string every time. Or to expand the Python code into a series of PyXxx() function calls, with something along the lines of PyObject_CallMethod(sys_module, "_getframe", PyLong_FromLong(1)) and some PyObject_GetAttrString() on the result.

I imagine you already know it, but the kind of check done here (for both CPython or PyPy) is quite hackish. I would guess the problem is that when the user writes BaseClassInCpp.method(self) it really invokes the derived method, which makes an infinite loop in the case of overloaded methods in Python that wants to call the parent implementation. It would be an API change, but couldn't BaseClassInCpp.method(self) always invoke the base class method (or the closest written in C++), which might be different from self.method()?