From 1d8b796c33d76f8dc982c2a7f0394c0767eab601 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 21 May 2019 09:33:40 +0200 Subject: [PATCH 1/5] PEP 590: Mark the main API as private to allow changes in Python 3.9 --- pep-0590.rst | 40 ++++++++++++++++++++++++++++++++++------ 1 file changed, 34 insertions(+), 6 deletions(-) diff --git a/pep-0590.rst b/pep-0590.rst index c8d114608b9..2a37bc0b467 100644 --- a/pep-0590.rst +++ b/pep-0590.rst @@ -16,6 +16,9 @@ It introduces a new "vectorcall" protocol and calling convention. This is based on the "fastcall" convention, which is already used internally by CPython. The new features can be used by any user-defined extension class. +Most new API is private in CPython 3.8, with plans to finalized and +made public in Python 3.9. + **NOTE**: This PEP deals only with the Python/C API, it does not affect the Python language or standard library. @@ -43,6 +46,13 @@ This is inefficient for calls to classes as several intermediate objects need to For a class ``cls``, at least one intermediate object is created for each call in the sequence ``type.__call__``, ``cls.__new__``, ``cls.__init__``. +This PEP proposes an interface for use by extension modules. +Such interfaces cannot effectively be tested, or designed, without having the +consumers in the loop. +For that reason, we provide private (underscore-prefixed) names. +The API may change (based on onsumer feedback) in Python 3.9, where we expect +it to be finalized, and the underscores removed. + Specification ============= @@ -64,12 +74,14 @@ Changes to the ``PyTypeObject`` struct -------------------------------------- The unused slot ``printfunc tp_print`` is replaced with ``tp_vectorcall_offset``. It has the type ``Py_ssize_t``. -A new ``tp_flags`` flag is added, ``Py_TPFLAGS_HAVE_VECTORCALL``, +A new ``tp_flags`` flag is added, ``_Py_TPFLAGS_HAVE_VECTORCALL``, which must be set for any class that uses the vectorcall protocol. -If ``Py_TPFLAGS_HAVE_VECTORCALL`` is set, then ``tp_vectorcall_offset`` must be a positive integer. +If ``_Py_TPFLAGS_HAVE_VECTORCALL`` is set, then ``tp_vectorcall_offset`` must be a positive integer. It is the offset into the object of the vectorcall function pointer of type ``vectorcallfunc``. -This pointer may be ``NULL``, in which case the behavior is the same as if ``Py_TPFLAGS_HAVE_VECTORCALL`` was not set. +This pointer may be ``NULL``, in which case the behavior is the same as if ``_Py_TPFLAGS_HAVE_VECTORCALL`` was not set. + +If ``_Py_TPFLAGS_HAVE_VECTORCALL`` is not set, then ``tp_vectorcall_offset`` must be zero. The ``tp_print`` slot is reused as the ``tp_vectorcall_offset`` slot to make it easier for for external projects to backport the vectorcall protocol to earlier Python versions. In particular, the Cython project has shown interest in doing that (see https://mail.python.org/pipermail/python-dev/2018-June/153927.html). @@ -121,13 +133,29 @@ For getting the actual number of arguments from the parameter ``n``, the macro ``PyVectorcall_NARGS(n)`` must be used. This allows for future changes or extensions. +Finalizing the API +------------------ + +The underscore in the names ``_PyObject_Vectorcall`` and +``_Py_TPFLAGS_HAVE_VECTORCALL`` indicates that this API may change in minor +Python versions. +When finalized (which is planned for Python 3.9), they will be renamed to +``PyObject_Vectorcall`` and ``Py_TPFLAGS_HAVE_VECTORCALL``, with the former +names provided using macros. + +The new API will be documented as normal, but will warn of the above. + +Semantics for the other names introduced in this PEP (``PyVectorcall_NARGS``, +``PyVectorcall_Call``, ``Py_TPFLAGS_METHOD_DESCRIPTOR``, +``PY_VECTORCALL_ARGUMENTS_OFFSET``) are final. + New C API and changes to CPython ================================ The following functions or macros are added to the C API: -- ``PyObject *PyObject_Vectorcall(PyObject *obj, PyObject *const *args, Py_ssize_t nargs, PyObject *keywords)``: +- ``PyObject *_PyObject_Vectorcall(PyObject *obj, PyObject *const *args, Py_ssize_t nargs, PyObject *keywords)``: Calls ``obj`` with the given arguments. Note that ``nargs`` may include the flag ``PY_VECTORCALL_ARGUMENTS_OFFSET``. The actual number of positional arguments is given by ``PyVectorcall_NARGS(nargs)``. @@ -148,7 +176,7 @@ The following functions or macros are added to the C API: Subclassing ----------- -Extension types inherit the type flag ``Py_TPFLAGS_HAVE_VECTORCALL`` +Extension types inherit the type flag ``_Py_TPFLAGS_HAVE_VECTORCALL`` and the value ``tp_vectorcall_offset`` from the base class, provided that they implement ``tp_call`` the same way as the base class. Additionally, the flag ``Py_TPFLAGS_METHOD_DESCRIPTOR`` @@ -202,7 +230,7 @@ Third-party extension classes using vectorcall To enable call performance on a par with Python functions and built-in functions, third-party callables should include a ``vectorcallfunc`` function pointer, -set ``tp_vectorcall_offset`` to the correct value and add the ``Py_TPFLAGS_HAVE_VECTORCALL`` flag. +set ``tp_vectorcall_offset`` to the correct value and add the ``_Py_TPFLAGS_HAVE_VECTORCALL`` flag. Any class that does this must implement the ``tp_call`` function and make sure its behaviour is consistent with the ``vectorcallfunc`` function. Setting ``tp_call`` to ``PyVectorcall_Call`` is sufficient. From cb45bd8ef74f714365bc3f732af987c6a1966a32 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 21 May 2019 10:04:29 +0200 Subject: [PATCH 2/5] Move "Finalizing the API" section after "New C API" --- pep-0590.rst | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/pep-0590.rst b/pep-0590.rst index 2a37bc0b467..6a48b33bccc 100644 --- a/pep-0590.rst +++ b/pep-0590.rst @@ -133,22 +133,6 @@ For getting the actual number of arguments from the parameter ``n``, the macro ``PyVectorcall_NARGS(n)`` must be used. This allows for future changes or extensions. -Finalizing the API ------------------- - -The underscore in the names ``_PyObject_Vectorcall`` and -``_Py_TPFLAGS_HAVE_VECTORCALL`` indicates that this API may change in minor -Python versions. -When finalized (which is planned for Python 3.9), they will be renamed to -``PyObject_Vectorcall`` and ``Py_TPFLAGS_HAVE_VECTORCALL``, with the former -names provided using macros. - -The new API will be documented as normal, but will warn of the above. - -Semantics for the other names introduced in this PEP (``PyVectorcall_NARGS``, -``PyVectorcall_Call``, ``Py_TPFLAGS_METHOD_DESCRIPTOR``, -``PY_VECTORCALL_ARGUMENTS_OFFSET``) are final. - New C API and changes to CPython ================================ @@ -188,6 +172,23 @@ This restriction may be lifted in the future, but that would require special-casing ``__call__`` in ``type.__setattribute__``. +Finalizing the API +================== + +The underscore in the names ``_PyObject_Vectorcall`` and +``_Py_TPFLAGS_HAVE_VECTORCALL`` indicates that this API may change in minor +Python versions. +When finalized (which is planned for Python 3.9), they will be renamed to +``PyObject_Vectorcall`` and ``Py_TPFLAGS_HAVE_VECTORCALL``, with the former +names provided using macros. + +The new API will be documented as normal, but will warn of the above. + +Semantics for the other names introduced in this PEP (``PyVectorcall_NARGS``, +``PyVectorcall_Call``, ``Py_TPFLAGS_METHOD_DESCRIPTOR``, +``PY_VECTORCALL_ARGUMENTS_OFFSET``) are final. + + Internal CPython changes ======================== From 84c91357e055a91050a960429dc87e4c6e78d825 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 21 May 2019 10:12:25 +0200 Subject: [PATCH 3/5] Clarify what the plan for old names is --- pep-0590.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0590.rst b/pep-0590.rst index 6a48b33bccc..debd3875e95 100644 --- a/pep-0590.rst +++ b/pep-0590.rst @@ -179,8 +179,8 @@ The underscore in the names ``_PyObject_Vectorcall`` and ``_Py_TPFLAGS_HAVE_VECTORCALL`` indicates that this API may change in minor Python versions. When finalized (which is planned for Python 3.9), they will be renamed to -``PyObject_Vectorcall`` and ``Py_TPFLAGS_HAVE_VECTORCALL``, with the former -names provided using macros. +``PyObject_Vectorcall`` and ``Py_TPFLAGS_HAVE_VECTORCALL``. +The the old underscore-prefixed names will remain available as aliases. The new API will be documented as normal, but will warn of the above. From 6ccba2b97a34e8d00bd0d46a6211892d48aa4075 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 21 May 2019 11:05:08 +0200 Subject: [PATCH 4/5] Address review comments --- pep-0590.rst | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/pep-0590.rst b/pep-0590.rst index debd3875e95..30c68cbf8f4 100644 --- a/pep-0590.rst +++ b/pep-0590.rst @@ -16,8 +16,8 @@ It introduces a new "vectorcall" protocol and calling convention. This is based on the "fastcall" convention, which is already used internally by CPython. The new features can be used by any user-defined extension class. -Most new API is private in CPython 3.8, with plans to finalized and -made public in Python 3.9. +Most of the new API is private in CPython 3.8. +The plan is to finalize semantics and make it public in Python 3.9. **NOTE**: This PEP deals only with the Python/C API, it does not affect the Python language or standard library. @@ -50,7 +50,7 @@ This PEP proposes an interface for use by extension modules. Such interfaces cannot effectively be tested, or designed, without having the consumers in the loop. For that reason, we provide private (underscore-prefixed) names. -The API may change (based on onsumer feedback) in Python 3.9, where we expect +The API may change (based on consumer feedback) in Python 3.9, where we expect it to be finalized, and the underscores removed. @@ -81,8 +81,6 @@ If ``_Py_TPFLAGS_HAVE_VECTORCALL`` is set, then ``tp_vectorcall_offset`` must be It is the offset into the object of the vectorcall function pointer of type ``vectorcallfunc``. This pointer may be ``NULL``, in which case the behavior is the same as if ``_Py_TPFLAGS_HAVE_VECTORCALL`` was not set. -If ``_Py_TPFLAGS_HAVE_VECTORCALL`` is not set, then ``tp_vectorcall_offset`` must be zero. - The ``tp_print`` slot is reused as the ``tp_vectorcall_offset`` slot to make it easier for for external projects to backport the vectorcall protocol to earlier Python versions. In particular, the Cython project has shown interest in doing that (see https://mail.python.org/pipermail/python-dev/2018-June/153927.html). @@ -180,7 +178,7 @@ The underscore in the names ``_PyObject_Vectorcall`` and Python versions. When finalized (which is planned for Python 3.9), they will be renamed to ``PyObject_Vectorcall`` and ``Py_TPFLAGS_HAVE_VECTORCALL``. -The the old underscore-prefixed names will remain available as aliases. +The old underscore-prefixed names will remain available as aliases. The new API will be documented as normal, but will warn of the above. From 999f399ece39fa31b3a3428d6e7f89e8397ac14b Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 21 May 2019 11:05:31 +0200 Subject: [PATCH 5/5] Typo fix: "the the" --- pep-0590.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0590.rst b/pep-0590.rst index 30c68cbf8f4..7d8934da140 100644 --- a/pep-0590.rst +++ b/pep-0590.rst @@ -89,7 +89,7 @@ Descriptor behavior One additional type flag is specified: ``Py_TPFLAGS_METHOD_DESCRIPTOR``. -``Py_TPFLAGS_METHOD_DESCRIPTOR`` should be set if the the callable uses the descriptor protocol to create a bound method-like object. +``Py_TPFLAGS_METHOD_DESCRIPTOR`` should be set if the callable uses the descriptor protocol to create a bound method-like object. This is used by the interpreter to avoid creating temporary objects when calling methods (see ``_PyObject_GetMethod`` and the ``LOAD_METHOD``/``CALL_METHOD`` opcodes).