gh-108867: Add PyThreadState_GetUnchecked() function

[vstinner]

Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't issue a fatal error if it is NULL. The
caller is responsible to check if the result is NULL. Previously,
this function was private and known as _PyThreadState_UncheckedGet().

• Issue: C API: Rename _PyThreadState_GetUnchecked() to public PyThreadState_GetUnchecked #108867

---

📚 Documentation preview 📚: https://cpython-previews--108870.org.readthedocs.build/

[vstinner]

I was worried about Py_TRASHCAN_BEGIN_CONDITION() macro which currently calls _PyThreadState_UncheckedGet(). This macro uses private functions:

/* Python 3.9 private API, invoked by the macros below. */
PyAPI_FUNC(int) _PyTrash_begin(PyThreadState *tstate, PyObject *op);
PyAPI_FUNC(void) _PyTrash_end(PyThreadState *tstate);

/* Python 3.10 private API, invoked by the Py_TRASHCAN_BEGIN(). */
PyAPI_FUNC(int) _PyTrash_cond(PyObject *op, destructor dealloc);

I was worried that we still provide Python 3.9 functions. Is it the a stable ABI? Nope, these functions are not part of Misc/stable_abi.toml.

Moreover, I explicitly removed the following macros from the limited C API in Python 3.9:

• PyTrash_UNWIND_LEVEL
• Py_TRASHCAN_BEGIN_CONDITION()
• Py_TRASHCAN_BEGIN()
• Py_TRASHCAN_END()
• Py_TRASHCAN_SAFE_BEGIN()
• Py_TRASHCAN_SAFE_END()

So in fact, we keep Python 3.9 functions for the ABI backward compatibility, whereas we do not support these APIs in the stable ABI.

In short, it's ok to rename _PyThreadState_UncheckedGet() to PyThreadState_GetUnsafe(), it's not used by the stable ABI.

[vstinner]

A bunch of projects currently uses _PyThreadState_UncheckedGet(), whereas I removed the function from the public C API in Python 3.13.

Affected projects (18):

• Cython (0.29.36)
• catboost (1.2)
• cffi (1.15.1)
• debugpy (1.6.7)
• dlib (19.24.2)
• fastobo (0.12.2)
• frozendict (2.3.8)
• onnx (1.14.0)
• onnxoptimizer (0.3.13)
• onnxsim (0.4.33)
• orjson (3.9.1)
• osmium (3.6.0)
• praat-parselmouth (0.4.3)
• ptvsd (4.3.2)
• pybind11 (2.10.4)
• pydevd (2.9.6)
• pydevd-pycharm (232.8296.19)
• pythonnet (3.0.1)

In this list, IMO the interesting one is Cython:

$ git grep -E '(_PyThreadState_UncheckedGet|__Pyx_PyThreadState_Current)'
Cython/Utility/AsyncGen.c:    tstate = __Pyx_PyThreadState_Current;
Cython/Utility/AsyncGen.c:    tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Coroutine.c:    __Pyx_PyGen__FetchStopIterationValue(__Pyx_PyThreadState_Current, pvalue)
Cython/Utility/Coroutine.c:    tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Coroutine.c:    __Pyx_PyGen__FetchStopIterationValue(__Pyx_PyThreadState_Current, &val);
Cython/Utility/Exceptions.c:    __pyx_assertions_enabled_flag = ! _PyInterpreterState_GetConfig(__Pyx_PyThreadState_Current->interp)->optimization_level;
Cython/Utility/Exceptions.c:#define __Pyx_PyThreadState_assign  $local_tstate_cname = __Pyx_PyThreadState_Current;
Cython/Utility/Exceptions.c:        PyThreadState *tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Exceptions.c:        (void) __Pyx_CLineForTraceback(__Pyx_PyThreadState_Current, c_line);
Cython/Utility/Exceptions.c:    PyThreadState *tstate = __Pyx_PyThreadState_Current;
Cython/Utility/ModuleSetupCode.c:    // Py3<3.5.2 does not support _PyThreadState_UncheckedGet().
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current PyThreadState_Get()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current PyThreadState_GET()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current PyThreadState_GET()
Cython/Utility/ModuleSetupCode.c:  #define __Pyx_PyThreadState_Current _PyThreadState_Current
Cython/Utility/ModuleSetupCode.c:  current = tcur == __Pyx_PyThreadState_Current;
Cython/Utility/ObjectHandling.c:    PyThreadState *tstate = __Pyx_PyThreadState_Current;
Cython/Utility/Profile.c:          tstate = __Pyx_PyThreadState_Current;                                          \
Cython/Utility/Profile.c:      PyThreadState* tstate = __Pyx_PyThreadState_Current;                                 \
Cython/Utility/Profile.c:              tstate = __Pyx_PyThreadState_Current;                                       \
Cython/Utility/Profile.c:          PyThreadState* tstate = __Pyx_PyThreadState_Current;                            \
Cython/Utility/Profile.c:      PyThreadState* tstate = __Pyx_PyThreadState_Current;                                \
Cython/Utility/Profile.c:              tstate = __Pyx_PyThreadState_Current;                                        \
Cython/Utility/Profile.c:          PyThreadState* tstate = __Pyx_PyThreadState_Current;                             \
Cython/Utility/Profile.c:      PyThreadState* tstate = __Pyx_PyThreadState_Current;                                 \

I can easily add PyThreadState_GetUnsafe() to pythoncapi-compat by calling _PyThreadState_UncheckedGet() on Python 3.12 and older. This private function is available since Python 3.5. It seems to be available on PyPy:

$ cd /usr/include/pypy3.9/
$ grep _PyThreadState_UncheckedGet -R .
./pylifecycle.h:    (_Py_IsFinalizing() ? _PyThreadState_UncheckedGet() : NULL)
./pypy_decl.h:#define _PyThreadState_UncheckedGet _PyPyThreadState_UncheckedGet
./pypy_decl.h:PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void);

[vstinner]

I asked in the C API Working Group which function name is better: use Unchecked suffix or Unsafe suffix?

[vstinner]

I renamed the function to PyThreadState_GetUnchecked() and addressed @encukou's suggestion on the documentation.

[vstinner]

I renamed the function to PyThreadState_GetUnchecked()

Oops, my PR still used PyThreadState_GetUnsafe() name. I messed up something.

Anyway, I updated it again, and now it uses PyThreadState_GetUnchecked() name everywhere.

[vstinner]

I added PyThreadState_GetUnchecked() to pythoncapi-compat: python/pythoncapi-compat@f78c780