_decimal: pi benchmark 31-45% slower compared to 3.9

[skrah]

Bug report

Bug description:

The pi benchmark in Modules/_decimal/tests/bench.py is up to 31% slower with --enable-gil and up to 45% slower with --disable-gil compared to Python 3.9 as the baseline.

Change the number of iterations to 100000 for more stable benchmarks.

This is heavily OS and hardware dependent. The worst case cited above was observed on a six core i5-8600K.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-114682: Use METH_METHOD calling convention in _decimal #115196
• gh-114682: Improve performance of module state access in _decimal #115401

[sunmy2019]

Can you try --disable-gil with current main?

[skrah]

The figures are almost the same, around 30% (enable) and 43% (disable).

[skrah]

The base slowdown of 30% is of course due to the module state changes. For comparison, and to see that the i5-8600K is not inherently underperforming with threads:

The ideal situation is implemented in libmpdec++: It has a C++11 thread local context, which is also called for every inline operation. The C++11 TLS adds just 4% overhead compared to pure C (where the context is passed directly to the functions).

This of course would be quite difficult to achieve in CPython. But the thread local context may not even be the primary culprit. In Martin von Loewis' first module state implementation the basic slowdown for _decimal was 20%, and a good part of that was due to the heap types.

[skrah]

The ideal situation is implemented in libmpdec++: It has a C++11 thread local context, which is also called for every inline operation. The C++11 TLS adds just 4% overhead compared to pure C (where the context is passed directly to the functions).

This of course would be quite difficult to achieve in CPython.

The wording was not ideal: The key point is that C++ achieves this without a cached context, which is of course now in the module state.

[ericsnowcurrently]

@encukou, any thoughts on this?

[neonene]

PyModule_GetState() Getting a module state via PyType_GetModuleByDef() seems to be expensive for heavy use.
It would be nice if a heap type could own the module state's address at its creation time to recover more than 10%.

[erlend-aasland]

PyModule_GetState() via PyType_GetModuleByDef() seems to be expensive for heavy use. It would be nice if a heap type could own the module state's address at its creation time to recover more than 10%.

_PyModule_GetState() is used, not PyModule_GetState(); I'd be surprised if this is the culprit:

cpython/Include/internal/pycore_moduleobject.h

Lines 32 to 35 in 1aec064

	static inline void* _PyModule_GetState(PyObject* mod) {
	assert(PyModule_Check(mod));
	return ((PyModuleObject *)mod)->md_state;
	}

OTOH, PyType_GetModuleByDef() can be more expensive. Storing the state pointer in the type struct is a possibility.

[neonene]

@ericsnowcurrently Can a module state be accessed correctly with the following?

_Py_thread_local module_state *thread_local_state; // set at the exec phase

If not invalid, it will be useful for the function which currently has no way to get the module state. A _Py_thread_local pointer may not be so fast to access, but it can also bypass a state struct?

[ericsnowcurrently]

Multiple interpreters can run in the same thread, so a thread-local may be ambiguous in that case.

---

As to storing a reference to the module state on the type object, that shouldn't be necessary. Every heap type as a ht_module field that points to the module, accessible via PyType_GetModule(). From there you call PyModule_GetState(). In fact, we already have PyType_GetModuleState() that combines those two calls. That's fairly efficient, though obviously not as efficient as a static global.

The only caveat is that PyType_GetModule() (and thus PyType_GetModuleState()) operates relative to the given type. Thus subclasses make things trickier. IIRC, when they're involved, you end up having to walk the MRO one way or another, which is what PyType_GetModuleByDef() does.

FWIW, the common case where PyType_GetModuleByDef() is used is either in "tp slot" functions or intermediate static functions that aren't doing the right thing. In both cases the problem lies in throwing away the expensive information we already have. For tp slot functions the API doesn't pass the module in, even though it could. We're stuck with backward compatibility issues there. For intermediate static functions, I expect it's all too common to figure out the module (or module state) at some "leaf node" function but not pass it through the intermediate functions. Thus the relatively expensive PyType_GetModuleByDef() gets called more than it needs to be.

I suppose another problem case is with callbacks that don't provide a way to pass the module through. I'm not sure that's a common issue though.

Back to the _decimal module, I haven't looked at what usage patterns are resulting in such a significant slowdown.

[erlend-aasland]

Thus subclasses make things trickier. IIRC, when they're involved, you end up having to walk the MRO one way or another, which is what PyType_GetModuleByDef() does.

Yes; heap types that are not base classes can go the fast route via PyType_GetModuleState() (and _PyType_GetModuleState()).

[encukou]

On February 2, 2024 11:10:49 PM GMT+01:00, Eric Snow ***@***.***> wrote: Multiple interpreters can run in the same thread, so a thread-local may be ambiguous in that case. ---- As to storing a reference to the module state on the type object, that shouldn't be necessary. Every heap type as a `ht_module` field that points to the module, accessible via `PyType_GetModule()`. From there you call `PyModule_GetState()`. In fact, we already have `PyType_GetModuleState()` that combines those two calls. That's fairly efficient, though obviously not as efficient as a static global. The only caveat is that `PyType_GetModule()` (and thus `PyType_GetModuleState()`) operates relative to the given type. Thus subclasses make things trickier. IIRC, when they're involved, you end up having to walk the MRO one way or another, which is what `PyType_GetModuleByDef()` does. FWIW, the common case where `PyType_GetModuleByDef()` is used is either in "tp slot" functions or intermediate static functions that aren't doing the right thing. In both cases the problem lies in throwing away the expensive information we already have. For tp slot functions the API doesn't pass the module in, even though it *could*. We're stuck with backward compatibility issues there. For intermediate static functions, I expect it's all too common to figure out the module (or module state) at some "leaf node" function but not pass it through the intermediate functions. Thus the relatively expensive `PyType_GetModuleByDef()` gets called more than it needs to be. I suppose another problem case is with callbacks that don't provide a way to pass the module through. I'm not sure that's a common issue though. Back to the _decimal module, I haven't looked at what usage patterns are resulting in such a significant slowdown. -- Reply to this email directly or view it on GitHub: #114682 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[hauntsaninja]

@encukou I think your comment might have gotten lost? All I see is quote text

[skrah]

The float object gets the luxury of its own freelist and avoids the slow PyObject_GC_* functions altogether. The Decimal constructor is a pure function (the context is only used for recording errors) and Decimals are immutable, just like floats.

It is likely that 15 percentage points can be shaved off just by doing the same as in floatobject.c and then focus on the thread local context (indeed, PyType_GetModuleByDef() is called at dozens of places).

[erlend-aasland]

AFAICS, we can trim away a lot of the PyType_GetModuleByDef calls. For example, a lot of them are happening in methods; we can use the METH_METHOD calling convention on practically all of these, so we can access the module state using _PyType_GetModuleState:

cpython/Include/internal/pycore_typeobject.h

Lines 100 to 109 in 15f6f04

	_PyType_GetModuleState(PyTypeObject *type)
	{
	assert(PyType_Check(type));
	assert(type->tp_flags & Py_TPFLAGS_HEAPTYPE);
	PyHeapTypeObject *et = (PyHeapTypeObject *)type;
	assert(et->ht_module);
	PyModuleObject *mod = (PyModuleObject *)(et->ht_module);
	assert(mod != NULL);
	return mod->md_state;
	}

Also (but not performance relevant), the find_state_left_or_right() calls in the Py_tp_richcompare slots can be replaced with get_module_state_by_def() calls.