Better specialization of calls, post introduction of `PRECALL`.

[markshannon]

With the new opcodes, the sequence of instructions executed during a call looks like this:

The stats https://github.com/faster-cpython/ideas/blob/main/stats.md#call show that we only have a 72% hit rate on the standard benchmark suite.

Specialization attempts

	Count	Ratio
Success	2909102	24.4%
Failure	9035032	75.6%

Failure kind	Count	Ratio
bound method	2136774	23.6%
complex parameters	1252534	13.9%
python class	1245231	13.8%
pycfunction with keywords	859904	9.5%
class no vectorcall	723781	8.0%
kwnames	634422	7.0%
pycfunction	586186	6.5%
pycfunction noargs	407864	4.5%
class mutable	327462	3.6%
other	232681	2.6%
bad call flags	179897	2.0%
cmethod	168390	1.9%
pycfunction fast with keywords	162744	1.8%
str	79833	0.9%
method wrapper	32561	0.4%
operator wrapper	4768	0.1%

Each of these failures need a different strategy, so multiple strategies.

Bound methods.

Bounds methods can come from two places, bound methods object explicitly used in the program, and classmethods.
The first should be handled in PRECALL_FUNCTION, the second in LOAD_METHOD.

Complex parameters.

This needs more investigation, to see if some of these can be specialized.

Python class

Should be handled in PRECALL_FUNCTION which will create the self object and push a clean-up frame, leaving the __init__ method to be handled by CALL.

Builtin functions and classes using the older caller conventions

These should be fixed by modernizing the callee, not accommodating them in the interpreter.

Mutable classes

Mystery classification. Needs investigation, possibly a bug in the classification.

[markshannon]

The above sequence is still a bit messy, with an unnecessary distinction between calling an attribute, and other calls.
Ideally we want one PRECALL instruction that can be specialized for the type (or in some cases, value) of the callable, and the CALL instruction that can be specialized for the "shape" of the call (i.e. number of args, named args, etc).

LOAD_METHOD puts two values on the stack, either func, self or NULL, func.
The extra stack space allows us to unpack a bound-method without expensive stack manipulations.
There is no reason why we can't generalize this to all calls, allowing us that freedom to unpack (or repack) as needed.
By prefixing non-method call sequences with a PUSH_NULL instruction we can unify the two PRECALLs and reduce the number of special cases required.

Now PRECALL n expects the n+1 and n+2 items on the stack to be func, self or NULL, func, regardless of how the callable was evaluated.

The downside is that we are adding yet another instruction into the call sequence. Stats show that PUSH_NULL represents about 3% of instructions executed. Given it does so little work, the cost is just dispatching, so should be ~0.3%.

However, PUSH_NULL occurs before LOAD_GLOBAL in about 2/3s of cases.
Specializations of LOAD_GLOBAL can include a flag to push a NULL and skip an instruction without branching, so the expected overhead should be < 0.2%

More uniform and effective specialization should pay this back and plenty more.

[markshannon]

Specialization strategy for all currently specialized types/values and most of the types that we currently fail on.

What	PRECALL	Named keywords?	CALL
Python function	pass	yes	Two specializations: One for exact args. One to handle defaults and kwnames.
type(1), str(1), tuple(1)	Here, skip over CALL	no	no
Most builtin functions	Here, skip over CALL	no	no
builtin functions with METH_FASTCALL | METH_KEYWORDS	pass	yes	No change to current specialization
len/isinstance	Here, skip over CALL	no	no
Most method descriptors	Here, skip over CALL	no	no
method descriptors with METH_FASTCALL | METH_KEYWORDS	pass	yes	Much the same as for the equivalent builtin functions
Bound methods	Unpack onto stack, if space*	yes	No, should be unpacked so will specialize as Python functions.
Python class	Allocate object, push cleanup frame. Leave __init__ self on stack, if space*	yes	No, expect __init__ to be a Python function

^* If the lower slot on the stack is NULL we treat can this the same as PRECALL_FUNCTION.

The only class of failures in https://github.com/faster-cpython/ideas/blob/main/stats.md not covered is "complex parameters" which are Python functions, so would be handled by adding more cases to the specialization of Python functions.

[markshannon]

Looking at the above table, it is clear that we should place KW_NAMES before PRECALL.

Builtin functions and method descriptors with METH_FASTCALL | METH_KEYWORDS cannot currently be specialized in the PRECALL because we don't know if we have kwnames.

Swapping KW_NAMES and PRECALL fixes that. It makes no difference to any other calls.

[markshannon]

Resulting in this table:

What	PRECALL	Named keywords?	CALL
Python function	pass	yes	Two specializations: One for exact args. One to handle defaults and kwnames.
type(1), str(1), tuple(1)	Here, skip over CALL	no	no
Most builtin functions	Here, skip over CALL	no	no
builtin functions with METH_FASTCALL | METH_KEYWORDS	As current specialization	yes	no
len/isinstance	Here, skip over CALL	no	no
Most method descriptors	Here, skip over CALL	no	no
method descriptors with METH_FASTCALL | METH_KEYWORDS	Much the same as for the equivalent builtin functions	yes	no
Bound methods	Unpack onto stack, if space	yes	No, should be unpacked so will specialize as Python functions.
Python class	Allocate object, push cleanup frame. Leave __init__ self on stack, if space	yes	No, expect __init__ to be a Python function

[markshannon]

In summary, we will:

1. Move KW_NAMES before PRECALL.
2. Move all CALL specializations, except for Python functions, to PRECALL
3. Add specializations for bound methods and Python classes to PRECALL.
4. Extend CALL specializations to handle Python functions with more complex arguments.

[markshannon]

It looks like specializing calls to Python classes is more complex than I allowed for.

The three possible approaches we have so far come up with are:

1. My original idea: Push a temporary frame to clean. I've done some work on implementing this and it involves some moving of arguments to the new frame, which has a cost. It also needs one or more dynamically created Python functions for each class to allow effective specialization of the call to __init__ and do the cleanup.
2. Add some state to the frame add cleanup in RETURN_VALUE bpo-46939: Specialize calls to Python classes python/cpython#31707
3. Add a third bytecode to call sequence, POSTCALL. POSTCALL would be a no-op for all calls, except calls to Python classes. POSTCALL could be jumped over by the specializations of PRECALL as they do with CALL, so the cost would be small for those cases and a bit larger for calls to bound methods and Python functions.

TBH, all these solutions are a bit unsatisfying.

1. Adds some overhead and a fair bit of complexity, for what might be quite limited performance advantages.
2. Seems fragile and may slow down calls to normal Python functions and bound methods.
3. Makes the code objects even larger and will slow down calls to normal Python functions and bound methods.

I think we should put this on hold for now.
Maybe a better approach will emerge.
Maybe we will need to change other things and one of the existing approaches will become more viable.

[TeamSpen210]

I'm not sure how well this would work, but would it be possible to do modifications to the __init__ function itself? When specialising class calls go through that function and swap out all the RETURN_VALUE opcodes, and add another opcode to the start. Adding a flag indicating if the code object was altered this way would ensure it only gets scanned once. There would need to also be a frame flag like in the second method that the new opcodes would check to deactivate the extra functionality if __init__ was called manually.