Implement the __init_extra__ protocol of categories for Cython classes.

[nthiery]

Component: categories

Author: Simon King

Branch/Commit: u/mkoeppe/implement_the___init_extra___protocol_of_categories_for_cython_classes_ @ 0ec1c06

Issue created by migration from https://trac.sagemath.org/ticket/15718

[simon-king-jena]

comment:1

It would probably be rather easy:

Currently, we only look into the mro of the class; if the class is a Python class, then it is a subclass of the category's parent class (since the quest for __init_extra__ takes place after category initialisation). But in the case of Cython class, we simply need to look explicitly into both the class' mro and the category's parent class' mro.

It would detect a lot of Cython classes in which the call to Parent.__init__ happens too early for __init_extra__ to work... But these would be exactly the classes that currently do not allow Python subclasses.

[nthiery]

comment:2

Replying to @simon-king-jena:

It would probably be rather easy:

Currently, we only look into the mro of the class; if the class is a Python class, then it is a subclass of the category's parent class (since the quest for __init_extra__ takes place after category initialisation). But in the case of Cython class, we simply need to look explicitly into both the class' mro and the category's parent class' mro.

+1!

For the record: this will require calling
sage.misc.structure.getattr_from_other_class, to emulate the parent
being an instance of the category's parent class.

[simon-king-jena]

comment:3

Replying to @nthiery:

For the record: this will require calling
sage.misc.structure.getattr_from_other_class, to emulate the parent
being an instance of the category's parent class.

I disagree. If I recall correctly, we look up the whole class hierarchy and execute all __init_extra__ methods that we can find. getattr_from_other_class would only return one __init_extra__, but we want all, and thus we need to go up both the Cython class' and the category's parent class' mro.

[nthiery]

comment:4

Replying to @simon-king-jena:

If I recall correctly, we look up the whole class hierarchy and execute all __init_extra__ methods that we can find.

Yes!

Still, some magic will be needed to bind each such method to the
object and call it (by default Python refuses to bind a method of a
class to a non-instance of that class); that's were
getattr_from_other_class (or some subpiece thereof) will come into
play.

[simon-king-jena]

comment:5

Replying to @nthiery:

Still, some magic will be needed to bind each such method to the
object and call it (by default Python refuses to bind a method of a
class to a non-instance of that class); that's were
getattr_from_other_class (or some subpiece thereof) will come into
play.

Do we need to bind it? We just want to call it.

[simon-king-jena]

comment:6

Replying to @simon-king-jena:

Do we need to bind it? We just want to call it.

Such as:

sage: R.<x,y> = ZZ[]
sage: C = Algebras(ZZ).parent_class
sage: C.__init_extra__.__func__(R)

[simon-king-jena]

comment:7

I just tested: When one empties the coercion cache and the coerce_from_list, then calling C.__init_extra__.__func__(R) will fill it again, which is its purpose. So, it works without binding.

[nbruin]

comment:8

Replying to @simon-king-jena:

I just tested: When one empties the coercion cache and the coerce_from_list, then calling C.__init_extra__.__func__(R) will fill it again, which is its purpose. So, it works without binding.

Well ... that works when C.__init_extra__ is an unbound method that has a __func__ attribute. Likely all unbound methods do, so you've probably dealt with the case that __init_extra__ is an unbound method. But attributes aren't necessarily! I think only unbound methods bind when called as attributes on instances. Other objects just get called. But that's a case you should test for if you want this to be properly compatible with python's inheritance. Unbound methods aren't the only things that lead to callable attributes on instances.

[nthiery]

comment:9

I guess it's fair enough to write in the specifications of the protocol that __init_extra__ shall be a method. And if some day we have a serious use case (which I kind of doubt), it will always be time to generalize the protocol.

[nbruin]

comment:10

Replying to @nthiery:

I guess it's fair enough to write in the specifications of the protocol that __init_extra__ shall be a method.

You'll have to be careful how you formulate that: cdef classes can have methods too (some custom cython object of course) and their unbound versions specifically call themselves "method" and have no __func__ attribute. They are callable, though, further suggesting one should probably check if the attribute is callable and if not, see if there is a __func__ attribute that is callable (or some safe permutation of this that performs better)

And if some day we have a serious use case (which I kind of doubt), it will always be time to generalize the protocol.

Famous last words... I've seen many cases happen in sage where little design issues (which, at the time perpetrated seemed perfectly reasonable) lead to weird, unforseen and silent failures. The problem this ticket is trying to fix is one of them.

[simon-king-jena]

comment:11

Replying to @nbruin:

You'll have to be careful how you formulate that: cdef classes can have methods too (some custom cython object of course) and their unbound versions specifically call themselves "method" and have no __func__ attribute.

Note that the idea is to go up self.__class__.mro() on the one hand, and self.category().parent_class.mro(), on the other hand. The former is no problem, since self is an instance of its class and thus we don't need the __func__ attribute. The latter is (currently, at least) no problem, since parent classes of categories are Python classes and thus have __func__ on their methods. Or am I mistaken?