[BUG]: [CPython 3.11] GC not implemented properly for objects with managed dictionaries

[Fidget-Spinner]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

Problem description

Pybind11 originally reported a CPython bug at python/cpython#92678. It was later found that the current solution for Py_TPFLAGS_MANAGED_DICT implemented by Pybind11 does not properly support GC for their dictionaries.

Suggestion: used the newly added (3.11b5) unstable functions:

• _PyObject_VisitManagedDict
• _PyObject_ClearManagedDict

to properly implement tp_clear and tp_traverse for types with Py_TPFLAGS_MANAGED_DICT set.

Functions were added in https://github.com/python/cpython/pull/95256/files.

Reproducible example code

No response

[Fidget-Spinner]

CC @Skylion007 and @henryiii

[Skylion007]

Yikes... and this is will be a 3.11 only fix since we want to switch APIs once the stable APIs from 3.12 are added....

[Skylion007]

I found another potential issue I am trying to address in this PR: #4106 . That atleast moves us to a somewhat stable get_dict / set_dict API.

[Fidget-Spinner]

Not needed anymore. Fixed upstream in CPython.

[vstinner]

Follow-up: the Py_TPFLAGS_MANAGED_DICT flag is set to late in make_static_property_type() causing an assertion error (Python debug build) or crash (Python release build) on Python 3.13: see #4970. I proposed a fix.