`tp_bases` of `object` is `NULL`: undocumented or unintentional behavior change in 3.12?

[rwgk]

Documentation

See #103912 (comment) for full context.

Specific problem:

It appears tp_bases of object was an empty tuple in the past, but is now NULL, could that be? Is that intentional?

If intentional, could it please be documented? I think the behavior change is likely to trip up fair a number of people.

I looked in https://github.com/python/cpython/blob/edd0cb8e77d7b65f5a9c2c69dc81f9c4514878d5/Doc/whatsnew/3.12.rst but don't see tp_bases being mentioned there.

I also verified that the tp_bases == nullptr patch is still needed with current 3.12 HEAD (edd0cb8).

Linked PRs

• gh-105020: Share tp_bases and tp_mro Between Interpreters For All Static Builtin Types #105115
• [3.12] gh-105020: Share tp_bases and tp_mro Between Interpreters For All Static Builtin Types (gh-105115) #105124

[sunmy2019]

NULL means it is stored in the interpreter. To get the correct value, you will need an internal function

Objects/typeobject.c: lookup_tp_bases

or

Objects/typeobject.c: _PyType_GetBases

These changes and APIs usages are private and undocumented.

[ronaldoussoren]

The same appears to be true for tp_dict of static builtin classes, and that change is also not documented.

For both changes there's no clean alternative other than using the private API (_PyType_GetDict for tp_dict and _PyType_GetBases for tp_bases). One could use PyAttr_GetAtring(type, "__dict__") to access tp_dict instead, but that's not necessarily equivalent when subtypes are involved.

I'm running into this issue while porting PyObjC to 3.12.

[ronaldoussoren]

This is related to #94673

@ericsnowcurrently, do you recall why there are no public accessors to replace directly accessing tp_bases and tp_dict?

[ronaldoussoren]

PyObjC 9.2 will use _PyType_GetDict with Python 3.12 to get it to work without invasive changes. It would be nice if that API could be moved to the unstable or even stable tier as there's currently no way to get the same functionality with a public API. The same is true for _PyType_GetBases (use case for the OP).

In particular, the only public way I've found to get this information is to use PyObject_GetAttr for the corresponding dunder name. That doesn't fit my use-case for two reasons:

1. Semantically this is not the same because types can override __getattribute__ and could return different information.

   In PyObjC the PyObject_GetAttr path could lead to unintentional recursion because I have a custom __dict__ descriptor on some types to lazily initialise those types.

2. The PyObject_GetAttr alternative results in differences in refcount (its result is a new reference, while accessing the corresponding tp_ slot is a borrowed reference). That leeds to fairly large changes to correctly handle refcounts in all paths through the code (some of which is from the Python 2.1 era).

@rwgk, sorry about hijacking your issue, but the problem we're running into is basically the same with to different slots on PyTypeObject and should have a similar solution.

[rwgk]

sorry about hijacking your issue

No worries about that, on the contrary, I feel like I should edit the title. Something like:

Urgently needed: Guidance for binding tool maintainers: How to deal with the consequences of #103912

Based on the comments I saw thus far, I'm sorry to say, it seems like this wasn't actually planned out.

[gpshead]

@ericsnowcurrently

[ericsnowcurrently]

I'm working on a fix.

[ericsnowcurrently]

do you recall why there are no public accessors to replace directly accessing tp_bases and tp_dict?

I don't, though it seem very sensible for us to add getter functions to the public API.