Assignment to __bases__ of direct object subclasses

[glchapman]

BPO	672115
Nosy	@mwhudson, @amauryfa, @benjaminp, @florentx, @ambv, @kynan, @maggyero
Files	typeobject.c.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2003-01-21.22:45:40.000>
labels = ['interpreter-core', 'type-feature', '3.11']
title = 'Assignment to __bases__ of direct object subclasses'
updated_at = <Date 2022-03-23.00:08:37.846>
user = 'https://bugs.python.org/glchapman'

bugs.python.org fields:

activity = <Date 2022-03-23.00:08:37.846>
actor = 'maggyero'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2003-01-21.22:45:40.000>
creator = 'glchapman'
dependencies = []
files = ['751']
hgrepos = []
issue_num = 672115
keywords = []
message_count = 18.0
messages = ['14163', '14164', '14165', '14166', '14167', '14168', '14169', '14170', '14171', '70132', '70133', '86225', '88508', '88569', '106633', '154116', '189987', '270220']
nosy_count = 16.0
nosy_names = ['mwh', 'glchapman', 'amaury.forgeotdarc', 'lorph', 'benjamin.peterson', 'memeplex', 'farcat', 'flox', 'Hari.Krishna.Dara', 'lukasz.langa', 'kynan', 'eli.collins', 'aventurella', 'krzysiekj', 'borisompol', 'maggyero']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue672115'
versions = ['Python 3.11']

Linked PRs

• gh-37817: Allow assignment to __bases__ of direct subclasses of builtin classes #137585

[glchapman]

I'm not entirely sure this is a bug, but I think it is
surprising:

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> class A(object):
...     pass
...
>>> class B(object):
...     pass
...
>>> B.__bases__ = (A,)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: __bases__ assignment: 'A' deallocator differs 
from 'object'

It seems like you should be able to change the
__bases__ of a new-style class (implemented in
Python) which inherits directly from object to another
new-style class. (Will the deallocator issue ever come
into play if the only types involved are HEAPTYPES and
object as the ultimate base?)

[mwhudson]

Logged In: YES
user_id=6656

I agree this is a bit surprising. When I was writing this
code I went for the conservative-as-possible approach as I
didn't want to introduce instabilities to Python.

It certainly may be that I've overdone it. In this case I
probably have; if the tp_dealloc of the class being adjusted
is subtype_dealloc and the tp_dealloc that ultimately gets
invoked is the same we're probably OK. But I'm not sure and
it's been a while since I thought about this.

It also happens that my motivating use-case for this isn't
troubled by this restriction.

I don't understand your last, parenthetical, comment.
HEAPTYPES as such doesn't come into it, does it?

You might be right that we don't need to worry about
tp_dealloc if the ultimate solid_base doesn't change and all
the tp_deallocs on the way there are subtype_dealloc...

[glchapman]

Logged In: YES
user_id=86307

Sorry about the parenthetical comment; I think what I was trying
to say is basically what you have in your last paragraph.

As for use cases, I don't have any myself (I ran into this with
some test code for a metaclass which "overrides" __bases__).
However, grepping through the standard library, I note that one
place where assignment to __bases__ is used is in
xmlrpclib.SlowParser. It appears to me that if SlowParser and
xmllib.XMLParser (neither of which has a base class) were
converted to new-style classes, the assignment to __bases__
would generate this exception. Of course, that shouldn't be too
hard to work around if that turns out to be necessary.

[mwhudson]

Logged In: YES
user_id=6656

Are you interested in working up a patch for this? Hacking
this kind of stuff requires motivation I'm not sure I can
drum up in time for 2.3...

[glchapman]

Logged In: YES
user_id=86307

Well, I wrote a small patch which I'm attaching. However, I
can't say that I'm partcularly confident in it. It works for the
simple cases I've been able to think of (and for
test_descr.py), but looking at typeobject.c, I get the nagging
feeling that a lot more tests are required to be sure it's OK.
The problem is, I'm just not sure what they (the tests) are.

[mwhudson]

Logged In: YES
user_id=6656

Mhmm. That looks OK to me (after realizing that solid_base
worries about __slots__).

But I don't know how one can be sure :-(

[mwhudson]

Logged In: YES
user_id=6656

Two years on, I think about this again. Still here? :)

The motivating thought is that:

class A(object): pass
class B(object): pass
B.__bases__ = (A,)

and

class A(object): pass
class B(A): pass

should be equivalent.

An issue that hadn't occurred to me before is that in the first example both
A and B have a __dict__ (and __weakref__) descriptor, and in the second
B doesn't. Should B's __dict__ descriptor be removed on the
__bases__ assignment?

[glchapman]

Logged In: YES
user_id=86307

Still here -- sorry not to reply sooner. I couldn't
actually remember what my patch was supposed to do, or more
specifically I couldn't remember what it did to check that
this sort of change in __bases__ was safe. So, anyway, I
finally got around to looking at the patch again, and at
typeobject.c, and I can say that I'm less sure of the
subtleties involved now than I was then. Anyway, with that
caveat, what you suggest sounds reasonable enough, though I
suppose you'd have to reinsert a dict descriptor if
__bases__ was later changed back to (object,). (It looks
like the patch would have supported changing __bases__ back
to (object,), though perhaps it shouldn't.)

It seems to me nobody is particularly troubled by this
limitation on assignment to __bases__ (perhaps you know
differently?). Maybe it's best just to close this as "not a
bug."

[mwhudson]

Logged In: YES
user_id=6656

Well, my motivation is to better understand what's going on
in typeobject.c. There are a number of unclear sections (to
put it mildly) and undocumented assumptions and this makes
it hard to work out whether things (like this) are bugs or not.

I'm now fairly confident that the line
"compatible_for_assigment(new_base, self->tp_base)" is
asking the right question (and, thus, your patch is not
really correct, though I doubt it's actually unsafe). The
spelling is a bit funny though.

This issue in and of itself isn't that high a priority, but
understanding (and documenting that understanding) of
typeobject.c does seem worth working on...

[lorph]

Is anyone still working on this? It seems like an oddity of python that
has been a stumbling block for me to create a super reload.

I've found that i am able to bypass this problem by creating the
following definition:

class object(object):pass

However, this feels like an ugly hack.

[mwhudson]

Another 3 and a bit years on <wink> I still think my comment
http://bugs.python.org/msg14169 is the crux of the issue. It's even
relevant to your class object(object): pass hack!

I'm not at all likely to work on this any time soon myself.