Unexpected behaviour of `__subclasshook__` with `abc`

[SimonKoop]

Bug report

Bug description:

When using __subclasshook__ on a childclass of another class, this can lead to unexpected behaviour for issubclass when used on a parent class or sibling class as shown in the following two code examples:

import abc

class X(metaclass=abc.ABCMeta):
    pass

class Y(X):
    pass

class Z(X):
    pass

define_magic = True  # try toggling this value
if define_magic:
    class Magic(Y):
        @classmethod
        def __subclasshook__(cls, maybe_subclass):
            if issubclass(maybe_subclass, Z):
                return True
            return NotImplemented
    
assert issubclass(Z, Y) == define_magic

and

import abc

use_meta:bool = False  # try toggling this

if use_meta:
    class Module(metaclass=abc.ABCMeta):
        pass
else:
    class Module:
        pass 

class StatefulModule(Module):
    pass


class RegularLayer(Module):
    pass

class MaybeStatefulLayer(StatefulModule):
    pass

class SpecificRegularLayer(RegularLayer):
    pass

class SpecialModuleCollection(Module, metaclass=abc.ABCMeta):
    @classmethod
    def __subclasshook__(cls, maybe_subclass):
        if issubclass(maybe_subclass, (RegularLayer, MaybeStatefulLayer)):
            return True
        return NotImplemented
    pass
    
class SpecificSpecialModule(StatefulModule, SpecialModuleCollection):
    pass
    
assert issubclass(SpecificRegularLayer, StatefulModule) == use_meta

The code examples are based on patrick-kidger/equinox#893

CPython versions tested on:

3.10, 3.11, 3.13

Operating systems tested on:

Linux, Windows

[AlexWaygood]

Hmm, not sure I see anything to do with the typing module or typing internals here @picnixz :-)

To me, this looks like a bug report to do with the abc module

[picnixz]

Oh, I thought that typing and abc were actually tight to each others hence the label. Thanks for the clarification Alex!

[ZeroIntensity]

Isn't abc pure-Python? I would think this needs the stdlib label, there's no C code at play here.

[picnixz]

The abc module has a C implementation but also a Python implementation. I assume that the issue is with the C implementation in this case but I can add the stdlib label as well!

See: https://github.com/python/cpython/blob/main/Modules/_abc.c.

[brianschubert]

I'm not too familiar with abc internals, but after taking a quick look at the implementation, I think this is the relevant code responsible for this behavior:

cpython/Modules/_abc.c

Lines 799 to 825 in 54c63a3

	/* 6. Check if it's a subclass of a subclass (recursive). */
	subclasses = PyObject_CallMethod(self, "__subclasses__", NULL);
	if (subclasses == NULL) {
	goto end;
	}
	if (!PyList_Check(subclasses)) {
	PyErr_SetString(PyExc_TypeError, "__subclasses__() must return a list");
	goto end;
	}
	for (pos = 0; pos < PyList_GET_SIZE(subclasses); pos++) {
	PyObject *scls = PyList_GetItemRef(subclasses, pos);
	if (scls == NULL) {
	goto end;
	}
	int r = PyObject_IsSubclass(subclass, scls);
	Py_DECREF(scls);
	if (r > 0) {
	if (_add_to_weak_set(impl, &impl->_abc_cache, subclass) < 0) {
	goto end;
	}
	result = Py_True;
	goto end;
	}
	if (r < 0) {
	goto end;
	}
	}

This comes from inside _abc__abc_subclasscheck_impl, which is ultimately called from ABCMeta.__subclasscheck__.

My (naive) reading of this is that this is intentional behavior. Checking whether a class is a subclass of an ABC involves performing subclass checks against each subclass of the ABC, hence why checking issubclass(Z, Y) involves the result of issubclass(Z, Magic)

edit: this is also replicated by the Python fallback module:

cpython/Lib/_py_abc.py

Lines 140 to 144 in 54c63a3

	# Check if it's a subclass of a subclass (recursive)
	for scls in cls.__subclasses__():
	if issubclass(subclass, scls):
	cls._abc_cache.add(subclass)
	return True

[ZeroIntensity]

The abc module has a C implementation but also a Python implementation. I assume that the issue is with the C implementation in this case but I can add the stdlib label as well!

Ah, I didn't realize it had an accelerator.

[SimonKoop]

My (naive) reading of this is that this is intentional behavior. Checking whether a class is a subclass of an ABC involves performing subclass checks against each subclass of the ABC, hence why checking issubclass(Z, Y) involves the result of issubclass(Z, Magic)

You're right, in the second code example, I forgot that the __subclasshook__ would be inherited by SpecificSpecialModule resulting in SpecificRegularLayer being a subclass of SpecificSpecialModule which in turn is a subclass of StatefulModule.
My apologies for the incorrect bug report.